Skip to main content

Python Connector User Guide

Overview​

The Koverse Data Platform (KDP) Python Connector allows API access via the KDP Python Client, which includes all of the available API calls and is included when the connector is installed. Python 3.8.5+ and pip3 are prerequisites for installation of the connector.

The Python Client and Python Connector allow you to establish a connection to the KDP API and handle requests in order to upload and manipulate data in KDP via API with more ease than by using the KDP Python Client directly. All of the endpoints defined in the Open-API spec are available through the KDP Python Client. The Python Connector provides helpful methods which simplify the use of the underlying Python Client. Some examples of Python Connector methods are to convert from a KDP dataset to a Python Pandas DataFrame, or to read a batch of records from KDP. There are many more helpful methods that simplify using the endpoints available in the Python Client.

Python Client​

This client is automatically installed when the connector is installed. Prerequisites for the client and connector are the same, and dependencies are automatically installed as part of the Python Connector installation procedure.

Installation​

To install the KDP Python Connector from PyPi, run the following command:

pip install kdp-python-connector

To install only the KDP Python Client from PyPi, run the following command:

pip install kdp-api-python-client

Usage​

Using the connector, you can establish a Python-backed connection with the KDP API. This can be done directly via a Python Shell, via an application like Databricks, or another custom solution. See KDP Python Connector Examples for demonstrations of several ways the KDP Python Connector can be utilized.

Step 1​

Set System Variables to provide the values for the variables listed below:

Required​

  • EMAIL - KDP user's email address
  • PASSWORD - KDP user's password
  • WORKSPACE_ID - KDP user's workspace id
  • DATASET_ID - KDP user's dataset id

Optional​

  • KDP_URL - KDP url to connect to:

    default=https://api.app.koverse.com

  • PATH_TO_CSV_FILE - location to the csv file to be ingested:

    default=['https://kdp4.s3-us-east-2.amazonaws.com/test-data/cars.csv']

  • STARTING_RECORD_ID - record to start reading:

    default=''

  • PATH_TO_CA_FILE - when not provided will not verify ssl of request:

    default=''

  • INPUT_FILE - file with data to ingest:

    default='../datafiles/actorfilms.csv'

  • BATCH_SIZE - number of records in a batch:

    default=100000

Step 2​

Execute your code:

python3 <filename>