Repository files navigation

The Databricks SQL Connector for Python allows you to develop Python applications that connect to Databricks clusters and SQL warehouses. It is a Thrift-based client with no dependencies on ODBC or JDBC. It conforms to thePython DB API 2.0 specification.

This connector uses Arrow as the data-exchange format, and supports APIs (e.g.fetchmany_arrow) to directly fetch Arrow tables. Arrow tables are wrapped in theArrowQueue class to provide a natural API to get several rows at a time.PyArrow is required to enable this and use these APIs, you can install it viapip install pyarrow orpip install databricks-sql-connector[pyarrow].

The connector includes built-in support for HTTP/HTTPS proxy servers with multiple authentication methods including basic authentication and Kerberos/Negotiate authentication. Seedocs/proxy.md andexamples/proxy_authentication.py for details.

You are welcome to file an issue here for general use cases. You can also contact Databricks Supporthere.

Python 3.8 or above is required.

For the latest documentation, see

• Databricks
• Azure Databricks

Install usingpip install databricks-sql-connector

Install usingpip install databricks-sql-connector[pyarrow]

export DATABRICKS_HOST=********.databricks.comexport DATABRICKS_HTTP_PATH=/sql/1.0/endpoints/****************

Example usage:

importosfromdatabricksimportsqlhost=os.getenv("DATABRICKS_HOST")http_path=os.getenv("DATABRICKS_HTTP_PATH")connection=sql.connect(server_hostname=host,http_path=http_path)cursor=connection.cursor()cursor.execute('SELECT :param `p`, * FROM RANGE(10)', {"param":"foo"})result=cursor.fetchall()forrowinresult:print(row)cursor.close()connection.close()

In the above example:

• server-hostname is the Databricks instance host name.
• http-path is the HTTP Path either to a Databricks SQL endpoint (e.g. /sql/1.0/endpoints/1234567890abcdef),or to a Databricks Runtime interactive cluster (e.g. /sql/protocolv1/o/1234567890123456/1234-123456-slid123)

Note: This example usesDatabricks OAuth U2Mto authenticate the target Databricks user account and needs to open the browser for authentication. So itcan only run on the user's machine.

Starting fromdatabricks-sql-connector version 4.0.0 SQLAlchemy support has been extracted to a new librarydatabricks-sqlalchemy.

• Github repositorydatabricks-sqlalchemy github
• PyPIdatabricks-sqlalchemy pypi

Users can now choose between using the SQLAlchemy v1 or SQLAlchemy v2 dialects with the connector core

• Install the latest SQLAlchemy v1 usingpip install databricks-sqlalchemy~=1.0
• Install SQLAlchemy v2 usingpip install databricks-sqlalchemy

SeeCONTRIBUTING.md

Apache License 2.0