- Notifications
You must be signed in to change notification settings - Fork126
Description
I am developing a Streamlit app for deployment to Databricks Apps. I am having trouble getting the same authentication flow to work both locally as well as on Databricks Apps.
Consider the following snippet:
fromdatabricksimportsqlwithsql.connect(server_hostname=host,http_path=http_path,catalog=catalog,)asconnection:withconnection.cursor()ascursor:cursor.execute("SELECT 1")result=cursor.fetchall()
This works fine locally.
However, when I deploy this to Databricks Apps, it just hangs forever.
I figured out that to get this snippet to work on Databricks Apps, it needs this tweak:
fromdatabricks.sdk.coreimportConfigwithsql.connect(server_hostname=host,http_path=http_path,catalog=catalog,credentials_provider=lambda:Config().authenticate,)asconnection: ...
This works on Databricks Apps, but now when I try to run it locally I get a new error:
ValueError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authenticationto configure credentials for your preferred authentication method.OK great. I have one thing that works locally but not on Databricks Apps, and another that works the other way around.
Usingthe tip shared here, I combined these into a hybrid approach that seems to work both locally and on Databricks Apps:
fromdatabricks.sdk.coreimportConfig,databricks_clidefuniversal_databricks_credentials_provider():try:returndatabricks_cli(Config(host=host))except:returnConfig().authenticatewithsql.connect(server_hostname=host,http_path=http_path,catalog=catalog,credentials_provider=universal_databricks_credentials_provider,)asconnection: ...
But this is all very awkward and unpleasant.
I think the default authentication chain that this library uses should work on Databricks Apps so that I don't have to specifycredentials_provider.