feat(databricks): add Databricks SQL adapter - #241
Merged
Conversation
Three-level Unity Catalog (catalog.schema.table) via databricks-sql-connector. Supports PAT, OAuth U2M (browser), and OAuth M2M (service principal) auth.
Owner
Author
|
Anyone using databricks: feel free to test this PR. |
The information_schema filter used `table_type IN ('MANAGED', 'EXTERNAL',
'BASE TABLE')`. 'BASE TABLE' is not a Unity Catalog table_type at all, and
the include-list hid FOREIGN (Lakehouse Federation), STREAMING_TABLE,
MANAGED_SHALLOW_CLONE and EXTERNAL_SHALLOW_CLONE from the explorer tree.
Invert it: tables are everything that is not VIEW or MATERIALIZED_VIEW, so
table types Databricks adds later show up without another code change.
get_views now reads the same information_schema.tables column instead of
information_schema.views, which keeps the two lists complementary and picks
up materialized views regardless of whether they appear in the views table.
Claude-Session: https://claude.ai/code/session_01S6TsbrUgqAv3UETAfkg1ip
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Databricks SQL support with Unity Catalog browsing, legacy Hive-metastore metadata, PAT authentication, browser OAuth, and service-principal OAuth.
Provider token fields use sqlit's existing protected credential slot. Saved connections can be reopened and renamed without putting tokens in connection JSON or driver-restart caches. Legacy plaintext token fields migrate into the credential backend, and changing authentication mode clears an incompatible old secret. Browser OAuth avoids database-password prompts. Connection URL parameters are normalized before validation, and the Databricks extra installs the OAuth SDK.
Verification at
93d3f78d53e3c8914725e20796204a2d223e8b9c:See
tests/integration/test_cloud_provider_credentials.pyandCONTRIBUTING.mdfor the reproducible cloud test. No cloud secrets are required by ordinary CI.