Databricks Unity Catalog Metric View converter

[jackstein21]

Summary

Adds a bidirectional Python converter between OSI YAML and Databricks Unity Catalog Metric View YAML (v1.1). This fills the Databricks spoke in the hub-and-spoke converter architecture — DATABRICKS is listed as a supported vendor in the spec and converters/index.md but had no converter implementation until now.

What's included

• Import (Metric View → OSI): Parses Metric View YAML and produces a valid OSI document
• Export (OSI → Metric View): Generates deployable Metric View YAML from OSI semantic models
• CLI: osi-databricks import / osi-databricks export entry points
• Tests: Unit tests + Hypothesis property-based tests for round-trip fidelity
• Documentation: README with mapping table, usage examples, and known limitations

Mapping highlights

Metric View	OSI
fields[].expr	field.expression.dialects[DATABRICKS]
measures[].expr	metric.expression.dialects[DATABRICKS]
joins[].on	relationship.from_columns / to_columns
synonyms, display_name	ai_context.synonyms
comment	description
filter, materialization, window, format	custom_extensions (vendor: DATABRICKS)

Dialect selection follows the converter guide: prefer DATABRICKS, fall back to ANSI_SQL.

Conventions followed

• Python package at converters/databricks/ with src/osi_databricks/ layout
• Hatchling build backend, matching existing Python converters (GoodData, dbt)
• Depends on osi-python>=0.2.0.dev0 and PyYAML>=6.0
• Tests use TPC-DS-based fixtures as baseline
• Round-trip fidelity for both directions (MV→OSI→MV and OSI→MV→OSI)

Testing

cd converters/databricks
uv sync
uv run pytest

All tests pass locally on Python 3.14.

Notes
This is a tooling/converter contribution, not a spec change
The .gitignore commit adds standard entries (build artifacts, Hypothesis DB, IDE files) that benefit all contributors
Metric View materialization support is included but marked as preview (Databricks docs note it's in preview)

[Haoranli503]

Hi, we're currently working on a Databricks converter internally, which should be available later this week. We really appreciate the effort, but we'd prefer to use our version of the converter in OSI so it's easier for us to manage and update over time.

[jbonofre]

@Haoranli503 maybe we can collaborate all together on a converter?

[Haoranli503]

Yes, let me go through this, seems we are using similar approach.

[Haoranli503]

Hi @jackstein21

I scanned over this PR. Overall very well done. Thanks for this implementation! We also built the same bidirectional converter and have been validating it against the live Metric View engine, so sharing a quick comparison here to help converge on one implementation.

What this PR does well:

• Hypothesis property-based round-trip tests (per-field invariants, ~100 examples each) - this is the standout. None of the existing spokes have it, and we'd like to adopt the same approach.

Something that this PR doesn't have:

The converter assumes a denormalized OSI model: all fields flattened onto a single fact dataset (as in osi_tpcds.yaml), with dimensions present only as relationship targets. On a normalized model - each dimension its own dataset with its own fields, the common OSI shape - this leads to:

1. One MV per dataset, measures duplicated onto each (has_metrics = len(metrics) > 0):
   a star schema becomes N views, and the dimension views carry measures that cannot resolve.
2. No nested joins on export: a snowflake A -> B -> C drops C from A's view.
3. Joined columns emitted unqualified (item_category: i_category) - a real MV
   needs item.i_category to resolve.
4. Import builds a single dataset, so relationships point at to-datasets that are
   never defined (dangling targets).
5. Cardinality / one_to_many only via custom_extensions - not derived from
   from/to direction.
6. ON parsing assumes the left operand is the source side; a reversed condition mis-maps.
7. is_time inferred from text + ANSI_SQL auto-added - so OSI -> MV -> OSI is not identity.

We have all of the above working and engine-validated: a single fact + join tree with a grain selector, diamond fan-out for shared dimensions, alias requalification, cardinality/one_to_many derivation, the primary-key <-> at_most_one_match bridge, and exact-fidelity round-trip via the DATABRICKS stash.

The other parts are very similar.

For maintainability, we'd like to land one final version in OSI. Would you be open to a new PR that combines your property-test work with our converter and adds you as a co-contributor? Happy to work and discuss further on Slack. I am Evan on the OSI slack channel.

[jackstein21]

@Haoranli503 thanks for the detailed feedback on this. Extremely happy to hear that there is an internal effort to bridge the Databricks/OSI gap. I am completely open to new PR that takes my property tests and combines with your converters. Would love the co-contributor shout out too.

[Haoranli503]

@jackstein21 thank you for your support! We are finalizing everything and should have the pr this week.