Skip to content

feat(schema): resolve system column spellings per graph - #548

Draft
azimafroozeh wants to merge 2 commits into
ModernRelay:mainfrom
azimafroozeh:system-columns-underscore-rename
Draft

feat(schema): resolve system column spellings per graph#548
azimafroozeh wants to merge 2 commits into
ModernRelay:mainfrom
azimafroozeh:system-columns-underscore-rename

Conversation

@azimafroozeh

Copy link
Copy Markdown
Contributor

What & why

This PR implements the coexistence layer of RFC 0040 (#546): the engine stops assuming the spellings of its implicit stored columns and resolves them per graph, keyed on the graph's schema IR version. Motivated by #529, whose companion reservation patch this design supersedes.

  • Newly created graphs record IR version 3 and spell the implicit columns __id, __src, __dst; every existing graph keeps version 2 and its current spellings, for every table it has and every table it later gains, with no migration anywhere.
  • Ordinary schema applies re-emit the accepted version, so no unrelated apply can change a graph's spellings; migration planning refuses cross-version evolution.
  • The catalog carries the one resolution point (Catalog::system_columns); roughly one hundred hardcoded "id"/"src"/"dst" occurrences across the engine, compiler lowering, CLI, and server now route through it.
  • Property names starting with _ are rejected at new schema admission, subsuming the per-name Lance reservation list; on new graphs id, src, and dst become ordinary property names.
  • Admission is vintage-keyed: evolving a legacy graph keeps its historical name rules (a legally persisted _row_id can still be restated), still rejects the exact Lance virtual names and the legacy spellings, and additionally reserves the __ spellings the future upgrade must claim.
  • Hand-authored schema IR is refused fail-closed for shapes source admission never produces: edge key/range/check constraints, and range/check constraints referencing system columns.
  • GET /schema gains an optional system_columns field (also printed by omnigraph schema show) so multi-graph clients discover each graph's spellings instead of guessing; payloads carry each graph's own column names.
  • A failpoints-gated fixture creates a genuine legacy-vintage graph and exercises it end to end: load, traversal, keyed mutation, cascade delete, evolution with the vintage preserved, refused claims of the freed and upgrade-target names, and reopen.

Backing issue / RFC

Checklist

  • Change is focused (one concern: per-graph system column spelling resolution and its reservation rules)
  • Tests added/updated for behavior changes (vintage minting and carry-forward, per-vintage reservation rules, upgrade-target reservation, IR-shape refusals, the legacy-vintage end-to-end fixture, and the integration suites now exercise the new spellings throughout)
  • Public docs updated if user-facing surface changed (docs/user/schema/index.md documents the reserved namespace, per-graph spellings, and the discovery field)
  • Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit (the spelling becomes per-graph logical state resolved from the schema authority, never inspected from storage)

Local verification

  • cargo test --workspace --no-fail-fast — [landing; on-track: only the known sandbox blob failure]
  • cargo test --workspace --no-fail-fast --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints — [landing; this graph exists locally because review caught a failpoints-only break invisible to default features]
  • cargo test -p omnigraph-engine --test legacy_columns --features failpoints — green
  • cargo clippy --workspace --all-targets — zero diagnostics
  • cargo fmt --all --check — clean
  • vocabulary guard, all three surfaces vs merge base — clean (inventory rows for new and re-keyed occurrences included)
  • OpenAPI drift — regenerated, drift test green

Notes for reviewers

  • No flag day: existing graphs, stored queries, and clients see no behavior change; new graphs speak the new names from creation. The upgrade that moves an old graph across is specified in RFC 0040 and not part of this PR; until it lands, declaring id/src/dst on a legacy graph fails with an error naming the upgrade, and the __ spellings stay reserved on legacy graphs so the upgrade's rename targets remain free.
  • Error-message text changes where messages name a column: they now print the graph's actual spelling; BTREE index names derive from spellings (__id_idx on new graphs).
  • SchemaOutput.system_columns is additive and optional; older clients ignore it, newer clients tolerate its absence.
  • Wire formats deliberately unchanged: change-feed from/to envelopes and the JSONL edge envelope keep their fixed keys; export JSONL id keys follow the graph's spelling so export/import round-trips are internally consistent.
  • The vintage-override init used by the legacy fixture is #[cfg(feature = "failpoints")] only and re-validates the flipped IR before any write.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant