rfc: add RFC 0046, read-only index status

[azimafroozeh]

What & why

This PR adds RFC 0042: a read-only index-status surface. omnigraph index status and GET /index-status report, per index, a state (ready, unbuildable, missing, degraded) with a machine-readable reason and coverage counts, derived from committed metadata under snapshot isolation; omnigraph index await blocks until the gating states clear. Requested in #550: the only structured index readout today is pending_indexes inside optimize --json, so observing index state requires a command that also commits, a command-query separation violation this surface removes.

• The derivation reuses the checks optimize already runs (the catalog-intent diff and fragment-bitmap coverage), exposed through one read-only entry point, so endpoint and reconciler cannot disagree on their inputs.
• degraded carries an open reason vocabulary, giving Index coverage reports a degenerate mono-partition vector index as fully covered — the failure is invisible to monitoring #486's degenerate-index detection a reporting surface with no schema change.
• Registry updates: the index row for 0042 and the next-available bump to 0043 (0040 stays reserved for rfc: add RFC 0040, the system column namespace #546).

Backing issue / RFC

• Add read-only index readiness status and a blocking wait command #550 (the feature request this designs); adds docs/rfcs/0042-index-status.md. Related: Index coverage reports a degenerate mono-partition vector index as fully covered — the failure is invisible to monitoring #486 (the degraded reporting hook); distinct from performance: traversal mode selection does not adapt as a multi-hop frontier grows #533 (planner behavior when indexes exist).

Checklist

• Change is focused (one RFC document plus its two registry lines in docs/rfcs/README.md)
• Tests added/updated for behavior changes (design record only; the RFC's Evidence and tests section defines the implementation's test obligations)
• Public docs updated if user-facing surface changed (the RFC is the document; this PR ships no behavior)
• Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit (the RFC's Invariants section addresses the two nearest items: derived-not-stored work lists, and why await is not a coverage-based engine precondition)

Local verification

• python3 scripts/check-docs.py — Documentation OK (110 Markdown files checked)
• cargo test — not run: docs-only change, no code touched

Notes for reviewers

• The state for declared-but-unbuildable indexes is named unbuildable, and pending is deliberately reserved: an in-progress reading is unrepresentable on a committed-state-only surface, so the word stays free for a future background builder (note after the states table).
• await gates on missing and degraded only and never triggers reconciliation; rollout phase 3 (await) is independently droppable if review reads the deny-list differently.
• RFC number 0042 taken per the registry's next-available line at this base; re-checked for races before opening.

Greptile Summary

The PR adds RFC 0046, specifying a read-only index-status API and CLI surface derived from committed metadata.

• Defines index states, reason tokens, coverage counts, and snapshot-isolated reporting semantics.
• Specifies polling behavior for index await and a repair path for legacy indexes without coverage metadata.
• Registers RFC 0046, reserves RFC 0045, and advances the next available number to 0047.

Confidence Score: 5/5

The documentation-only PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
docs/rfcs/0046-index-status.md	Adds the read-only index-status RFC and consistently identifies it as RFC 0046, resolving the prior numbering issue.
docs/rfcs/README.md	Registers RFC 0046, documents the RFC 0045 reservation, and advances the next available number to 0047.

_{Reviews (5): Last reviewed commit: "review" | Re-trigger Greptile}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[gordonmurray]

Thanks for writing this RFC. I had two questions about the proposed status contract:

1. Could each record include the indexed field or fields? For example, two missing FTS indexes on different fields can otherwise produce indistinguishable records when name is null.
2. Lance defines fragment_bitmap: None as unknown coverage. Could treating it as covered cause the command to report ready without proof that all fragments are indexed?

I also noticed that await does not gate on unbuildable. Is the intended guarantee that all buildable indexes are ready, rather than all selected indexes?

[azimafroozeh]

Thanks Gordon, both are right. RFC updated:

Could each record include the indexed field or fields? For example, two missing FTS indexes on different fields can otherwise produce indistinguishable records when name is null.

Yes. Every row now carries a fields array, and (table_key, kind, fields) is the row's identity (rule 10), since name is null before the first build.

Lance defines fragment_bitmap: None as unknown coverage. Could treating it as covered cause the command to report ready without proof that all fragments are indexed?

Right. A None fragment_bitmap now classifies as degraded / coverage_unknown, and optimize repairs it with a one-time full rebuild that writes a real bitmap. What it can't rebuild keeps coverage_unknown and never blocks.

I also noticed that await does not gate on unbuildable. Is the intended guarantee that all buildable indexes are ready, rather than all selected indexes?

Yes, and it's now stated explicitly: exit 0 means every declared index is ready or unbuildable. Waiting can't build an unbuildable index, so it's reported instead of gated.

Does the coverage_unknown handling look right to you?

[aaltshuler]

Heads-up: RFC number 0042 is now taken — the branch-incarnation RFC landed with #579 (docs/rfcs/0042-incarnation-suffixed-branch-refs.md), and the registry's next available number is 0043. This PR's 0042-index-status.md will need renumbering to 0043 (filename, frontmatter, heading, registry row) on its next rebase.