fix(ci): gate required-context drift instead of discovering it on a stuck PR

[avrabe]

Closes #372.

The failure

Branch protection's required-check list and the workflows that produce those
checks live in different places and drift apart with nothing watching. Three
distinct drifts, three different failures:

drift	symptom
a job runs on every PR and is not required	it enforces nothing — osate-corpus was this shape
a required context names a job that no longer exists, or one behind a workflow-level path filter	the PR sits at "Expected — waiting for status" forever. Unmergeable, with nothing red to point at
a continue-on-error job is required	the tick is green whatever happened; requiring it is decoration

The distinction the gate turns on

This is the part that is invisible in workflow YAML unless you go looking for
it, and getting it backwards is what produces the unmergeable PR:

• A job skipped by a job-level if: reports skipped — which
  satisfies a required context.
• A job in a workflow that never triggered (workflow-level
  on.pull_request.paths) reports nothing at all.

So job-level filtering is gateable and workflow-level filtering is not. A
mechanical "every job must be required" rule gets this exactly wrong, and
would have deadlocked every PR that misses those paths.

tools/check_required_contexts.py therefore sorts every PR-triggered job into
GATEABLE / ADVISORY / UNDELIVERABLE from the YAML alone, and asserts
.github/required-contexts.txt is exactly the GATEABLE set.

Evidence it models reality, not just itself

--print emits the GATEABLE set derived purely from workflow YAML. It is
byte-identical to the 18 contexts branch protection actually enforces today
(diff → no output). The classifier and GitHub agree on the live repo.

On its first live run it flagged exactly two violations and correctly did
not flag osate-corpus — the deadlock trap avoided on the real repo, not
only in a fixture.

The two fixes it demanded

bazel-test and kani are both continue-on-error but named as if they were
verdicts. "Kani Bounded Model Checking ✓" is precisely the claim a reader
should not draw from a tick that is green regardless of outcome. Both now carry
(advisory) in the name — the only place a PR reader sees it. Neither is a
required context, so the rename cannot strand one (guard 2 would fail if it
could).

Two design constraints, recorded so they aren't re-litigated

Why a committed list rather than the live API. Reading branch protection
needs an admin token; the Actions GITHUB_TOKEN is not one. --against-api
exists for a human with credentials.

Why steps of rivet-validate rather than a job of its own. A new job would
itself be GATEABLE, so it would have to be added to the required list — and
syncing branch protection needs that same admin token. It would ship
permanently red.

Stdlib-only, fail-closed

Following the constraint already documented above the human-scoped guardrail:
no workflow here installs a Python package, so importing PyYAML would make a
required check depend on an unverified runner package — and a guardrail that
cannot run reads as approval. The reader is fail-closed (unrecognised syntax at
the depths it reads exits 2, rather than scanning past it), narrowed to fire
only where a misread would change a verdict. --cross-check validates it
against PyYAML on the real corpus wherever PyYAML exists: agrees on every
field it reads, across all 10 workflows.

Self-tests

11 cases, each recording the failure it proves is still caught — including
both halves of the path-filter trap: required-and-undeliverable must fail,
and not-required-and-undeliverable must pass.

11 self-test(s) passed.
scanned 21 PR-triggered job(s): 18 gateable, 2 advisory, 1 undeliverable; 18 required context(s)
PASS: the committed required set is exactly the gateable set.

Scope

CI plumbing, no rivet artifact — matching the #353/#363/#364/#366 precedent;
this repo carries no REQ-CI-* ids. Flagged so the omission reads as a choice.

🤖 Generated with Claude Code

[avrabe]

Clean-room audit: 3 of 5 claims about this gate were refuted

I asked a fresh-context subagent to refute this PR's claims, not confirm
them. It confirmed B3 (the derived set matches live branch protection exactly,
18 contexts, verified with --against-api), B4 (the gate runs on the PR that
edits a workflow — checked against real runs #359 and #367), and B5 — the
premise the whole design turns on, with direct evidence from this repo's own
history rather than documentation:

• if:-skipped jobs DO report skipped, and that satisfies a required check.
  PR chore(artifacts): tag the three ordeal artifacts human-scoped #367 touched only artifacts/requirements.yaml; ten required contexts
  reported skipped — and it merged (2b11af9).
• A path-filtered workflow reports NOTHING. aadl-interop.yml produced an
  OSATE check-run on fix(parser): reject C-style radix literals (0x/0b/0o) instead of silently parsing to 0 (#337) #343 and feat(codegen): emit typed WIT list from Array data types (#345) #346 (paths matched) and no check-run at all
  on feat(codegen): emit target constants module + linker memory map (#338) #349 (paths didn't). Absent, not skipped.

The rest were wrong. Fixed in 00393b2:

	defect	why it mattered
B2a	pull_request: {branches: [main], paths: [...]} read as an opaque string	paths evaporated → job ruled GATEABLE → the gate emitted an ::error:: ordering you to require an undeliverable context. It handed you the exact deadlock it exists to prevent.
B2b	four-space indentation → every job at indent 4, invisible	zero jobs → zero violations → PASS. Fail-open, which the docstring asserted could not happen.
B2c/d	block scalars and anchors in name:	context read as >- / *n; the real context goes ungated
—	glob was *.yml	Actions loads .yaml too — a sneaky.yaml was invisible and the gate reported PASS on a directory it never fully read
B1a	self-test 9 was provably vacuous	the reader never reads if:, so its fixture parses byte-identically to the clean one; no mutant can distinguish them. It claimed to prove "the distinction the whole gate turns on" — a claim about GitHub runtime behaviour no YAML fixture can establish. The real evidence is PR #367 above, now cited in the docstring.
B1b	the paths-ignore guard had no test	it survived the entire suite — deleting it reddened nothing, refuting the comment "each fixture kills exactly one guard"

--cross-check catches all four parser bugs, but it needs PyYAML and so
deliberately does not run in CI — the one place this gate is load-bearing. So
safety there can't rest on agreeing with a reference parser; it rests on the
reader refusing what it cannot represent. All four now raise.

Docstring correction. osate-corpus was cited as a live instance of failure
mode 1. It has the symptom, but aadl-interop.yml carries a workflow-level
paths filter, so this gate buckets it UNDELIVERABLE, rules it must not be
required, and reports PASS. The flagship motivating example is one the gate
declines to fix. Now said plainly, with the remedy (move the filter to a
job-level if:).

Verification. 15 self-tests. Each new guard mutation-tested — no-op it and a
named fixture goes red, including WORKFLOW_GLOB itself (the fixtures build
their own glob and cannot see it, so that one is asserted directly). Live corpus
unchanged: 21 PR-triggered jobs → 18 gateable / 2 advisory / 1 undeliverable;
--cross-check still agrees with PyYAML across 10 workflows; --against-api
still matches branch protection exactly.

Not claimed: that the reader now handles arbitrary YAML. It handles less than
before — it refuses more.

🤖 Generated with Claude Code

[github-actions]

Rivet verification gate

✅ 20/20 passed

	count
Passed	20
Failed	0
Skipped (no steps)	0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

_{Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!