ci: six reviewed guardrail PRs as one branch, plus two gate repairs (#381, #383) - #387
Open
avrabe wants to merge 13 commits into
Open
ci: six reviewed guardrail PRs as one branch, plus two gate repairs (#381, #383)#387avrabe wants to merge 13 commits into
avrabe wants to merge 13 commits into
Conversation
#361) `fuzz_scheduler_solver` has ended in `libFuzzer: out-of-memory` on every nightly run the API still returns — 97 of 97, 2026-04-25 through 2026-07-30. Nothing surfaced it because fuzz-nightly is not a required context, so a workflow that has never once passed sat red for three months as a gate nobody reads. It is not a leak. ASan's own accounting at the moment of death (run 30516991027) rules the code under test out: used: 2056Mb; limit: 2048Mb Live Heap: 27.7 MB in 8,340 chunks quarantined: 155.7 MB in 2,104,213 chunks total chunks: 4,345,508 Live plus quarantined *contents* account for 183 MB of 2056 MB. The remaining ~1.87 GB is ASan's per-chunk bookkeeping spread over 4.3M chunks, which is a product of two terms, so the fix addresses both: * `quarantine_size_mb=64` (from 256) bounds the chunk count. The 2.1M quarantined chunks are retained only to catch use-after-free, which is not the bug class this harness hunts — it asserts `solve_milp` never panics. * `malloc_context_size=10` (from 30) bounds the per-chunk cost. Ten frames still names the allocation site, so crash triage is intact. `-max_len=128` is applied to this leg ONLY, via a matrix `include`. The cap is lossless rather than a coverage trade: the harness opens with `.min(8)` / `.min(4)` on the two Vecs while `Arbitrary` grows them to consume whatever buffer it is handed, so bytes past the ~120 the capped domain can encode yield `Task` values that are allocated and discarded unread. Keeping it per-leg is the point. `fuzz_aadl_parse` and `fuzz_codegen_roundtrip` concluded `success` in all 97 runs and parse arbitrarily long source text; a shared `-max_len` on the existing single run line would have shrunk two healthy fuzzers' input space to fix a third's bug. `-rss_limit_mb=4096` is headroom, not the mechanism — the measures above should land well under even the 2048 default. The oracle is two-sided and is the nightly itself: scheduler_solver must flip failure -> success AND the other two legs must stay success. A green run where a parser leg regressed is not a fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 5199cec)
…EQ (#360) `rivet validate` reported 466 errors on a clean tree. 190 of them were phantom: every id in four STPA files collided with ITSELF, producing "declared more than once" messages with the same path on both sides. Cause: rivet.yaml listed `safety/stpa` as a directory scan AND listed four files inside it individually (to force `format: generic-yaml`). Each file was loaded twice, so every id it declared was declared twice. Listing a directory and its own contents is not additive — it is duplicative, and the diagnostic could not say so because both copies had identical provenance. Removing the four explicit entries is safe, and the artifact set was compared rather than assumed: 883 ids and 1195 links, byte-identical before and after via `rivet list --format json`. The format change loses nothing; stpa-yaml is a superset of what generic-yaml extracted from these files. That left 6 real duplicates: RENDER-REQ-001..006 were defined in BOTH artifacts/requirements.yaml and safety/stpa/rendering-analysis.yaml. rivet kept one copy and dropped the other with no diagnostic at all, and which one won was load order rather than a decision. The stubs are deleted, not the canonical copy, because only the STPA file carries the hazards/satisfies/verified-by links that make these part of the STPA V. The stubs carried none. The stubs were also stale in BOTH directions, which is why this was checked against source instead of picking the newer-looking file: - RENDER-REQ-006 claimed `partial`; it is `implemented`. etch/src/html.rs emits `svg.zoom-low` / `svg.zoom-overview` rules and asserts a `wheel` handler (html.rs:65-68, :131). - RENDER-REQ-003 claimed `implemented`; it is `partial`. `minimap` is still `/// Show minimap (Phase 3b - reserved)`. Verified at the pinned etch rev 4c06709, not by grepping this repo. A repo-scoped grep says "zoom-low appears only in a test fixture", which reads exactly like "not implemented" when the implementation lives in a git dependency. That nearly produced a false finding in the other direction. Also normalises the status vocabulary: `pass` -> `passing` (5) and `pending` -> `planned` (4), collapsing singletons into the dominant spelling. MEASUREMENTS, and one correction to the issue --------------------------------------------- 466 errors -> 282 (drop the source overlap: 190 duplicates -> 6) -> 276 (delete the 6 RENDER-REQ stubs: 6 -> 0) -> 276 (normalise the vocabulary: NO CHANGE) Duplicate-id errors are now 0. But #360 predicted that with the overlap and the vocabulary both fixed "the residual error count should be near zero", and that is FALSE. The normalisation cannot reduce the count, because its targets are not in the allowed set either: allowed: draft, proposed, approved, implemented, verified, released, accepted, deprecated, rejected `passing` and `planned` are both absent, so `pass` -> `passing` moves 5 errors between buckets (172 -> 177) and `pending` -> `planned` moves 4 (36 -> 40). Net zero, as measured. It is still worth doing: it leaves 4 non-conforming values to resolve instead of 6. It is just not an error fix, and recording it as one would have overstated what this change buys. The residual 276 are entirely status-vocabulary skew: 177 `passing`, 51 `not-implemented`, 40 `planned`, 8 `partial`. These are test/verification states, not approval-lifecycle states, and `common@0.3.0` is embedded in the rivet binary -- schemas/ only overrides sysml2 -- so the repo cannot widen them locally. This does NOT unblock lifting the v0.4.3 pin in ci.yml; see #360 for the measurement. The pin is load-bearing for the green check, which is a fact worth stating plainly rather than leaving implicit in a comment. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 7547c0a)
… test-names The equivalence check this PR cited was measured on `rivet list --format json` WITHOUT `--full`, which emits only the summary view (`--help` states it omits `description`, `tags`, and `fields`). That view is structurally incapable of detecting field-level loss — precisely the class of change a `format:` switch produces. "Byte-identical" was true and meaningless. Re-measured with `--full` on base vs. this branch: +86 artifacts GAIN `mitigates` and `traces-to` -5 artifacts LOSE half of `test-name` The five are VAL-010, VAL-011, VAL-014, VAL-015, VAL-021 — exactly the five whose on-disk `test-name` is a comma-separated pair. The stpa-yaml parser keeps only the first element: on disk: tests::boolean_matches_aadl_boolean, tests::boolean_mismatches_aadl_string rivet: tests::boolean_matches_aadl_boolean A/B on identical bytes with only `format:` differing confirms the parser is the cause: generic-yaml round-trips the full string. The trade is still worth taking — five truncated evidence pointers against 190 phantom duplicate-id errors — but it is a trade, not a free win, and the comment now says so. Filed upstream as pulseengine/rivet#747; revert the trade if that lands a fix. No behaviour change: `rivet validate` is FAIL (276 errors, 629 warnings) before and after this commit. Comment-only. Found by cold-context adversarial verification of this PR's own claims. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 3c9ea9d)
spar carried a private status vocabulary — `passing`, `planned`,
`not-implemented`, `partial` — on 276 artifacts. None of the four is in the
schema's allowed set, so every one was a `status-allowed-values` error and
`rivet validate` could not run clean.
The mapping, and what it costs:
passing -> implemented (177)
planned -> proposed
not-implemented -> proposed
partial -> approved (8, lossy)
`passing` was the largest group and the one worth naming. It is a verification
VERDICT sitting in a lifecycle field, and a hand-edited YAML file cannot
honestly assert that a test passes — only that the measure exists. That is what
`implemented` says.
Only `partial` loses information: it sits strictly between `approved` and
`implemented` and the vocabulary has no such rung. `approved` is chosen as the
conservative floor — the strongest claim certainly true — over `implemented`,
which would overstate. All eight say so in their own description. Two
(RENDER-REQ-003, COVERAGE-GAPS) already itemized their residue and were left
alone; the other five state the residue is NOT itemized. That wording is
deliberate — an explicit "not measured" is falsifiable, an invented figure
("32 of 50 rules") is not, and inventing one is the exact failure #294 was
reopened for.
No new guard ships with this. rivet already enforces `status-allowed-values` at
ERROR severity and `rivet validate` is already required, so the regression is
blocked the moment this lands; a bespoke checker would only add a weaker second
oracle beside a working one. That is the opposite of REQ-GUARD-RELEASE-PLANE-001,
which needed a tool precisely because rivet was blind to the defect.
Verification, on three planes that agree:
- the migration asserts per file that the artifact-id set is unchanged, that
every record is byte-identical outside `status`, and that the text-rewrite
count equals the semantic-move count. That last one matters: "N lines
rewritten" is equally consistent with having rewritten N *prose* lines inside
`description:` blocks while leaving N real statuses alone.
- `rivet validate` exits 0 with 0 error-severity diagnostics under
common@0.3.0 (embedded), down from 276.
- the gate is live, not vacuous: re-introducing one off-vocabulary status
reproduces exactly one `status-allowed-values` error, and restoring it
returns exit 0 with a byte-identical file. Distinct inputs, distinct
verdicts — the property #327's withdrawn certificate did not have.
Reconciliation of the raw text count against rivet's: 303 `status:` matches
across 9 committed files, minus 21 in safety/requirements.yaml (which rivet
never loads — it is under no path in rivet.yaml's `sources:`), minus 6 in
safety/stpa/architecture.yaml (nested in the `fields:` extension bag, where
rivet cannot see them) = 276. Both remainders are left in place and named in
REQ-GUARD-STATUS-VOCAB-001 rather than silently absorbed; the second is the #370
defect one field over, on `status:` instead of `release:`.
Addresses #371.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit a3512eb)
…rk (#366) Two releases shipped short and nothing noticed. v0.31.0 was cut 5m06s before a parser fix (9557dd0, "reject C-style radix literals") merged to main; v0.33.0 excluded two commits the same way. Both had `check-versions` green. `check-versions` cannot catch this, and not by oversight — it measures the version *window*. Cargo.toml reads X.Y.Z from the bump commit until the next bump, and that window spans both the commits a tag included and the ones it left out, so the invariant is circular with respect to the question. The first gate I designed was also wrong, and issue #366 records it so it is not re-proposed: asking "which release does this commit belong to" makes the verdict a function of what has merged since, and a gate whose answer drifts as unrelated commits land is not a gate. So this asks the question that was actually at stake — is anything already finished being left out, and did you say so? Given tag vX.Y.Z at commit C with tag date T: skip if C is not an ancestor of origin/main (release-branch tag); otherwise enumerate C..origin/main restricted to --before=T; empty passes; non-empty passes if the tag message states a tag point, and fails otherwise, listing what was dropped. The --before=T restriction is what makes it both race-proof (a merge landing seconds after the tag push postdates T and cannot fail the job) and idempotent (both inputs are immutable, so a re-run months later returns the same verdict). Backtested over all 42 v0.* tags by running this script, not a transcription of it: 39 clean passes, 1 acknowledged pass (v0.34.0, whose message already states its tag point), 2 fails — v0.31.0 and v0.33.0, exactly the two confirmed bad ones. No false positives across 40 good tags. Mutation-tested for non-vacuity: two scratch tags on the identical commit with the identical 8-commit excluded set, differing only in whether the message says "tag point", produce PASS and FAIL. The acknowledgement branch is load-bearing, not incidental. It lives in tools/ci/ rather than inline in the YAML specifically so the backtest exercises the shipped artifact. A backtest of transcribed shell proves the transcription works. Wired into create-release's `needs`, so it gates the publish rather than the builds: a red verdict stops everything outward-facing (publish-vsix already needs create-release) while the maintainer still gets full build feedback to act on. What it does not do is stated in the script header rather than left to be discovered: it cannot verify the acknowledgement is true — it greps for a stated tag point, so "tag point: whatever" passes. It is a speed bump against autopilot, not a proof. Its value is forcing the tagger to look at the list, which is precisely what did not happen twice. Scope is CI plumbing with no rivet artifact, matching the #353/#363/#364 precedent — this repo carries no REQ-CI-* ids. Flagged so the omission reads as a choice. Closes #366 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 8e8e1ad)
Same cold-subagent audit as the required-context gate. Three of five claims about this script were refuted, one severely: the gate could be switched off silently, by ordinary mistakes, on exactly the sloppier path. 1. A LIGHTWEIGHT TAG MADE IT VACUOUS — unconditional PASS. `creatordate` on a lightweight tag is the tagged commit's OWN committer date, so `--before=$T` asks for commits descended from C that predate C: on a linear history, always empty. Demonstrated on this repo, same commit 945799a tagged both ways — annotated FAIL (13 excluded), lightweight PASS. Compounding: `%(contents)` on a lightweight tag returns the COMMIT message, so the acknowledgement could be satisfied by text the tagger never wrote. The header called this "a slightly weaker check". It was no check. The trigger is `git tag v0.36.0` instead of `git tag -s`. Now rejected outright, pointing at `git tag -s`. Both counterexamples exit 1. 2. THE ACKNOWLEDGEMENT DISARMED ITSELF. Rule 4 grepped the whole message for `tag[ -]point`. The commit that ADDS this gate has the subject "ci(release): gate the tag point — …", and release-notes tag messages here list commit subjects verbatim (v0.35.0's does). The first release whose notes quoted that subject would auto-acknowledge. Now anchored to the start of a line: a quoted subject sits mid-line after a hash, an acknowledgement is something the tagger began a line with. A trailer-only form (`Tag-Point:`) was tried and rejected — it fails v0.34.0, whose "Tag point is the bump commit ff470a2" is specific and correct. A gate that rejects real acknowledgements teaches people to route around it. 3. NOT IDEMPOTENT, AND IT OVER-REPORTED. `--before` filters COMMITTER date — when a commit was written — but the question is when it LANDED on main. A branch dated the 3rd and merged the 10th injects commits "dated before T" into a range they were not reachable from at tag time. Built in a scratch repo: byte-identical tag object, PASS at tag time, FAIL re-run after the merge. That is the drift #366 rejected in its first design, reintroduced one level down — and it is a FALSE ACCUSATION against a tagger who did tag the tip of main, refuting the header's claim to err "in the PERMISSIVE direction". That sentence is deleted. Fixed with `--first-parent`: the spine holds exactly the squash and merge commits, whose committer dates ARE their landing times, so `--before` becomes exact. The same scratch repo now gives PASS/PASS. Detection is untouched — v0.31.0's missed commit 9557dd0 is a squash commit on the spine. This repo squash-merges, which made the defect latent; main carries 91 merge commits, so latent is not impossible. Backtest over all 42 tags is unchanged end-to-end: 39 PASS, 1 acknowledged (v0.34.0), 2 FAIL (v0.31.0, v0.33.0) — the two real misses #366 was filed for. The holes closed without moving a single verdict on real history. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 36d2688)
…tuck PR
Branch protection's required-check list and the workflows that produce
those checks are edited in different places and drift apart silently.
Three distinct ways, each with a different failure:
- a job runs on every PR and is NOT required -> it enforces nothing.
`osate-corpus` was this shape.
- a required context names a job that no longer exists, or one behind a
workflow-level path filter -> the PR sits at "Expected - waiting for
status" forever. Unmergeable, with nothing red to point at.
- a `continue-on-error` job is required -> the tick is green whatever
happened, so requiring it is decoration.
tools/check_required_contexts.py sorts every PR-triggered job into
GATEABLE / ADVISORY / UNDELIVERABLE from the workflow YAML alone, and
asserts .github/required-contexts.txt is exactly the GATEABLE set.
The distinction the whole gate turns on is invisible in the YAML unless
you look for it: a job skipped by a job-level `if:` reports "skipped",
which SATISFIES a required context. A job in a workflow that never
triggered reports *nothing at all*. Job-level filtering is therefore
gateable and workflow-level `on.pull_request.paths` is not - and getting
that backwards is what produces the unmergeable PR.
Evidence it models reality rather than itself: `--print` emits the
GATEABLE set derived purely from YAML, and it is byte-identical to the 18
contexts branch protection actually enforces today.
Two follow-on fixes it demanded on its first live run: `bazel-test` and
`kani` are both `continue-on-error` but named as if they were verdicts.
"Kani Bounded Model Checking" with a green tick is a claim the run does
not support. Both now carry "(advisory)" in the name - the only place a
PR reader sees it. Neither is a required context, so the rename cannot
strand one.
Why a committed list rather than the live API: reading branch protection
needs an admin token, which the Actions GITHUB_TOKEN is not.
Why steps of rivet-validate rather than a job of its own: a new job would
itself be gateable, so it would have to be added to the required list -
and syncing branch protection needs that same admin token. It would ship
permanently red.
Stdlib-only, following the constraint already documented above the
human-scoped guardrail: no workflow here installs a Python package, so
importing PyYAML would make a *required* check depend on an unverified
runner package. The reader is fail-closed - unrecognised syntax at the
depths it reads exits 2 rather than scanning past it - and `--cross-check`
validates it against PyYAML on the real corpus wherever PyYAML exists.
11 self-tests, each recording the failure it proves is still caught,
including both halves of the path-filter trap: required-and-undeliverable
must fail, and not-required-and-undeliverable must pass. A mechanical
"every job must be required" rule gets that second one wrong.
Closes #372
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 17fe8d5)
A cold subagent was asked to refute this gate's own claims. Five of ten
held; the rest are fixed here. Two of the defects were the gate's stated
failure mode reproduced inside the gate itself.
VERDICT-FLIPPING (the reader classified a job wrongly, silently):
* `pull_request: {branches: [main], paths: ["src/**"]}` — a flow mapping
read as an opaque string. `classify_workflow` does
`if not isinstance(pr, dict): pr = {}`, so the paths filter evaporated,
the job was ruled GATEABLE, and the gate emitted an ::error:: ordering
the author to require a context GitHub will never deliver. The gate
handed you the exact deadlock it exists to prevent.
* four-space indentation — every job id lands at indent 4, where the
reader is not looking, and the indent-2 fail-closed check never fires.
Result: zero jobs, zero violations, PASS. That is fail-OPEN, which the
docstring asserted could not happen.
LOSS-OF-INFORMATION (wrong name, so the real context goes ungated):
* a folded/literal block scalar as a job `name:` read as the literal `>-`;
* YAML anchors and aliases read verbatim.
All four now raise via `_reject_unsupported` plus an empty-`jobs:` check.
Refusing is the only safe answer: `--cross-check` catches these, but it
needs PyYAML and so deliberately does not run in CI — the one place the
gate is load-bearing. Safety there cannot rest on agreeing with a reference
parser; it has to rest on refusing what cannot be represented.
DISCOVERY:
* the glob was `*.yml`. Actions loads `.yaml` too, so a `sneaky.yaml`
holding a red-capable job was invisible and the gate reported PASS on a
directory it never fully read. Now `*.y*ml`, asserted on the constant
itself — the fixtures build their own glob and cannot see it.
TESTS:
* self-test 9 ("job-level if: is required") was provably vacuous:
the reader never reads `if:`, so its fixture parses byte-identically to
the clean one and no mutant can distinguish them. It claimed to prove
"the distinction the whole gate turns on" — a claim about GitHub's
runtime behaviour that no fixture over a YAML string can establish.
Replaced with the `paths-ignore` case, whose guard survived the entire
previous suite (deleting it reddened nothing), refuting the comment
"each fixture kills exactly one guard".
* fixtures added for all four refusals. 15 self-tests; each new guard
verified by mutation — no-op it, and a named fixture goes red.
DOCSTRING:
* `osate-corpus` was cited as a live instance of failure mode 1. It has
the symptom, but `aadl-interop.yml` has a workflow-level paths filter,
so this gate buckets it UNDELIVERABLE, rules it must NOT be required,
and reports PASS. The flagship motivating example is one the gate
declines to fix. Said plainly, with the remedy.
Live corpus unchanged: 21 PR-triggered jobs -> 18 gateable, 2 advisory,
1 undeliverable; --cross-check still agrees with PyYAML across 10
workflows; --against-api still matches branch protection exactly (18).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 00393b2)
…closure
spar has a minimum supported Rust version whether or not anyone writes it
down, because cargo enforces one. Nothing in the repo stated it:
[workspace.package] declared no rust-version and there is no
rust-toolchain.toml. So the effective floor was
max(rust-version) over the entire RESOLVED DEPENDENCY CLOSURE
an emergent property of Cargo.lock that moves on any `cargo update` with
no diff anywhere in this repo, discovered at build time by whichever
consumer happened to have the oldest toolchain.
Measured on the committed lock: 152 of 212 dependencies declare a
rust-version; the maximum is 1.89, from smol_str@0.3.6, one tier above
the 1.87.0 cluster (wit-bindgen, wasip3). A single transitive bump of
smol_str moves the number with nothing here changing.
This already caused a defect. In #364 the fixture-vm nixpkgs pin was
moved 24.05 -> 25.05 and recorded as "cannot compile -> can compile":
edition 2024 needs rustc >= 1.85, 25.05 ships 1.86.0, 1.86 >= 1.85, done.
Every step true, conclusion wrong:
error: rustc 1.86.0 is not supported by the following package:
smol_str@0.3.6 requires rustc 1.89
The edition floor is a NECESSARY condition mistaken for THE condition.
Worth naming precisely because it is not "forgot to check": the probe
genuinely ran and genuinely returned true. A one-variable check that
comes back green is the most convincing way to not verify something.
A bare rust-version line does not fix that — a declaration is just
another number that can go stale against the closure. So the declaration
is the SUBJECT of the check, not the answer to it.
- [workspace.package] rust-version = "1.89", inherited by all 23
members via `rust-version.workspace = true` (the same way they
already inherit version/edition/license/repository). Cargo.lock is
unchanged: this is a declaration, not a dependency change.
- tools/check_msrv.py re-derives the closure maximum from
`cargo metadata --locked` and fails if it exceeds the declared floor,
naming the package responsible.
It reads cargo metadata rather than parsing Cargo.toml because
`rust-version.workspace = true` means the effective value is computed by
cargo, not written in the member manifest. Metadata reports what cargo
actually enforces, and a member that failed to inherit simply has no
rust_version — so that guard falls out for free.
The asymmetry is deliberate: declared < closure fails (consumers cannot
build); declared > closure is reported, not failed, because spar's own
source may legitimately need more than its dependencies do and nothing
here can distinguish that from staleness.
Verified two-sidedly against the real workspace, not just fixtures, since
a gate that cannot fire is worse than none — it reads as coverage:
positive declared 1.89, committed lock -> PASS
negative declared 1.86 (the 25.05 rustc that broke #364) -> exit 1,
"the dependency closure requires rustc 1.89 ... raised by:
smol_str@0.3.6"
Nine self-tests, each recording the failure it proves is still caught.
Two are about the comparison itself: `1.89` and `1.89.0` must be the same
floor (crates.io mixes both spellings and this lockfile contains both),
and `1.100` must be ABOVE `1.99` — as strings it is below, so a
lexicographic compare would pass a floor that is actually too low. The
issue's sketch used `sort -V | tail -1`, which is correct but a GNU
extension; this does not depend on which sort is installed.
The self-test captures its fixtures' stderr instead of letting it
through. Six cases are supposed to fail and their messages are
`::error::` lines — GitHub Actions workflow commands, not plain text.
Emitted from a passing self-test they would stamp eight red annotations
onto the PR diff while the step exits 0, which is the same defect class
this gate exists to prevent.
Wired into rivet-validate rather than a code-gated job: the PR that
raises the real floor is a `cargo update` touching only Cargo.lock, which
is exactly the PR most likely to be filtered out by a path filter.
What this does NOT claim: it does not verify spar's own source compiles
on 1.89. That needs a job pinned to that toolchain. The claim is narrower
and exact — the declared floor is not below what the closure demands.
The channel-rustc half of #369 (compare nixpkgs' rustc against this
floor at nix EVALUATION time, turning a 14-minute build failure into a
seconds-long one) is deliberately NOT in this PR: it lands in
trace-fixtures.yml, which #364 is currently trying to get green for the
first time, and I cannot verify the eval expression locally. Follow-up
once #364 lands.
Refs #369
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7e46618)
Every other statement in this workflow calls rivet.yaml a rivet artifact — the `changes` header says the filter passes "rivet artifacts", and rivet-validate says it catches regressions "in artifacts/, safety/stpa/, and rivet.yaml". The allow-list disagreed: it matches `artifacts/.*\.ya?ml`, and rivet.yaml lives at the repo root, so a rivet.yaml-only PR was classified as code and ran the whole heavy suite — proofs, mutants, fuzz, Miri. Safe to skip those for it. Nothing that compiles reads the file: grepping the tree finds it only in AGENTS.md, two docs/plans pages, artifacts/requirements.yaml, a test fixture, and this workflow — no .rs, no build script, no Cargo.toml. And it does not go unchecked, because rivet-validate deliberately has no `changes` gate at all, which is the point of that exemption. The escape is load-bearing and was verified by running the regex from the file itself (not a retyped copy) over positive and negative controls: rivet.yaml allow-listed; rivet.yaml.bak, subdir/rivet.yaml, myrivetXyaml, Cargo.toml, src/lib.rs and .github/workflows/ci.yml all still classified as code. Closes #379 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
) `cargo mutants --output DIR` creates `mutants.out` INSIDE DIR, so the report lands at `mutants-out/mutants.out/`. The gate read `mutants-out/missed.txt`, one level up, which has never existed: `grep` failed into `|| echo 0`, the `-f` test was false, and every run printed "Surviving mutants: 0" and passed. The read path is byte-identical since 563eb8c (2026-03-22), the commit titled "mutation testing as hard gate". Four months as a required status check that never once read its own results. The property that made it invisible is that it failed *flatteringly*. Zero is not a neutral error value here — it is the best possible score. Every part of the failure produced the ideal reading, so nothing about a green run looked different from a green run that had actually measured something. The proof is the job's own artifact. It uploads with `path: mutants-out/`, so the artifact root is a listing of that directory — and contains exactly one entry, `mutants.out/`. Run 30563879100 (main @ cd4d429) uploaded 210 survivors against a threshold of 142 and reported green. Extracting the old step body from origin/main and handing it that exact directory layout reproduces it: "Surviving mutants: 0 / within threshold (142)", exit 0. The tell was in the tree the whole time. 5d6d98a, 58 minutes after the gate became hard: "Current survivors: 142 (down from 220)." That measurement was real, and structurally impossible for CI to have produced. Someone measured by hand, wrote the number down, and shipped a gate that could not see it. Three changes beyond the path: * Fail closed on an absent report. The cargo-mutants step ends in `|| true` and this job runs on lean-mem because it is RAM-hungry, so "the run died" is expected — and with the path fixed it would still produce no files and still score zero survivors, the best possible result. * Reject a report with zero caught. That means the suite never ran, and it also scores zero survivors. Absence must not render as success. * timeout-minutes: 240. The job had none, so it inherited GitHub's 6-hour default; the one run to reach success took 182.3 minutes. MAX_MISSED is re-baselined 142 -> 210, the measured count on main. This relaxes nothing that was ever in force — 142 was never enforced, so 210 is the first value this gate has actually checked against. The 142 -> 210 drift over four blind months is what a ratchet exists to prevent; walking it back to zero is #382. mutants-weekly.yml had the same wrong path, where it rendered a summary table of 0 missed / 0 caught / 0 unviable — not merely wrong but impossible, since zero caught and zero unviable means cargo-mutants found nothing to test. It still read as a plausible table. `[ -f X ] && VAR=...` is exempt from `set -e`, so the zeros stood and the step passed. THE LOGIC IS NOT IN THE WORKFLOW Fixing the path in place would have left the fix exactly as testable as the bug was — which is to say, only by pushing and reading a log that cannot distinguish "0 survivors" from "0 files read". So the logic moved out of inline shell into tools/check_mutants_report.py, and both workflows became one-line calls. Three consequences: * The nesting fact — cargo-mutants writes into `mutants.out/` under `--output` — is now stated in one place, `_REPORT_SUBDIR`. Two copies of a path is how one of them gets fixed and the other does not; that is precisely what happened between ci.yml and mutants-weekly.yml. * `--self-test` is a 19-case decision table run as a step BEFORE the real check, matching the sibling guardrails (check_msrv.py, check_release_plane.py). The table's load-bearing case reproduces #381 directly: a fixture with the report at the shallow path must exit 1. This gate spent four months unable to fail; it now has to prove it still can before it is allowed to judge. * When the nested report is absent the script prints a tree listing of what it did find under `--output-dir`, and says in the error that cargo-mutants writes `mutants.out` inside `--output`. A layout mismatch now names itself instead of scoring zero — executed against a flattened copy of the real artifact, the listing is the three outcome files sitting one level too high. The counts are cross-checked against `outcomes.json` when present, so the number the gate prints has a second, independent derivation. On disagreement it fails and says so rather than picking one: "Do not adjust the threshold to match — find out which reading is wrong." The weekly's summary path shares the same script via `--summary-file`. It is advisory and never fails the job, but it now refuses to render absence as a table of zeros; it writes "Counts are omitted rather than rendered as 0" instead. One catch worth recording, because it is #381's own shape one layer up: the script was `-rw-r--r--` while both workflows invoke it bare. Every local test I ran said PASS because I ran them as `python3 tools/...`. The thing I measured was not the thing that runs. Fixed to 100755, matching check_msrv.py, and both commands re-run bare to confirm. Verified (all executed, on the real 15MB artifact from run 30563879100 unless noted): --self-test, invoked bare -> 19/19, exit 0 no mutants-out at all -> exit 1 (was: exit 0, "0 survivors") mutants-out without mutants.out (#381) -> exit 1 (was: exit 0, "0 survivors") the real 210/603 report, --max-missed 210 -> exit 0, "Surviving mutants: 210" the same report, --max-missed 209 -> exit 1 the same report, --max-missed 142 -> exit 1 (the old threshold) report with 0 caught -> exit 1 outcomes.json disagreeing with missed.txt -> exit 1 weekly summary, no report -> warns, omits the table, exit 0 weekly summary, real report -> 210 / 603 / 2 / 793 plus an end-to-end simulation running the step bodies extracted from both workflows over all three states, and `rivet validate` under the CI-pinned v0.4.3: PASS, exit 0. Artifacts: REQ-GUARD-GATE-EVIDENCE-001 / TEST-GUARD-GATE-EVIDENCE. The requirement states what this does NOT claim: 210 survivors out of 1608 is not a claim of good tests, and the general property — a gate proves it measured something before it reports a pass — is not mechanically enforced anywhere. Refs #381, #382 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The required `Format` check has been green on main over unformatted
tracked Rust. `cargo fmt --all` means "all members of THIS workspace",
not "all crates in the repo", and this repo has four workspaces:
Cargo.toml the 23-crate root
fuzz/Cargo.toml own [workspace] table
codegen-exec-oracle/Cargo.toml own [workspace] table
codegen-kiln-oracle/Cargo.toml own [workspace] table
A nested crate with its own [workspace] table is a separate workspace,
and the root's --all never opens it.
TWO GATES, ONE TREE, OPPOSITE VERDICTS. Executed against main's version
of fuzz/fuzz_targets/fuzz_codegen_roundtrip.rs:
cargo fmt --all -- --check -> 0
cargo fmt --manifest-path fuzz/Cargo.toml --all -- --check -> 1
(13 diff lines, :32)
tools/check_fmt_workspaces.py -> 1
4 workspace(s) checked, 3 formatted.
With :32 reformatted the checker exits 0 and prints "4 workspace(s)
checked, 4 formatted." Nothing in CI has ever formatted fuzz/.
WHY IT LOOKED FINE. This is #381's property arriving by a different
mechanism. #381's error path produced the ideal reading; here there is
no error path at all. cargo prints nothing on success, so 23-of-26
crates clean is byte-identical to 26-of-26 clean, and a gate that says
nothing when it passes cannot distinguish "checked everything" from
"checked a subset". The fix is therefore not a corrected path but a
mandated output: the checker prints "N workspace(s) checked, M
formatted" on every path, including success. A count is not producible
by a broken scan. That is the entire trick.
DISCOVERY WITH A FLOOR, not a list of four. Four hard-coded manifests
would fix today's bug and re-open it the day someone adds a fifth
workspace, so the set is walked for `[workspace]` tables and floored at
--min-workspaces (4). The walk is what makes it exhaustive; the floor is
what makes a discovery failure loud instead of green.
THE PRUNE LIST IS LOAD-BEARING, not cosmetic. target/ fills with
vendored crates.io sources, and plenty of published crates ship their
own [workspace] table. A naive walk on a machine that has built once
finds dozens of them, clears the >= 4 floor comfortably, and then asks
cargo to format other people's build cache. That bug PASSES, which is
why it is two self-test rows — target/, and a nested fuzz/target/, the
second because it fails if pruning is applied only at the top level
instead of at every depth — rather than a comment.
SELF-TEST FIRST, as with the sibling guardrails: 10 cases run as their
own step before the real check, so the gate proves it can still fail
before it may pass. The table carries the #383 regression itself (root
clean + fuzz/ dirty must exit 1), the floor in both directions, and the
`[workspace]` vs `[workspace.package]` distinction — counting
inheriting members would inflate the total past the floor and disarm it.
THREE-VALUED EXIT, from check_msrv.py: 0 formatted, 1 violation, 2 could
not measure (cargo absent, root unreadable, discovery empty). Spelling
"I could not measure" the same way as "I measured and it was fine" is
how #381 ran for four months.
Verified in this tree, not reasoned:
| claim | evidence |
|------------------------------------------|---------------------------------|
| four workspaces, discovered not listed | root, fuzz, codegen-exec-oracle, codegen-kiln-oracle |
| old gate green on the drift | `cargo fmt --all -- --check` -> 0 |
| the drift is real and tracked | fuzz manifest -> 1, 13 diff lines |
| new gate red on that same tree | exit 1, "4 checked, 3 formatted" |
| new gate green once reformatted | exit 0, "4 checked, 4 formatted" |
| self-test, invoked bare | 10/10 pass, exit 0 |
| script is executable in git | mode 100755 |
| artifacts parse and validate | `rivet validate` PASS, exit 0 |
| sibling artifact guardrail unaffected | check_human_scoped.py exit 0, 887 artifacts |
NOT VERIFIED, and the reason this says so out loud: that python3 exists
on the `light` runner class. The fmt job is the first `light` consumer
of a tools/check_*.py guardrail — rivet-validate runs on rust-cpu and
mutants on lean-mem, and those are the only classes on which these
scripts have been observed. The failure mode is fail-CLOSED: a missing
interpreter exits 127 and turns the required check red. This PR's own
CI is the measurement; if it 127s, the job moves to rust-cpu.
SCOPE. `Clippy` has the identical `--workspace` gap and is untouched
here, so the three nested crates remain unlinted — #383 stays open on
that half. #386 tracks the other nine required contexts that share the
underlying property.
Artifacts: extends REQ-GUARD-GATE-EVIDENCE-001 with obligation (b),
formatting scope, and adds TEST-GUARD-FMT-WORKSPACES.
Refs #383, #381, #386
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on one tree REQ-GUARD-STATUS-VOCAB-001 shipped without its natural `traces-to: REQ-GUARD-RELEASE-PLANE-001`. That was deliberate, and the artifact says so in its own description: rivet resolves a `links:` target at validate time, so while the two requirements sat in separate unmerged PRs the link would have made each branch's oracle pass or fail depending on which PR merged first. A gate whose verdict depends on merge order is not a traceability improvement. #373 put REQ-GUARD-RELEASE-PLANE-001 on main (d4f9a83) and this branch is rebased onto it, so the target now resolves against the same tree that validates it. The "NOTE ON THE MISSING LINK" paragraph is rewritten rather than deleted. An artifact asserting it carries no link while carrying one is precisely the drift these guardrails exist to catch, and the reason for the delay is worth keeping — it generalises to any cross-PR link. Load-bearing, not decorative — established by mutation, not assumed: target: REQ-GUARD-RELEASE-PLANE-001 -> rivet validate exit 0, 0 errors target: REQ-GUARD-NO-SUCH-TARGET-999 -> rivet validate exit 1 ERROR: [REQ-GUARD-STATUS-VOCAB-001] link 'traces-to' targets 'REQ-GUARD-NO-SUCH-TARGET-999' which does not exist A link rivet silently ignored would have given the same exit code both ways. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Rivet verification gate✅ 20/20 passed
Filter: Failed artifacts(none) Updated automatically by |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One branch carrying six reviewed PRs, because
mainhasrequired_status_checks.strict = trueand this repo has no merge queue.Under
strict, the first merge advancesmainand every remaining PR mustupdate-branch and re-run all 18 required contexts. Six sequential merges is
six full CI rounds on an org-shared runner pool; this is one.
Most of this is not new work: six of the commits are the reviewed heads of
existing PRs, cherry-picked unmodified. Four additions arrived while the batch
was being assembled — a small change-filter fix (#379), a link that only became
addable once these landed together, and two gate repairs (#381, #383) that are
the substantive new items and are described below.
What is in it
ci(fuzz): stop the scheduler_solver nightly OOMing on ASan bookkeepingfix(rivet): stop double-loading safety/stpa …+correct the "loses nothing" claimfix(artifacts): move 276 statuses onto rivet's lifecycle vocabularyci(release): gate the tag point …+a clean-room audit disarmed this gate three waysfix(ci): gate required-context drift …+close four fail-open holesfix(msrv): declare the Rust floor and gate it against the dependency closureci: allow-list rivet.yaml in the change filterfix(artifacts): state the link that was deferred to avoid a flaky gatefix(ci): the mutation gate reads a path cargo-mutants never writesci(fmt): check every workspace in the repo, and print how manyThe item that is not a cherry-pick: #381
Mutation Testingis a required status check. It runs for roughly three hours.It has never once read its own results.
cargo mutants --output DIRcreatesmutants.outinside DIR, so the reportlands at
mutants-out/mutants.out/. The gate readmutants-out/missed.txt, onelevel up — a path that has never existed.
grepfailed into|| echo 0, the-ftest was false,0 -gt 142was false, exit 0.git log -Lputs that readpath byte-identical since 563eb8c on 2026-03-22. Four months.
The proof is the job's own artifact. It uploads with
path: mutants-out/, so theartifact root is a listing of that directory — and it contains exactly one
entry,
mutants.out/. Run 30563879100 (main @cd4d429) uploaded 210survivors against a threshold of 142 and reported green.
The tell was in the tree the whole time. 5d6d98a, 58 minutes after the gate went
hard:
That measurement was real and correct, and structurally impossible for CI to
have produced — CI was printing 0. Someone measured by hand, wrote the number
down, and shipped a gate that could not see it. Two numbers for one quantity sat
in the repository for four months and were never diffed. That diff is now
automated: the checker refuses to score a report whose
.txtline countsdisagree with
outcomes.json, so the only detector that would have worked fromday one is the one thing the gate can no longer skip.
Beyond the path, three changes, because a corrected path alone would still fail
open:
|| trueand the jobruns on
lean-membecause it is RAM-hungry, so "the run died" is expected —and it produces no files, which scores zero survivors, the best possible
result. The whole reason this was invisible is that the error path yielded the
ideal reading.
caught == 0→ fail. A report that exists but caught nothing means thesuite never ran. Also scores zero survivors.
timeout-minutes: 240. The job had none, inheriting GitHub's 6-hourdefault; the one run to reach
successin the last 25 took 182.3 minutes.The logic is no longer in the workflow. Inline shell in a YAML file is only
testable by pushing, which is exactly how a misread survived four months with
nobody able to run it against a report. It now lives in
tools/check_mutants_report.pywith a 19-case decision table that runs asits own step before the gate — the same convention as the human-scoped,
required-context and MSRV guardrails already in this branch. The table's
load-bearing row is the #381 regression itself: a complete, valid set of outcome
files placed at the shallow path must exit 1.
mutants-weekly.ymlcalls thesame script with
--summary-file, so the nesting fact lives in one constant;two copies of a path is how one gets fixed and the other does not.
MAX_MISSEDis re-baselined 142 → 210, the measured count on main. Thisrelaxes nothing that was ever in force — 142 was never enforced, so 210 is the
first value this gate has actually checked against. The 142 → 210 drift over four
blind months is precisely what a ratchet exists to prevent; #382 tracks walking
it back to zero and says not to raise it again.
mutants-weekly.ymlhad the same wrong path, where it rendered a summary tableof
0 missed / 0 caught / 0 unviable— not merely wrong but impossible, sincezero caught and zero unviable would mean cargo-mutants found nothing to test.
It still read as a plausible table.
This PR will be the first time the gate actually runs.
crates/spar-analysis/is untouched between
cd4d429and this branch's base, so 210 should hold; if itcomes in higher, that is the gate working and I will treat it as a finding rather
than adjust the number.
What #381 turned up next: #383
#381 is not a mutation-testing bug. It is an instance of a property — a check
whose "nothing happened" renders identically to "it worked" — so all 18 required
contexts were audited against one question: can this report success without
having measured the thing its name claims? The audit is #386. It found 11 with
a path to green; four are live today. This branch fixes one of them.
Formatrunscargo fmt --all -- --check.--allmeans "all members of thisworkspace", not "all crates in the repo", and this repo has four workspaces — the
23-crate root plus
fuzz/,codegen-exec-oracle/andcodegen-kiln-oracle/,each with its own
[workspace]table. Two gates, one tree, opposite verdicts:The drift is tracked code:
fuzz/fuzz_targets/fuzz_codegen_roundtrip.rs:32.A required status check named
Formathas been green over unformatted Rust, andnothing in CI has ever formatted
fuzz/.This one has no error path at all — cargo prints nothing on success, so 23-of-26
crates clean is byte-identical to 26-of-26 clean. The fix is therefore a
mandated output rather than a corrected path.
tools/check_fmt_workspaces.pydiscovers the workspace set, floors it at 4, and prints
4 workspace(s) checked, 4 formatted.on every path including success. A countis not producible by a broken scan — that is the general fix #386 proposes, and
this is its first application outside #381.
Two details worth review attention:
re-open it the day someone adds a fifth workspace. The floor is what makes a
discovery failure loud instead of green.
target/fills with vendored crates.iosources, many shipping their own
[workspace]table. A naive walk on a machinethat has built once finds dozens, clears the
>= 4floor comfortably, and askscargo to format other people's build cache. That bug passes, so it is two
self-test rows rather than a comment.
Clippyhas the identical--workspacegap and is not fixed here, so #383stays open on that half.
#369 stays open deliberately. #378 discharges the gate but not the
nix-eval channel-rustc check it owes, which is blocked on #364 landing. The
commit carries no
Closesfor that reason.Why a merge commit rather than a squash
These are unrelated fixes with substantive rationale in their individual
messages; squashing them into one commit would discard that. Merge commits are
enabled on this repo, and the tag-point gate this PR carries walks
--first-parent, so the feature commits sit off the spine and only the mergecommit lands on it, dated at merge time — exactly the case its header
anticipates.
Conflicts
Cherry-picking the nine cherry-picked commits in order produced one conflict:
a pure add/add in
.github/workflows/ci.ymlbetween #377's Required-contextguardrail steps and #378's MSRV guardrail steps. Resolved by keeping both, in the
order Human-scoped → Release-plane → Required-context → MSRV.
Worth recording that the earlier n-way merge of the same branches produced
three. A cherry-pick replays a patch against the accumulating tree, so each
earlier fix is already context when the next applies; an n-way merge reconciles
independent trees against one base, so every branch appending at the same anchor
collides with every other. Same content, different reconciliation, different
conflict count.
The deferred link, now stated
REQ-GUARD-STATUS-VOCAB-001(from #374) deliberately omittedtraces-to: REQ-GUARD-RELEASE-PLANE-001, because that target artifact arrivedin a different unmerged PR and a
links:target is resolved at validate time —the link would have made the branch's oracle pass or fail depending on merge
order. A flaky gate is not a traceability improvement. Both artifacts are now
on
maintogether, so the condition that forced the omission no longer holdsand the link is stated.
Verification run locally before pushing
Exit codes captured from the commands themselves, not from a pipeline tail.
Re-run on the rebased tree, not carried over from the pre-rebase worktree.
That distinction mattered:
check_release_plane.pydid not exist on this branchbefore the rebase (it arrives with #373), and one figure below moved — the
human-scoped artifact count went 887 → 888, because #373's merge added
REQ-GUARD-RELEASE-PLANE-001. Every number here is from the tree being pushed.Five guardrail decision tables, 56 cases, all invoked bare (so the exec bit
is part of what is tested):
check_human_scoped.py --self-test— 3 passed, exit 0check_required_contexts.py --self-test— 15 passed, exit 0check_msrv.py --self-test— 9 passed, exit 0check_mutants_report.py --self-test— 19 passed, exit 0check_fmt_workspaces.py --self-test— 10 passed, exit 0check_release_plane.py --self-test— 4 passed, exit 0and
check_fmt_workspaces.pybare against the rebased tree:4 workspace(s) checked, 4 formatted., exit 0 — the same command that exits 1 onmain.Then the same checkers against the real tree:
check_required_contexts.py— scanned 21 PR-triggered jobs: 18 gateable,2 advisory, 1 undeliverable; 18 required contexts — PASS
check_msrv.py— declared floor 1.89 ([workspace.package], inherited by 23members), closure maximum 1.89 (set by
smol_str@0.3.6; 152 of 212 declareone) — PASS
check_human_scoped.py— 888 artifacts scanned, 6 human-scoped — exit 0check_release_plane.py— 11 files scanned, 88release:keys on the artifactkey plane that
rivet list --releasecan see, 0 nested where it is blind —PASS. This is the post-release: field is on the wrong plane for 59 of 85 artifacts — rivet's release-planning query is blind to 11 shipped releases #370 steady state measured on the merged tree: the 59
misplaced assignments are gone and none came back through the rebase
check-tag-point.shbacktested against real history:v0.31.0exit 1 andv0.33.0exit 1 (both historical misses correctly re-detected),v0.34.0exit 0 "PASS (acknowledged) … excludes 6 commit(s)",
v0.35.0exit 0"nothing predating the tag was left out" — non-vacuous in both directions
rivet validate— PASS, exit 0#379allow-list regex, extracted from the file rather than retyped, run overpositive and negative controls:
rivet.yamlallow-listed;rivet.yaml.bak,subdir/rivet.yaml,myrivetXyaml,Cargo.toml,src/lib.rsand.github/workflows/ci.ymlall still classified as codethe same classifier run over this branch's own 43-file diff: 35 code-ish,
so
code=trueand the full suite runs — which is the reachability the two newcheckers depend on, and
rivet.yamlappears in the 43 but not in the 35, sothe CI path filter: rivet.yaml is not allow-listed, so pure-manifest PRs run the full heavy suite #379 commit is non-vacuous on the very PR that carries it
#381both step bodies parsed out of the YAML withyaml.safe_load(notretyped) and executed against the real 15 MB artifact from run 30563879100:
mutants-outat all (crashed / OOM run)mutants-out/present, nomutants.out/caught.txtemptyThe third row is the one that matters: the old step body, taken verbatim from
origin/mainand handed the exact directory layout cargo-mutants produces,reports zero survivors and passes.
#383the new checker run against the real tree, not only fixtures — onmain's version of the fuzz target it exits 1 with
4 workspace(s) checked, 3 formatted.and reproduces the 13-line rustfmt diff; with:32reformattedit exits 0 with
4 workspace(s) checked, 4 formatted.One assumption this PR's own CI is the measurement for: the
fmtjob runs onthe
lightrunner class, and it is the firstlightconsumer of atools/check_*.pyguardrail — the existing ones run onrust-cpuandlean-mem. Ifpython3is absent there the step exits 127 and the requiredcheck goes red, which is the correct direction to fail; the job then moves to
rust-cpu.Closes #360, #361, #366, #371, #372, #379, #381
Refs #383, #386
🤖 Generated with Claude Code