fix(artifacts): move 276 statuses onto rivet's lifecycle vocabulary - #374
fix(artifacts): move 276 statuses onto rivet's lifecycle vocabulary#374avrabe wants to merge 3 commits into
Conversation
…EQ (#360) `rivet validate` reported 466 errors on a clean tree. 190 of them were phantom: every id in four STPA files collided with ITSELF, producing "declared more than once" messages with the same path on both sides. Cause: rivet.yaml listed `safety/stpa` as a directory scan AND listed four files inside it individually (to force `format: generic-yaml`). Each file was loaded twice, so every id it declared was declared twice. Listing a directory and its own contents is not additive — it is duplicative, and the diagnostic could not say so because both copies had identical provenance. Removing the four explicit entries is safe, and the artifact set was compared rather than assumed: 883 ids and 1195 links, byte-identical before and after via `rivet list --format json`. The format change loses nothing; stpa-yaml is a superset of what generic-yaml extracted from these files. That left 6 real duplicates: RENDER-REQ-001..006 were defined in BOTH artifacts/requirements.yaml and safety/stpa/rendering-analysis.yaml. rivet kept one copy and dropped the other with no diagnostic at all, and which one won was load order rather than a decision. The stubs are deleted, not the canonical copy, because only the STPA file carries the hazards/satisfies/verified-by links that make these part of the STPA V. The stubs carried none. The stubs were also stale in BOTH directions, which is why this was checked against source instead of picking the newer-looking file: - RENDER-REQ-006 claimed `partial`; it is `implemented`. etch/src/html.rs emits `svg.zoom-low` / `svg.zoom-overview` rules and asserts a `wheel` handler (html.rs:65-68, :131). - RENDER-REQ-003 claimed `implemented`; it is `partial`. `minimap` is still `/// Show minimap (Phase 3b - reserved)`. Verified at the pinned etch rev 4c06709, not by grepping this repo. A repo-scoped grep says "zoom-low appears only in a test fixture", which reads exactly like "not implemented" when the implementation lives in a git dependency. That nearly produced a false finding in the other direction. Also normalises the status vocabulary: `pass` -> `passing` (5) and `pending` -> `planned` (4), collapsing singletons into the dominant spelling. MEASUREMENTS, and one correction to the issue --------------------------------------------- 466 errors -> 282 (drop the source overlap: 190 duplicates -> 6) -> 276 (delete the 6 RENDER-REQ stubs: 6 -> 0) -> 276 (normalise the vocabulary: NO CHANGE) Duplicate-id errors are now 0. But #360 predicted that with the overlap and the vocabulary both fixed "the residual error count should be near zero", and that is FALSE. The normalisation cannot reduce the count, because its targets are not in the allowed set either: allowed: draft, proposed, approved, implemented, verified, released, accepted, deprecated, rejected `passing` and `planned` are both absent, so `pass` -> `passing` moves 5 errors between buckets (172 -> 177) and `pending` -> `planned` moves 4 (36 -> 40). Net zero, as measured. It is still worth doing: it leaves 4 non-conforming values to resolve instead of 6. It is just not an error fix, and recording it as one would have overstated what this change buys. The residual 276 are entirely status-vocabulary skew: 177 `passing`, 51 `not-implemented`, 40 `planned`, 8 `partial`. These are test/verification states, not approval-lifecycle states, and `common@0.3.0` is embedded in the rivet binary -- schemas/ only overrides sysml2 -- so the repo cannot widen them locally. This does NOT unblock lifting the v0.4.3 pin in ci.yml; see #360 for the measurement. The pin is load-bearing for the green check, which is a fact worth stating plainly rather than leaving implicit in a comment. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… test-names The equivalence check this PR cited was measured on `rivet list --format json` WITHOUT `--full`, which emits only the summary view (`--help` states it omits `description`, `tags`, and `fields`). That view is structurally incapable of detecting field-level loss — precisely the class of change a `format:` switch produces. "Byte-identical" was true and meaningless. Re-measured with `--full` on base vs. this branch: +86 artifacts GAIN `mitigates` and `traces-to` -5 artifacts LOSE half of `test-name` The five are VAL-010, VAL-011, VAL-014, VAL-015, VAL-021 — exactly the five whose on-disk `test-name` is a comma-separated pair. The stpa-yaml parser keeps only the first element: on disk: tests::boolean_matches_aadl_boolean, tests::boolean_mismatches_aadl_string rivet: tests::boolean_matches_aadl_boolean A/B on identical bytes with only `format:` differing confirms the parser is the cause: generic-yaml round-trips the full string. The trade is still worth taking — five truncated evidence pointers against 190 phantom duplicate-id errors — but it is a trade, not a free win, and the comment now says so. Filed upstream as pulseengine/rivet#747; revert the trade if that lands a fix. No behaviour change: `rivet validate` is FAIL (276 errors, 629 warnings) before and after this commit. Comment-only. Found by cold-context adversarial verification of this PR's own claims. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spar carried a private status vocabulary — `passing`, `planned`,
`not-implemented`, `partial` — on 276 artifacts. None of the four is in the
schema's allowed set, so every one was a `status-allowed-values` error and
`rivet validate` could not run clean.
The mapping, and what it costs:
passing -> implemented (177)
planned -> proposed
not-implemented -> proposed
partial -> approved (8, lossy)
`passing` was the largest group and the one worth naming. It is a verification
VERDICT sitting in a lifecycle field, and a hand-edited YAML file cannot
honestly assert that a test passes — only that the measure exists. That is what
`implemented` says.
Only `partial` loses information: it sits strictly between `approved` and
`implemented` and the vocabulary has no such rung. `approved` is chosen as the
conservative floor — the strongest claim certainly true — over `implemented`,
which would overstate. All eight say so in their own description. Two
(RENDER-REQ-003, COVERAGE-GAPS) already itemized their residue and were left
alone; the other five state the residue is NOT itemized. That wording is
deliberate — an explicit "not measured" is falsifiable, an invented figure
("32 of 50 rules") is not, and inventing one is the exact failure #294 was
reopened for.
No new guard ships with this. rivet already enforces `status-allowed-values` at
ERROR severity and `rivet validate` is already required, so the regression is
blocked the moment this lands; a bespoke checker would only add a weaker second
oracle beside a working one. That is the opposite of REQ-GUARD-RELEASE-PLANE-001,
which needed a tool precisely because rivet was blind to the defect.
Verification, on three planes that agree:
- the migration asserts per file that the artifact-id set is unchanged, that
every record is byte-identical outside `status`, and that the text-rewrite
count equals the semantic-move count. That last one matters: "N lines
rewritten" is equally consistent with having rewritten N *prose* lines inside
`description:` blocks while leaving N real statuses alone.
- `rivet validate` exits 0 with 0 error-severity diagnostics under
common@0.3.0 (embedded), down from 276.
- the gate is live, not vacuous: re-introducing one off-vocabulary status
reproduces exactly one `status-allowed-values` error, and restoring it
returns exit 0 with a byte-identical file. Distinct inputs, distinct
verdicts — the property #327's withdrawn certificate did not have.
Reconciliation of the raw text count against rivet's: 303 `status:` matches
across 9 committed files, minus 21 in safety/requirements.yaml (which rivet
never loads — it is under no path in rivet.yaml's `sources:`), minus 6 in
safety/stpa/architecture.yaml (nested in the `fields:` extension bag, where
rivet cannot see them) = 276. Both remainders are left in place and named in
REQ-GUARD-STATUS-VOCAB-001 rather than silently absorbed; the second is the #370
defect one field over, on `status:` instead of `release:`.
Addresses #371.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Correction to the PR body: the base is now I opened this stacked on #368 for a clean diff. That was wrong, and wrong in a way worth recording because it is the exact defect class #372 is about.
Retargeting to Consequence for review: until #368 merges, this PR shows 3 commits — #368's two plus this one — because that is genuinely the diff against Written up in #372 as an argument that the proposed standing check should also assert every open PR has a non-zero required-context count — no amount of auditing |
Rivet verification gate✅ 20/20 passed
Filter: Failed artifacts(none) Updated automatically by |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Closes #371.
Stacked on #368 (base is
fix/360-rivet-source-dedup); GitHub will retarget tomainwhen that merges. The two touchartifacts/requirements.yamland so does #373 — whichever lands last needs a rebase.What this is
spar carried a private status vocabulary on 276 artifacts. None of the four words is in the schema's allowed set, so every one was a
status-allowed-valueserror andrivet validatecould not run clean.passingimplementedplannedproposednot-implementedproposedpartialapprovedpassingis the group worth naming. It is a verification verdict sitting in a lifecycle field, and a hand-edited YAML file cannot honestly assert that a test passes — only that the measure exists. That is whatimplementedsays.The lossy eight
partialsits strictly betweenapprovedandimplemented, and the vocabulary has no such rung.approvedis the conservative floor — the strongest claim certainly true — rather thanimplemented, which would overstate.Two of the eight (
RENDER-REQ-003,COVERAGE-GAPS) already itemized their residue in prose and were left alone. The other five now say the residue is not itemized. That wording is deliberate and I want it reviewed as a choice, not skimmed as boilerplate:lower.rshas 54#[test]functions, but STPA-REQ-036 requires one test per lowering rule — a mapping question a count cannot answer. Writing "54 tests exist" would be the necessary-but-not-sufficient trap. An explicit "not measured" is falsifiable; an invented figure is not, and inventing one is the exact failure #294 was reopened for.No new guard ships here — on purpose
rivet already enforces
status-allowed-valuesat ERROR severity andrivet validateis already a required check, so the regression is mechanically blocked the moment this lands. A bespoke checker would add a weaker second oracle beside a working one.This is the opposite of #370 / #373, which needed
tools/check_release_plane.pyprecisely because rivet was blind to that defect.Verification — three planes that agree
status, and the text-rewrite count equals the semantic-move count. That last assertion is the load-bearing one: "N lines rewritten" is equally consistent with having rewritten N prose lines insidedescription:blocks while leaving N real statuses alone. Block scalars are tracked and skipped rather thansed'd through.rivet validate→ exit 0, 0 error-severity diagnostics undercommon@0.3.0 (embedded), down from 276. The diff is276+/276-on the status lines — a strict 1:1 swap, no structural change.status-allowed-valueserror; restoring it returns exit 0 with a byte-identical file. Distinct inputs → distinct verdicts. This is the property Certify AADL↔WIT↔canonical-ABI byte-layout equivalence with ordeal (close the #319 layout gap) #327's withdrawn certificate did not have, so I no longer report an exit-0 without it.Reconciliation: why 303 raw hits but 276 errors
status:text matches, 9 committed filessafety/requirements.yamlrivet.yaml'ssources:safety/stpa/architecture.yamlfields:extension bag, where rivet cannot see themBoth remainders are left in place and named in
REQ-GUARD-STATUS-VOCAB-001rather than silently absorbed into the total. The second one is the #370 defect one field over —status:instead ofrelease:— andtools/check_release_plane.pyfrom #373 would catch it by changing a single module-level constant (FIELD = "release"). That generalization is a follow-up, not this PR: doing it here would stack this branch three-deep and break the "vocabulary only" scope I posted to #371.Deliberately NOT in scope
Re-typing the 198
TEST-*/VAL-*measures tounit-verificationand re-pointingsatisfies→verifies. As measured on #371, those are ASPICE types requiringsw-detail-design/sw-req, of which spar has zero across 883 artifacts — adopting them would drag in SWE.1/SWE.2/SWE.3 and take feature-stage coverage from 194 requirements to 22. Separately scoped; the root friction is filed as pulseengine/rivet#748.🤖 Generated with Claude Code