Skip to content

fix(artifacts): move 276 statuses onto rivet's lifecycle vocabulary - #374

Open
avrabe wants to merge 3 commits into
mainfrom
fix/371-status-vocabulary
Open

fix(artifacts): move 276 statuses onto rivet's lifecycle vocabulary#374
avrabe wants to merge 3 commits into
mainfrom
fix/371-status-vocabulary

Conversation

@avrabe

@avrabe avrabe commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Closes #371.

Stacked on #368 (base is fix/360-rivet-source-dedup); GitHub will retarget to main when that merges. The two touch artifacts/requirements.yaml and so does #373 — whichever lands last needs a rebase.

What this is

spar carried a private status vocabulary on 276 artifacts. None of the four words is in the schema's allowed set, so every one was a status-allowed-values error and rivet validate could not run clean.

from to n lossy?
passing implemented 177 no
planned proposed 40 no
not-implemented proposed 51 no
partial approved 8 yes

passing is the group worth naming. It is a verification verdict sitting in a lifecycle field, and a hand-edited YAML file cannot honestly assert that a test passes — only that the measure exists. That is what implemented says.

The lossy eight

partial sits strictly between approved and implemented, and the vocabulary has no such rung. approved is the conservative floor — the strongest claim certainly true — rather than implemented, which would overstate.

Two of the eight (RENDER-REQ-003, COVERAGE-GAPS) already itemized their residue in prose and were left alone. The other five now say the residue is not itemized. That wording is deliberate and I want it reviewed as a choice, not skimmed as boilerplate: lower.rs has 54 #[test] functions, but STPA-REQ-036 requires one test per lowering rule — a mapping question a count cannot answer. Writing "54 tests exist" would be the necessary-but-not-sufficient trap. An explicit "not measured" is falsifiable; an invented figure is not, and inventing one is the exact failure #294 was reopened for.

No new guard ships here — on purpose

rivet already enforces status-allowed-values at ERROR severity and rivet validate is already a required check, so the regression is mechanically blocked the moment this lands. A bespoke checker would add a weaker second oracle beside a working one.

This is the opposite of #370 / #373, which needed tools/check_release_plane.py precisely because rivet was blind to that defect.

Verification — three planes that agree

  1. The migration proves its own edit. Per file it asserts the artifact-id set is unchanged, every record is byte-identical outside status, and the text-rewrite count equals the semantic-move count. That last assertion is the load-bearing one: "N lines rewritten" is equally consistent with having rewritten N prose lines inside description: blocks while leaving N real statuses alone. Block scalars are tracked and skipped rather than sed'd through.
  2. The oracle flips. rivet validate → exit 0, 0 error-severity diagnostics under common@0.3.0 (embedded), down from 276. The diff is 276+/276- on the status lines — a strict 1:1 swap, no structural change.
  3. The gate is live, not vacuous. Re-introducing one off-vocabulary status reproduces exactly one status-allowed-values error; restoring it returns exit 0 with a byte-identical file. Distinct inputs → distinct verdicts. This is the property Certify AADL↔WIT↔canonical-ABI byte-layout equivalence with ordeal (close the #319 layout gap) #327's withdrawn certificate did not have, so I no longer report an exit-0 without it.

Reconciliation: why 303 raw hits but 276 errors

count why
raw status: text matches, 9 committed files 303
safety/requirements.yaml −21 rivet never loads it — it is under no path in rivet.yaml's sources:
safety/stpa/architecture.yaml −6 nested in the fields: extension bag, where rivet cannot see them
= rivet's error count 276 ✅ exact

Both remainders are left in place and named in REQ-GUARD-STATUS-VOCAB-001 rather than silently absorbed into the total. The second one is the #370 defect one field over — status: instead of release: — and tools/check_release_plane.py from #373 would catch it by changing a single module-level constant (FIELD = "release"). That generalization is a follow-up, not this PR: doing it here would stack this branch three-deep and break the "vocabulary only" scope I posted to #371.

Deliberately NOT in scope

Re-typing the 198 TEST-*/VAL-* measures to unit-verification and re-pointing satisfiesverifies. As measured on #371, those are ASPICE types requiring sw-detail-design/sw-req, of which spar has zero across 883 artifacts — adopting them would drag in SWE.1/SWE.2/SWE.3 and take feature-stage coverage from 194 requirements to 22. Separately scoped; the root friction is filed as pulseengine/rivet#748.

🤖 Generated with Claude Code

avrabe and others added 3 commits July 30, 2026 19:59
…EQ (#360)

`rivet validate` reported 466 errors on a clean tree. 190 of them were phantom:
every id in four STPA files collided with ITSELF, producing "declared more than
once" messages with the same path on both sides.

Cause: rivet.yaml listed `safety/stpa` as a directory scan AND listed four files
inside it individually (to force `format: generic-yaml`). Each file was loaded
twice, so every id it declared was declared twice. Listing a directory and its
own contents is not additive — it is duplicative, and the diagnostic could not
say so because both copies had identical provenance.

Removing the four explicit entries is safe, and the artifact set was compared
rather than assumed: 883 ids and 1195 links, byte-identical before and after via
`rivet list --format json`. The format change loses nothing; stpa-yaml is a
superset of what generic-yaml extracted from these files.

That left 6 real duplicates: RENDER-REQ-001..006 were defined in BOTH
artifacts/requirements.yaml and safety/stpa/rendering-analysis.yaml. rivet kept
one copy and dropped the other with no diagnostic at all, and which one won was
load order rather than a decision.

The stubs are deleted, not the canonical copy, because only the STPA file
carries the hazards/satisfies/verified-by links that make these part of the
STPA V. The stubs carried none.

The stubs were also stale in BOTH directions, which is why this was checked
against source instead of picking the newer-looking file:

  - RENDER-REQ-006 claimed `partial`; it is `implemented`. etch/src/html.rs
    emits `svg.zoom-low` / `svg.zoom-overview` rules and asserts a `wheel`
    handler (html.rs:65-68, :131).
  - RENDER-REQ-003 claimed `implemented`; it is `partial`. `minimap` is still
    `/// Show minimap (Phase 3b - reserved)`.

Verified at the pinned etch rev 4c06709, not by grepping this repo. A repo-scoped
grep says "zoom-low appears only in a test fixture", which reads exactly like
"not implemented" when the implementation lives in a git dependency. That nearly
produced a false finding in the other direction.

Also normalises the status vocabulary: `pass` -> `passing` (5) and `pending` ->
`planned` (4), collapsing singletons into the dominant spelling.

MEASUREMENTS, and one correction to the issue
---------------------------------------------
  466 errors  ->  282  (drop the source overlap: 190 duplicates -> 6)
              ->  276  (delete the 6 RENDER-REQ stubs: 6 -> 0)
              ->  276  (normalise the vocabulary: NO CHANGE)

Duplicate-id errors are now 0. But #360 predicted that with the overlap and the
vocabulary both fixed "the residual error count should be near zero", and that
is FALSE. The normalisation cannot reduce the count, because its targets are not
in the allowed set either:

  allowed: draft, proposed, approved, implemented, verified, released,
           accepted, deprecated, rejected

`passing` and `planned` are both absent, so `pass` -> `passing` moves 5 errors
between buckets (172 -> 177) and `pending` -> `planned` moves 4 (36 -> 40). Net
zero, as measured. It is still worth doing: it leaves 4 non-conforming values to
resolve instead of 6. It is just not an error fix, and recording it as one would
have overstated what this change buys.

The residual 276 are entirely status-vocabulary skew: 177 `passing`, 51
`not-implemented`, 40 `planned`, 8 `partial`. These are test/verification states,
not approval-lifecycle states, and `common@0.3.0` is embedded in the rivet binary
-- schemas/ only overrides sysml2 -- so the repo cannot widen them locally.

This does NOT unblock lifting the v0.4.3 pin in ci.yml; see #360 for the
measurement. The pin is load-bearing for the green check, which is a fact worth
stating plainly rather than leaving implicit in a comment.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… test-names

The equivalence check this PR cited was measured on `rivet list --format json`
WITHOUT `--full`, which emits only the summary view (`--help` states it omits
`description`, `tags`, and `fields`). That view is structurally incapable of
detecting field-level loss — precisely the class of change a `format:` switch
produces. "Byte-identical" was true and meaningless.

Re-measured with `--full` on base vs. this branch:

  +86 artifacts GAIN `mitigates` and `traces-to`
   -5 artifacts LOSE half of `test-name`

The five are VAL-010, VAL-011, VAL-014, VAL-015, VAL-021 — exactly the five
whose on-disk `test-name` is a comma-separated pair. The stpa-yaml parser keeps
only the first element:

  on disk: tests::boolean_matches_aadl_boolean, tests::boolean_mismatches_aadl_string
  rivet:   tests::boolean_matches_aadl_boolean

A/B on identical bytes with only `format:` differing confirms the parser is the
cause: generic-yaml round-trips the full string.

The trade is still worth taking — five truncated evidence pointers against 190
phantom duplicate-id errors — but it is a trade, not a free win, and the comment
now says so. Filed upstream as pulseengine/rivet#747; revert the trade if that
lands a fix.

No behaviour change: `rivet validate` is FAIL (276 errors, 629 warnings) before
and after this commit. Comment-only.

Found by cold-context adversarial verification of this PR's own claims.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spar carried a private status vocabulary — `passing`, `planned`,
`not-implemented`, `partial` — on 276 artifacts. None of the four is in the
schema's allowed set, so every one was a `status-allowed-values` error and
`rivet validate` could not run clean.

The mapping, and what it costs:

  passing         -> implemented   (177)
  planned         -> proposed
  not-implemented -> proposed
  partial         -> approved      (8, lossy)

`passing` was the largest group and the one worth naming. It is a verification
VERDICT sitting in a lifecycle field, and a hand-edited YAML file cannot
honestly assert that a test passes — only that the measure exists. That is what
`implemented` says.

Only `partial` loses information: it sits strictly between `approved` and
`implemented` and the vocabulary has no such rung. `approved` is chosen as the
conservative floor — the strongest claim certainly true — over `implemented`,
which would overstate. All eight say so in their own description. Two
(RENDER-REQ-003, COVERAGE-GAPS) already itemized their residue and were left
alone; the other five state the residue is NOT itemized. That wording is
deliberate — an explicit "not measured" is falsifiable, an invented figure
("32 of 50 rules") is not, and inventing one is the exact failure #294 was
reopened for.

No new guard ships with this. rivet already enforces `status-allowed-values` at
ERROR severity and `rivet validate` is already required, so the regression is
blocked the moment this lands; a bespoke checker would only add a weaker second
oracle beside a working one. That is the opposite of REQ-GUARD-RELEASE-PLANE-001,
which needed a tool precisely because rivet was blind to the defect.

Verification, on three planes that agree:

  - the migration asserts per file that the artifact-id set is unchanged, that
    every record is byte-identical outside `status`, and that the text-rewrite
    count equals the semantic-move count. That last one matters: "N lines
    rewritten" is equally consistent with having rewritten N *prose* lines inside
    `description:` blocks while leaving N real statuses alone.
  - `rivet validate` exits 0 with 0 error-severity diagnostics under
    common@0.3.0 (embedded), down from 276.
  - the gate is live, not vacuous: re-introducing one off-vocabulary status
    reproduces exactly one `status-allowed-values` error, and restoring it
    returns exit 0 with a byte-identical file. Distinct inputs, distinct
    verdicts — the property #327's withdrawn certificate did not have.

Reconciliation of the raw text count against rivet's: 303 `status:` matches
across 9 committed files, minus 21 in safety/requirements.yaml (which rivet
never loads — it is under no path in rivet.yaml's `sources:`), minus 6 in
safety/stpa/architecture.yaml (nested in the `fields:` extension bag, where
rivet cannot see them) = 276. Both remainders are left in place and named in
REQ-GUARD-STATUS-VOCAB-001 rather than silently absorbed; the second is the #370
defect one field over, on `status:` instead of `release:`.

Addresses #371.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@avrabe

avrabe commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Correction to the PR body: the base is now main, not fix/360-rivet-source-dedup.

I opened this stacked on #368 for a clean diff. That was wrong, and wrong in a way worth recording because it is the exact defect class #372 is about.

ci.yml triggers on pull_request: branches: [main]. Based on another PR's branch, no workflow triggered at alltotal_count: 0, zero runs. The PR presented as clean; tot=0 pend=0 bad{none} is indistinguishable from "all green" in every surface I check. I nearly reported this as awaiting-green when it was awaiting-nothing.

Retargeting to main did not fix it either: GitHub's default pull_request types are opened, synchronize, reopened, and changing a base is edited. It took a close/reopen to fire the event. Now at 7 check-runs across CI, Lean Proofs and Verification Gate.

Consequence for review: until #368 merges, this PR shows 3 commits#368's two plus this one — because that is genuinely the diff against main. Once #368 lands, GitHub recomputes and only a3512eb remains. The rivet validate exit-0 result holds either way; it was measured on top of #368's tree, which is what this branch contains.

Written up in #372 as an argument that the proposed standing check should also assert every open PR has a non-zero required-context count — no amount of auditing ci.yml catches a PR that never triggered it.

@github-actions

Copy link
Copy Markdown

Rivet verification gate

20/20 passed

count
Passed 20
Failed 0
Skipped (no steps) 0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verification verdicts are recorded as feature.status — the root cause of all 276 rivet validate errors

1 participant