Skip to content

fix: self-improvement instruments report what happened (trace-corpus guards, failure classes, selftest could-not-judge) - #854

Merged
dmmdea merged 2 commits into
mainfrom
fix/rsi-instruments
Nov 12, 2026
Merged

dmmdea merged 2 commits into
mainfrom
fix/rsi-instruments

Conversation

@dmmdea

@dmmdea dmmdea commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Intent

The self-improvement instruments (trace corpus, failure register, gate self-test) must report what happened. An audit that ran every free verb against real data found the statistics sound and four places where an instrument misreported: a frozen baseline could be overwritten silently, impossible dates were accepted, a seat that could not answer read as a scorer that mis-ranks, and 154 of the 157 other rows in a 30-day failure register were capacity and placement defers whose class the door had dropped.

Scope

  • trace-corpus argument and freeze guards, gate selftest's could-not-judge path, the failure register's writer stamps and read-time rules, usage text, .gitattributes for the embedded gate schema, and the three system pages.
  • Left out: stamping build and seat-config hashes into ledger rows (the audit's answer to "can a failure be tied to a build" is no; that is a schema decision), any new failure class, the recipe proposer, and every A/B path (compactionOutcomeScorer is unchanged).

What changed

  • trace-corpus. freeze creates its baseline exclusively and refuses an existing path (exit 2). harvest --since/--until must be real YYYY-MM-DD dates with since on or before until (exit 2). help, --help and -h print the usage and exit 0.
  • Failure classes, written. The remote lanes, the composite image row and the accelerator forward stamp the reason code from the defer class the result carries (ledger.CodeForLaneDefer: error class and the harness's own sentence first, the wire class when neither names the row). A deferred row with no reason is filed as deferred: no reason recorded.
  • Failure classes, read. Sentence rules for rows written before the stamp, each a sentence one writer owns that maps to one existing class. A deferred row stamped other is read through the same rules, because other is the stamp for "no member named it"; every other stamp is read as stamped.
  • gate selftest. A scorer whose seat call deferred or failed could not judge: exit 2, no receipt, an existing --out left alone, the reason named. It used to print a receipt with passed: false and replace the file.
  • Small fixes. ledger and recipe-audit usage text; LF forced on internal/eval/*.json so gate schema prints the same bytes on every checkout.
  • Docs. The self-improvement, measurement-gate and failure-register pages match the code; four pages no longer cite a docs lint test that does not exist.

The real run and its output

Built binary against a real delegation log and ledger:

trace-corpus --help                                         exit 0, usage with the flags
trace-corpus harvest --since 2026-13-45                     exit 2  --since "2026-13-45" is not a real YYYY-MM-DD date
trace-corpus harvest --since 2026-11-05 --until 2026-11-01  exit 2  --since 2026-11-05 is after --until 2026-11-01
trace-corpus freeze (first)                                 exit 0  (1,068 harvested runs)
trace-corpus freeze (same --out again)                      exit 2  refused: base.json already exists; sha256 unchanged; check still reproduces it
ledger --failures --since 30     installed: other 157 of 2,520   this build: other 55 (no_eligible_node 1 -> 76, queue_full 22 -> 49)
ledger --failures (whole ledger) installed: other 4,949 of 10,900 (45.4 %)   this build: 1,160 (10.6 %)
gate selftest, seat unreachable  installed: exit 1, receipt passed:false, --out replaced
                                 this build: exit 2, no receipt, --out sha256 unchanged, "the seat call deferred: ..."

Local gate: go build, go vet ./... clean; internal/ledger and internal/pairworkloads pass; the root and internal/pipeline subsets that cover the touched code pass (trace-corpus, gate, failure register, ledger, recipe-audit, leak gate, changelog fragments; accelerator, defer and reason rows). The full root and pipeline packages did not finish on the build box inside the 10-minute timeout, so CI is the gate for the rest.

Not exercised: a real accelerator or lane defer written by the new stamp (checked by build, vet and the existing ledger and pipeline tests), and the self-test under a held GPU lease (an unreachable seat takes the same code path).

Blast radius

Readers of ledger --failures: class counts move out of other into existing classes, and class_derived_rows rises. Callers of trace-corpus freeze that re-froze onto an existing file now get exit 2. Callers of gate selftest that treated exit 1 as "seat busy" now get exit 2 for that case. No config key, no wire format and no ledger schema changes.

Risk

Low. One new ledger behaviour on the write path (the fixed placeholder on a deferred row with an empty reason); the rest is read-side classification and verb guards.

🤖 Generated with Claude Code

dmmdea and others added 2 commits November 12, 2026 02:35
…guards, failure classes, selftest could-not-judge)

trace-corpus: freeze creates its baseline exclusively and refuses an existing path; harvest
validates --since/--until as real dates with since <= until (exit 2); help/--help/-h print the
usage with the flags and exit 0.

Failure register: the remote lane writer, the composite image row and the accelerator forward
stamp the reason code from the defer class the result carries; a deferred row with no reason is
filed with a fixed placeholder; legacy rows are classed by the harness's own sentences (wall
timeout, step budget, queue deadline, stalls, failed verification, schema, seat down, lane
placement, 503/429 dispatch refusals).

gate selftest: a deferred or failed scorer seat call is "could not judge": exit 2, no receipt,
an existing --out left alone, the defer reason named. The A/B path is unchanged.

Also: ledger and recipe-audit usage text, LF for the embedded gate schema, and the
self-improvement, measurement-gate and failure-register pages (plus the docs that cited the
deleted docs lint test).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…" through the sentence rules

"other" is what Record files when no member named the row at write time, so it is the one stamp a
later build may read better. Every other stamp is still read as stamped. On a real 30-day register
the other count goes from 157 to 55 (no_eligible_node 1 to 76, queue_full 22 to 49).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@dmmdea
dmmdea merged commit 1dc59ca into main Nov 12, 2026
6 checks passed
@dmmdea
dmmdea deleted the fix/rsi-instruments branch November 12, 2026 07:55
@dmmdea dmmdea mentioned this pull request Nov 12, 2026
dmmdea added a commit that referenced this pull request Nov 12, 2026
One change, merged to main through its own pull request (#854) after one
security pass (nothing to fix) and green CI:

- trace-corpus: freeze creates its baseline exclusively and refuses an
  existing path; harvest validates --since and --until as real dates;
  --help exits 0.
- Failure register: the remote lanes, the composite image row and the
  accelerator forward stamp the reason code from the defer class the
  result carries; rows written before the stamp, and deferred rows
  stamped "other", are classed from sentences the harness itself wrote.
  On one real 30-day register the "other" count goes from 157 to 55.
- gate selftest: a scorer whose seat call deferred or failed could not
  judge: exit 2, no receipt, an existing --out left alone.
- ledger and recipe-audit usage text, LF for the embedded gate schema,
  and the self-improvement, measurement-gate and failure-register pages.

Deletes: changes/unreleased/fix-rsi-instruments.md

Co-authored-by: dmmdea <dmmdea@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant