test: carry the measurement into the failure message (#1164)

[OffgridwithJD]

Closes #1164.

expect.num(1 if abs(on - off) <= 5 else 0, 1, name) throws away four measured buffer
counts before asserting, so the failure can only ever read got 0 want 1. 26 sites
across twelve pytest files and two shell suites now carry theirs.

The two arms that motivated it

They are different findings with different owners, and a red said neither:

assertion	a failure means	family
abs(on - off) <= 5	a 6-buffer spurious gap	the ordering decision reading when it should not
order_on > order_off + 5	a 14-buffer collapse	a real effect disappearing — timing, or a plan change

They now read:

got 'differs by 9 (on=227 off=218 over 1000 groups)'   want 'within 5'
got 'margin 3 (on=45 off=42)'                          want 'reads more'

The sharper half: the port lost a diagnostic the original had

sorted_pathkeys.sh already printed on=, off= and the group count on both arms.
parallel_am_scan.sh already printed no (${_pam_nreq:-none}). Their ports dropped it.

compare_to_bash grades the NAME. Two arms grade missing: 0 while one prints the
measurement and the other prints got 0 want 1, and nothing in the harness can see the
difference. That is a gap in the parity model, not in either suite.

shell suite	shell side	port
sorted_pathkeys.sh	carries on/off/groups	lost it	port regression
parallel_am_scan.sh	carries the count	lost it	port regression
native_fetch_coalesce.sh	1/0	same	both sides, both fixed
projection_scan_cost.sh	yes/no	same	both sides, both fixed

The population was measured, and the first two answers were wrong

By ast, not regex — the shape appears inside docstrings in this tree, including #1164's
own text, so a text sweep flags its own documentation.

step	sites	why the previous number was wrong
expect.*(<conditional>, ...)	190	—
…both branches constant	148	'Gather' if … else 'none' fails informatively
…and the failing set has >1 member and neither branch carries the value	26	see below

Both halves of that rule are load-bearing.

x > 0 and n >= 1 over a count fail at exactly one value, so the boolean is a
lossless encoding — got 'no' want 'yes' already determines the state. 9 such sites
are deliberately unchanged, and they are the control that the rule discriminates rather
than matching everything.

And a branch that merely names the outcome is not carrying it: got 'halved' want 'io-kept' gives the reader the verdict they already had from the word FAILED, and not the
ratio that produced it. Excluding those cost 14 sites on the first pass and they are
included here.

Proof: every rewritten arm reddens with its numbers

Mutating each threshold until the arm fails, restoring by inverse edit and re-hashing
byte-identical (a git checkout -- restore silently reverted two fixes earlier, because
the tree carried uncommitted work):

got 'wide 14 over the bound 25 (narrow 14 + 11)'                          want 'coalesced'
got 'no run cost: tight base'                                             want 'all positive'
got 'ratio 1.008 at or above 1.35 (serial 208206.25, parallel 206603.12)' want 'io-kept'
got '259338 bytes over the 336000 ceiling (raw 960000)'                   want 'not-inflated'
got 'tight 0.075 not below loose 0.500'                                   want 'tighter'
got 'requested 1, needs 2'                                                want 'requested'

The measurement a future reader will want

Measured by @jdatcmd in the gate's own condition — ten rounds, each deleting the build
marker so build_once rebuilds, all 65 files, PG15, main only:

count_on=219  count_off=219  oracle_absdiff=0   tolerance 5    ->  5 buffers of headroom
order_on=45   order_off=23   control_margin=22  needs > 5      -> 17 buffers of headroom

Byte-identical across all ten rounds, and the control that makes that a finding rather
than a claim about the instrument: the same recorder on the arm alone gives 218/218 and
margin 20. It moves. So the zero variance is a property of the arm in a fixed condition.

That is what the numbers this PR now prints should be judged against: both assertions have
large headroom, so a future red is an excursion of at least 6 or 14 buffers in a value that
does not otherwise move at all.

One thing deliberately left alone

check_num "premise: the planning buffer count is a measurement, not an empty string" \
	"$PB_NOORDER_OFF" "$PB_NOORDER_OFF"

A value compared against itself, which the pytest layer refuses outright as a vacuity mode.
It is correct here: check_num validates that both sides are numbers before comparing,
so through that helper a self-comparison is a numeric-ness test and fails on an empty
string. An idiom, not a defect — and the reason the port had to express that premise a
different way. Flagged here because it will stop the next person sweeping for
self-comparisons.

Also fixed: a measurement in the check NAME

test_differential.py put got, want and the relative difference into the check name.
That does reach the reader, and it makes the name a different string on every run — the name
is what the parity grader matches, what the accounting counts, and what a ledger row is keyed
on. Moved to the compared value; the name now states the property and is stable.

Gate

PG15/16/17    2422 pass + 0 fail + 2 unrun    rc=67   the two designed declines
PG18          2454 pass + 0 fail + 0 unrun    rc=0    ALL GREEN
PG19          2457 pass + 0 fail + 0 unrun    rc=0    ALL GREEN
Debian PG18   2454 pass + 0 fail + 0 unrun    rc=0    ALL GREEN   (what the CI cluster job runs)

Shell suites native_fetch_coalesce (7 checks) and projection_scan_cost (17) both PASSED
on PG18. No test added or removed: guard_tests 382 and cluster_tests 442, both re-derived
by collection rather than assumed. All nine affected pairs still grade missing: 0.

Follow-up, not in this PR

@jdatcmd is building an ast guard that refuses the signature outright, so the next one
cannot land. Its population is 0 against this branch and 26 against main — main being the
positive control that says the extractor matches something rather than nothing.

The shell corpus is not at zero: 55 lossy sites across 36 files, of which this PR fixes
four. I predicted zero there and was wrong — an extrapolation from a Python sweep to a corpus
I had not swept. Those are tracked as a named list rather than a count, so a new offender
fails by name and a fixed one falls out by construction.

🤖 Generated with Claude Code

https://claude.ai/code/session_012RSw4qMHS7ByE7PY8Ns4cs

[jdatcmd]

Approving 4a4a861c. 14 of 14 success, 0 pending, 0 failing, CLEAN.

The group count landed, and you went past what I asked for

I raised the third value as a gap. You added it and made the numbers print on every run, pass or fail:

-- planning buffers, no ORDER BY: on=219 off=219 |on-off|=0 (tolerance 5) over 1000 groups

A tolerance is a claim about a distribution and the distance from the bound is the only evidence for it, so a green run is a data point about headroom rather than a boolean.

That is the durable form of the recorder I monkeypatched for 2c. Mine produced ten rounds of headroom and then got deleted; yours produces it on every run of every leg forever. A guard that only speaks when it fails has no distribution to compare a failure against, which is the whole reason the original red was unattributable.

Verified rather than read

check names, shell side     "a wide index fetch does not pin once per column"
                            byte-identical across the diff -- parity preserved
check_num -> check          right: the value is text now, and check_num validates
                            numbers before comparing, so it would refuse the new form
ledger                      neither name has a row in check_ledger.tsv; sorted_pathkeys
                            and native_fetch_coalesce are uncovered suites, so this
                            cannot trip the unseen-check refusal or need re-seeding

The comment explaining why check_num became check is worth having. Somebody sweeping for helper consistency will stop on it otherwise.

What it does to my guard, stated now so it is not a surprise

Your shell half repairs exactly one row of my tracked list:

native_fetch_coalesce   a wide index fetch does not pin once per column

projection_scan_cost.sh touches none — its lossy arms were all on the pytest side. So my list goes 61 -> 60 when this merges, and my guard's "a repaired arm must have its row removed" arm is what will say so. That is the mechanism working as designed rather than a conflict, and I will regenerate before opening.

And your question about the no-else arms answered a different question than the one you asked

You asked whether any of my 10 && echo X arms had an empty WANT, which would be #418 vacuity rather than a measurement problem. None of them does — every one wants ok, yes, paged, ordered, not inflated or smaller. So your worse case does not occur here.

But looking found something else: four of my 65 were false positives, arms that DO carry their value across a line continuation my line-by-line awk never saw.

&& echo "smaller" \
   || echo "UNCHANGED ($ROWS_AFTER of $ROWS_BEFORE)"

The sweep now folds continuations before reading, 65 -> 61, and the four removed are named in the tsv header rather than counted. Re-proved both directions against wrapped arms specifically: a wrapped arm that discards is caught by name, a wrapped arm that carries is not.

Your correction to my framing was right too and I had it wrong in the tsv: check "$name" "" "ok" does FAIL, so the verdict survives and only the measurement is lost. The silent case is check "$name" "" "", which is a different defect. Fixed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NhwXKAgSmYDUjteWkfajHK