Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/commands/pharn-verify.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ test-infra` SPEC gets only the weaker BOOTSTRAP evidence (each level's gate ran
- **The absolute-threshold residual:** the verdict asks "are ALL gates green NOW?", not "did anything flip?". A
pre-existing red gate the feature did not cause also fails verify, by design — `/pharn-regress` is the stage that
excludes pre-existing failures. The feature-specific signal is the `structural:*` gates over the feature's evals.
- **The reuse residual (6.34.0):** inside a `/pharn-loop` or `/pharn-ship` run, a gate's result may be REUSED from that
run's `/pharn-regress` HEAD execution over the same tree and execution identity, not re-run here; the report's
`gate_reuse` block names each. That the result equals a fresh run is advisory, and this verify then gives no second
sample of a flaky gate (`pharn/floor/gate-reuse-core.mjs`).

## Step 0 — Resolve `<name>`, then set the writes-scope (fix #7, fail-closed)

Expand Down
87 changes: 87 additions & 0 deletions .dev/features/verify-head-gate-reuse/GRILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# GRILL — verify-head-gate-reuse

Plan: `.dev/features/verify-head-gate-reuse/PLAN.md`. Spec-hash check: `node .dev/floor/hash-doc.mjs
pharn/ARCHITECTURE.md` → `d831d30d399a37dc403080072763d13383de6f6f31875e7e8cb4eadeb642f4f4`, equal to the plan's
`spec_content_hash` (no drift). **Step 1b (FLOOR):** `node pharn/floor/check-plan-lessons.mjs
.dev/features/verify-head-gate-reuse/PLAN.md .dev/memory-bank/lessons-learned.md` → exit **0** (`GREEN — applied_lessons:
L24, L29, L34, L35, L41, L43, L45, L54, L59, L60, L62, L65 … all 12 cited id(s) resolve … and are referenced in the plan
body`).

**How this grill was run (recorded, advisory).** An independent, read-only Opus agent with a fresh context interrogated
the plan against the live code (it wrote nothing and ran no setter). It noted that the build had already started in the
worktree and spot-checked the in-progress diff where a finding bore on it. The orchestrator wrote this file from its
report. The agent's free text below inherits the plan's untrusted tag and is quoted as DATA (P2). The 13 registered
grillers (`node pharn/floor/count-grillers.mjs .`) were applied by axis; a11y, i18n, privacy and migrations do not apply
(no UI, no user-facing strings beyond fixed render lines, no personal data, no data migration).

## Findings

```yaml
- type: FINDING
rule_id: "P0"
severity: blocking
file: ".dev/features/verify-head-gate-reuse/PLAN.md:35"
problem: "Style gates are planned reusable, but a whole-tree style gate reads files the fingerprint excludes, and those files differ between the regress HEAD run and verify: REGRESSION.md/regression-report.json are written after the head drain, and a previous iteration's VERIFY.md/verify-report.json exist at regress time and are removed by verify's fresh start."
evidence: "worktree-fingerprint.mjs EXCLUDED_ARTIFACTS — 'POST-BUILD artifacts: written at or after regress, so a change cannot alter what a gate judged'; stage-regress.mjs renders REGRESSION.md after drain-head."
- type: FINDING
rule_id: "P0"
severity: blocking
file: ".dev/features/verify-head-gate-reuse/PLAN.md:238"
problem: "Under /pharn-ship --quick and /pharn-loop --quick no regress runs but the run marker is open, so the build agent can Write .pharn/pharn-regress/head/stamp.json and its logs (ALWAYS .pharn/**) after the marker; marker + mtime acceptance then offers a planted source, and verify would record gate exits nobody ran."
evidence: "enforce-writes-scope.cjs: const ALWAYS = ['.pharn/**']; the plan's L65 line claims regress clears head/ first, which quick mode never runs."
- type: FINDING
rule_id: "P0"
severity: important
file: ".dev/features/verify-head-gate-reuse/PLAN.md:104"
problem: "validateStamp would admit ANY id as reused, including reconcile and the AC level gates."
evidence: "the in-progress reusedRunDefect checks stage/shape but no id."
- type: FINDING
rule_id: "P0"
severity: important
file: ".dev/features/verify-head-gate-reuse/PLAN.md:200"
problem: "'Recorded only when its identity equals the live one' is floor at decision time only; nothing reading the verify stamp later re-derives it, and existing text says the map's values are exit codes the runner recorded from the listed argv."
evidence: "gate-run-core.mjs header lines 27-31; CLAUDE.md run-gates paragraph; gate-run-record.md 'What a validating stamp PROVES'."
- type: FINDING
rule_id: "P0"
severity: important
file: ".claude/commands/pharn-verify.md:56"
problem: "The command's claims, and LIMITS.md ('/pharn-verify re-runs the project's own gates'), overstate a reused gate; the new residual (no independent second sample of a flaky gate) is unstated; the command's byte ceiling leaves 916 bytes."
evidence: "pharn-verify.md 17,516 bytes vs COMMAND_BYTE_CEILINGS 18,432; LIMITS.md is human-only."
- type: FINDING
rule_id: "P0"
severity: minor
file: ".dev/features/verify-head-gate-reuse/PLAN.md:61"
problem: "Git HEAD (and the index) are outside the identity; a gate that reads git state is unbound."
evidence: "worktree-fingerprint.mjs: HEAD is deliberately NOT part of the digest."
- type: FINDING
rule_id: "P0"
severity: minor
file: ".dev/features/verify-head-gate-reuse/PLAN.md:215"
problem: "The budget check runs before run --next knows an entry is a HIT, so reuse saves processes and wall-clock, not Bash calls or model turns; the measurement must use the pinned --budget-ms."
evidence: "stage-runtime.mjs drainGates: budget.may() precedes the spawn; 540000/570000 allow a second slow step only within 30 s."
- type: FINDING
rule_id: "P0"
severity: minor
file: ".dev/features/verify-head-gate-reuse/PLAN.md:98"
problem: "results_sha256 === null means 'no REGULAR results file', not 'wrote nothing' (a directory, link or FIFO there also hashes null); e2e reads build output, so a reused build leaves e2e reading the head run's git-ignored output; the usage strings and CLAUDE.md lines need the new flag."
evidence: "run-gates.mjs sha256RegularFile returns null for a non-regular file; gate-run-core.mjs E2E_SET comment."
```

## Checked, no issue (the agent's own list)

cwd is the invoking directory on both sides, so its realpath is stable; leaving the value of `PHARN_TEST_RESULTS`, `seq`
and `<out>` out of the identity is correct; `timeout_ms` in the identity does not cause misses (both pinned 540000); a
crash between the log copy and the state write re-decides the entry (spawnGate truncates the logs, `clearResultsPath`
runs first); a freshness RERUN of verify misses by fingerprint and a `/pharn-ship` Step 2b retry re-runs regress, which
clears `head/`; the BASE-reuse increment is unaffected (`baseSpecFrom`/`specPart` project only `{id, shell, argv,
files}`); `check-verify`, `check-loop-fresh` C/E/F/G/J and `ac-gate-core`'s `ranAsPinned` do not branch on `ran` for a
non-level gate, and J re-hashes the copied logs under verify's own names; the stage-verify fixture closure is
regex-derived, so new literal `./x.mjs` imports are covered.

## Verdict (ADVISORY)

**NOT READY as planned — 2 blocking, 3 important, 3 minor.** The orchestrator accepts every finding and amends the plan
before the build continues (`PLAN.md`, "Grill amendments"): style gates join the never-reused set; the source is bound
by a record in the git dir that `/pharn-regress` publishes for the current run and verify requires; validateStamp
refuses a reused non-reusable id; git HEAD joins the identity; the wording and the residuals are corrected. The Step 1b
floor verdict above is GREEN and is not folded into these counts.
61 changes: 61 additions & 0 deletions .dev/features/verify-head-gate-reuse/MEASUREMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# MEASUREMENT — verify-head-gate-reuse (2026-09-28)

Measured on the maintainer's machine (darwin, Node 24), with `.dev/features/verify-head-gate-reuse/measure.mjs`, 3
repetitions per variant, the machine otherwise idle. Each repetition builds a FRESH fixture repo carrying the floor
under test as its own `pharn/floor/` (an install's layout) and the write guards, opens a `/pharn-loop` run marker,
anchors the reconcile epoch as the build does, makes the build's edit, then runs `stage-regress.mjs` and
`stage-verify.mjs` with the pinned flags (`--timeout-ms 540000 --budget-ms 570000`; regress also `--base <sha>
--no-install`).

**The fixture.** Four gates: `test` (node:test, one test, 0.3 s), `lint` (0.8 s), `typecheck` (1 s), `build` (1 s).
Every gate appends one line naming itself and the stage it ran under (read from `PHARN_TEST_RESULTS`, whose value is
that gate's own path under the stage's `<out>`) to a counter file outside the repo. The build edit touches
`eslint.config.mjs`, so regress runs its style gate too — the widest overlap a discovered set has. The COUNTS are the
result; the wall-clock shows what the counts save on this fixture's shape, not what any real project saves.

- **before** — `main`'s floor at `2cf0e85` (`git archive 2cf0e85 pharn/floor`);
- **after** — this branch's floor.

## Counts (identical in all 3 repetitions)

| variant | regress/HEAD processes | regress/BASE processes | verify processes | verify entries reused | total gate processes |
| ------- | ---------------------: | ---------------------: | -------------------------------- | -----------------------: | -------------------: |
| before | 4 | 4 | 4 (test, lint, typecheck, build) | 0 | 12 |
| after | 4 | 4 | **2** (test, lint) | **2** (typecheck, build) | **10** |

`reconcile` (PHARN's own gate, always last at verify and never reused) is not counted above; it runs in both variants.
`test` runs at both stages because it is never the same execution (regress: the outside-scope subset; verify: the
whole suite) and is an AC level gate; `lint` runs at both because style gates are never reused (grill B1). In the
common case where the build touches no style config, regress skips `lint` by design, so regress spawns 3 + 3 and
verify still spawns the same 2 — the reused pair is the same.

Both variants' verdicts are `PASS` in every repetition, and the after-variant's `verify-report.json` reads
`gate_reuse.reused = [typecheck, build]`.

## Wall-clock (ms, per repetition)

| variant | regress (whole stage) | verify (whole stage) |
| ------- | --------------------- | -------------------- |
| before | 9822 / 9827 / 9763 | 4815 / 4833 / 4885 |
| after | 9738 / 9755 / 9843 | 2506 / 2465 / 2508 |

- **Verify drops by ~2.35 s (−49%) on this fixture** — the two reused gates' own time (2 × 1 s) plus two `npm run`
start-ups. Everything else verify does is unchanged: the chain check, the pairs, the verifier count, the
completeness capture, `test`, `lint`, `reconcile`, two fingerprints per SPAWNED gate, the verdict and the render.
- **Regress is unchanged within noise** (−89 ms to +80 ms): it publishes one small offer record once its HEAD stamp is
final, and records an identity per run.
- **The offer decision's own cost, timed in-process over the finished fixture (20 calls), is 16.9–17.8 ms**: one `git
rev-parse --absolute-git-dir` spawn, reading the markers, the offer and the head stamp and hashing them, plus one
`findReusable`. It runs once per verify invocation. The per-entry cost inside `run --next` (re-reading and hashing
the source stamp, the decision, reading, hashing and writing two logs) is inside the verify wall time above and
replaces a gate spawn plus its post-gate fingerprint.

## Not measured, and not claimed

- **Tokens.** The stages are deterministic code; the orchestrator's calls per verify are unchanged, and a reused entry
still spends one budgeted `run --next` call (so reuse saves processes and wall-clock inside an invocation, never a
Bash call or a model turn).
- **A real project's saving.** It scales with the duration of the project's `typecheck` / `type-check` / `build` (and
any structural pair both stages run). The fixture's 1 s gates stand in for minutes in a real project.
- **How often a real run offers.** Only inside a `/pharn-loop` or `/pharn-ship` run whose regress published an offer
and whose tree did not move between regress and verify; every other verify runs every gate, as before.
Loading
Loading