Skip to content

feat(floor): cost.json shows stage elapsed time and deterministic work per run (6.35.0) - #299

Merged
PrzemekGalarowicz merged 1 commit into
mainfrom
run-performance-breakdown
Sep 28, 2026
Merged

PrzemekGalarowicz merged 1 commit into
mainfrom
run-performance-breakdown

Conversation

@PrzemekGalarowicz

Copy link
Copy Markdown
Contributor

Summary

A maintainer can now answer, for one concrete /pharn-loop or /pharn-ship run, where the model usage went, where the observed wall-clock time went, which expensive deterministic operations ran, and which were avoided by the 6.33.0 / 6.34.0 reuse mechanisms — from cost.json, with no second telemetry system.

  • executions (a VIEW, no new instrumentation): derived from the existing phase markers (pharn/floor/stage-executions-core.mjs, method stage-start-to-return/1). Every current-run stage-start is one row, ended only by the next marker when that marker is the orchestrator's return. Re-runs are run 2, never merged. Anything else is unmeasured with a closed reason and elapsed_ms: null — never 0, never a guessed end. It is labelled everywhere as observed wall clock, not CPU, model or tool time, and not monotonic.
  • work[] (FACTS captured at the moment): /pharn-regress and /pharn-verify each append one line at done to .pharn/cost/<feature>/work.jsonl (pharn/floor/stage-work.mjs). The counts come from the gate-run stamp the verdict used: executed / reused / nothing-to-run / required, BASE evidence fresh|reused, and the install's exit and ms, which is the only new timer. Per-iteration capture is needed because both reports are overwritten every iteration. The append is observational and best-effort: no exit, verdict, reuse, route or commit depends on it.
  • Checker rule 9: validates work[] rows and their run-window membership, and recomputes executions from the file's own facts. /2 admits exactly the current key set or the pre-6.35.0 one. No schema bump.
  • Run report and stop table: three separate blocks — tokens, elapsed, work. Nothing is subtracted from anything else.
  • Unchanged: mark-phase.mjs and its printed binding line, run membership, context binding, dedup, stage attribution (✧ equality test), and --verify-transcript.

SKILLS_VERSION 6.34.0 → 6.35.0. MIN_CLI stays 0.5.0. Contract: pharn/pharn-contracts/cost-ledger.md, "Stage executions and deterministic work".

Evidence

  • .dev/features/run-performance-breakdown/DEMO.md — a controlled end-to-end fixture (not a live run). It uses the real marker writer, work-record writer, emitter, checker and run report, and answers the five questions from cost.json. It also reports the measured overhead: under 1 ms added to render and check on a 126-request run, one extra small file read, no transcript pass, no process, no model call.
  • The real stage scripts are exercised in stage-regress.test.mjs (★ HIT, the budgeted chain) and stage-verify.test.mjs (★ EQUIVALENCE, OBSERVATIONAL). Those tests assert the work counts against the processes the fixtures counted spawning.

Pipeline verdicts (/pharn-dev-ship)

  • plan lessons check: exit 0. validate: GREEN.
  • regress: no-regressions. The first regress attempt was void — it read another session's stale result files in shared .pharn/ scratch. It was redone from run-private scratch; see REGRESSION.md, "A void first run" (promoted as L66).
  • verify: raw FAIL on lint:md only. The single offending file is .pharn/pr-body.md, another session's gitignored scratch file, not part of this PR. Measured on a clean copy of the tree, lint:md exits 0 and the verdict over the same map is PASS; VERIFY.md carries both. npm test: 4502/4502. Reconcile: CLEAN.
  • Independent opus review (REVIEW.md): nine findings. Two were major: a FIFO planted at work.jsonl could hang a stage before done, and an unknown run window read as "no work". All are fixed with tests, or stated as bounds, before verify ran.
  • GATE 1 was delegated to the model by the maintainer's prompt; the merge decision is the maintainer's.

Named residuals

gate-process-duration (no per-gate timer), ship-spec-elapsed (/pharn-ship marks no spec stage-start), work-on-non-done-exit, work-record-provenance.

🤖 Generated with Claude Code

…k per run (6.35.0)

One /pharn-loop or /pharn-ship run's cost.json now carries, beside its unchanged
model usage, two additive keys:

- `executions`: a VIEW over the existing phase markers (stage-executions-core.mjs,
  method stage-start-to-return/1). Each current-run stage-start is one row, ended
  only by the next marker when it is the orchestrator return; re-runs are `run 2`;
  anything else is unmeasured with a closed reason and elapsed_ms null, never 0.
- `work[]`: facts /pharn-regress and /pharn-verify append at `done`
  (stage-work.mjs): gate processes executed / reused / nothing-to-run / required,
  BASE evidence fresh|reused, and the install's exit and ms (the one new timer).
  Best-effort and observational: no exit, verdict, reuse, route or commit reads it.

check-cost-ledger rule 9 validates work rows and recomputes executions; /2 admits
exactly the current key set or the pre-6.35.0 one (no schema bump). The run report
and the stop's table show tokens, elapsed and work as three separate blocks.
mark-phase.mjs, its printed binding line, run membership and stage attribution
are unchanged. Promotes lesson L66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 28, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 50979471-2494-4f41-bb98-35f78103e78a


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@PrzemekGalarowicz
PrzemekGalarowicz merged commit c9737b4 into main Sep 28, 2026
8 checks passed
@PrzemekGalarowicz
PrzemekGalarowicz deleted the run-performance-breakdown branch September 28, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant