Repository navigation
Commit 47bfcd0
perf(pm): discover each tree once per process — the dispatch-gates self-test now reaches a verdict inside the foreground cap (#18255)
Fixes #18201
## The profile, which is what the card asked for first
`pnpm check:pm-dispatch-gates` was measured end to end on `b3b43b6e`,
detached, with every
case line timestamped. It does not merely run past the container's
~10-minute foreground cap —
**it completes in 760 s (12m40s)**, so the card's "9m0s then exit 124"
was the kill, never the
battery's real length. 1723 cases, exit 0.
Where the 760 s went, measured three ways on the same tree:
| reading | value |
|:--|:--|
| total wall clock, 1723 cases | **760 s** |
| `spawnSync` calls from the battery | 253 (46 `dispatch-gates` CLI
children, 198 `git`, 2 `bash`) |
| time inside those children | 500.7 s = **66.2 %** |
| of which: 24 children doing a FULL derivation | ~20.7 s each = **496
s** (the other 22 are argv/`--tier` probes at ~0.1 s) |
| parent's own in-process `discoverFamilies()` | **157.1 s = 20.8 %** |
| one plain derivation, timed alone | **21.2 s** |
Then a V8 CPU profile of that single 21.2 s derivation, which is where
the root cause is:
```
discoverFamilies — 2 profile node(s)
incl=6.86s <- derive:12935
incl=7.06s <- gateFamilyFiles:9455 <- matches:9918 <- changeKindGates:9972
```
**The same whole-tree discovery runs twice per derivation**, over a tree
that cannot have changed
between them — once from `derive()`, once from `gateFamilyFiles()` a
whole call chain away under
`changeKindGates()`. 13.9 s of the 21.2 s, 66 %. And the self-test's
live cases call the same
function a further twenty-odd times in one process, each one a fresh
pass re-reading every workflow
and re-masking every gate source. No hot spot, no quadratic: just the
identical derivation, run N
times.
## The fix: one discovery pass per tree, per process
Exactly the shape — and the argument — the source maskers in this same
file already landed:
*"the same derivation run once instead of a dozen times, which is the
only kind of speed-up this
tool may take."*
- `repoCorpus()` lists the checkout once per process and hands every
caller the SAME listing and
the SAME tree bundle. `derive()` now takes both halves from it, which
also strengthens the
"one read, N answers" invariant its own comment states — the pair is now
the same two objects,
not two equal copies.
- `discoverFamilies` is memoised **on the tree OBJECT**. A caller that
means a different tree
hands a different object and gets its own pass; `null` — the deliberate
no-tree probe — is not
an object and is never memoised. The answer is a pure function of the
tree plus the checkout on
disk, and nothing in this process writes to the checkout: the CLI is
one-shot and every write the
self-test makes goes to a `mkdtemp` directory under the system temp
root.
- `check-dispatch-gates.mjs`: the kill branch now spends its one chance
on the **remedy** rather
than on naming the signal, and every run prints what the battery cost
**on that box** — a reading
that belongs to the run that took it, which is the one shape that cannot
rot. No figure is frozen
in the header; the header's convention is kept.
## Before / after
| | before (`b3b43b6e`) | after (`0f901d56`) |
|:--|:--|:--|
| `--self-test` wall clock | **760 s** | **466 s** (−38.7 %) |
| cases | 1723 pass, exit 0 | **1730 pass, exit 0** |
| one plain derivation | 21.2 s | **13.2 s** (−38 %) |
| longest single gap between case lines | 42.9 s | 27.2 s |
466 s = 7m46s, inside the cap. **The 13.2 s derivation is the half every
dispatch pays**, not only
this gate: every `os-dev` running `dispatch-gates.mjs --commands` on its
own card gets the same
38 % back.
## ⛔ No gate weakened, and nobody else has to make the assertions
Every one of the 1723 cases is still made by this same battery, in this
same CI step. **Zero cases
removed** — verified by diffing the case NAMES of both runs, not their
counts:
```
before cases: 1724 after cases: 1731 (the extra line is the verdict line)
REMOVED (3):
- the tier constant's VALUE is spelled in exactly ONE site ... the definition at
scripts/pm/dispatch-gates.mjs:10689 ... <- same case, line number moved by the insert
- ⭐ 49 case(s) really handed this harness a diagnostic reading ... <- same case, 49 -> 53
- dispatch-gates self-test: 1723 cases pass. <- the verdict line itself
ADDED (10): the 7 new cases below, plus the updated spellings of those same 3
```
All three "removed" names carry a live reading inside the name; all
three are present with the
reading updated. Nothing was shrunk, skipped, `timeout`-wrapped or
marked `continue-on-error`.
Seven cases were ADDED, because the memo's whole symptom is work that
does **not** happen, and
absent work is invisible to every other case here — each of them asks
what discovery ANSWERS, and
the answer is identical either way:
- the default tree is discovered ONCE per process — two further calls
compute no pass
- and those calls hand back the SAME object, so an entry ablated and
restored is one entry
- the corpus behind it is listed once too — one bundle, so the sweep and
the discovery cannot describe different revisions
- and that listing is the tracked corpus itself, not a trimmed copy of
it
- the tree `derive` hands down and the default `gateFamilyFiles` asks
for are ONE pass, not two
- a tree object that is not the corpus's is never served its answer —
same content, its own pass
- the no-tree probe is never memoised — it is not an object, and it
still answers
## Byte-identity: the output did not move
Five invocations (`plain`, `--commands`, `--json`, `--residue`, and a
`scripts/pm` card),
stdout **and** stderr, run against the unmodified `b3b43b6e` checkout
and against this branch:
```
byte-identical after path normalisation: 10 stream(s); differing: 0
225585 total bytes compared
```
The only raw difference was the checkout directory in the provenance
banner, which is what that
banner prints by design.
## Reverse verification
Direction predicted before running: **red**. Committed first, mutated on
disk with the blob hash
proving the bytes moved, `trap ... EXIT INT TERM` restoring, and the
restore proven by
`git hash-object` matching the HEAD blob with `git diff HEAD` empty.
**Ablation A — the memo never hits** (`const hit =
discoveryMemo.get(tree)` becomes
`const hit = undefined`):
```
on-disk proof: anchor lines 1 -> 0 ; injected lines 0 -> 1 ;
blob 52d58ee... -> 1f8b91d1...
✗ the default tree is discovered ONCE per process — two further calls compute no pass
✗ and those calls hand back the SAME object, so an entry ablated and restored is one entry
✗ the tree `derive` hands down and the default `gateFamilyFiles` asks for are ONE pass, not two
✗ dispatch-gates self-test: 3 of 1730 case(s) failed. SELFTEST_EXIT=1
wall clock back to 807 s
restored: hash matches HEAD blob and git diff HEAD is empty
```
The other four new cases stay green under A, correctly: a memo that
never hits breaks the collapse,
not the tree-keying.
## What this PR deliberately does NOT do
The card and the dispatch both floated classifying the family as CI-only
so a dev "is not owed it
locally". **Measured first, and then not taken.** The derivation reaches
a card by **gate-script
identity** — the change kind *"adds or edits a GATE SCRIPT (a file some
discovered check family
runs)"* — which is wider than the dispatch's hypothesis of `scripts/pm`
paths:
```
scripts/pm/check-clause2-carriers.mjs -> pnpm check:pm-dispatch-gates IS in the 35 commands
packages/spec/src/data/filter.zod.ts -> NOT in the 73 commands
scripts/pm/dispatch-gates.mjs -> IS in the 31 commands
```
So it is not in every roster, and the cards it does reach are cards
editing a gate script — exactly
the population that should read this verdict. With the battery now
inside the cap, taking it off
their roster would remove a reading they can finally get, which is a
weakening dressed as a
scoping fix. The reconciler is untouched: exit 3 still reads NOT
MEASURED and a 124/143 kill is
still refused as a run.
## Acceptance notes
Observed, not filed, not fixed here:
- `lint.yml`'s own note for this step records a CI median of **597 s**
over ten merge-group runs
(from #16496). That reading predates this change and is a runner
reading, not a container one —
it will move, and this PR deliberately does not edit that note, which
refuses to carry a figure
anyone has not re-measured on a real run of the step.
- `deadHintSweep` is the next-largest single stage in a derivation (~5.7
s of the remaining
13.2 s). It runs once per derivation, so there is nothing redundant to
collapse; making it
cheaper would be an optimisation of what it computes, which is a
different card and a different
risk.
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01HZfg2AwVX191qCizp88gQr
---
_Generated by [Claude
Code](https://claude.ai/code/session_01HZfg2AwVX191qCizp88gQr)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 917b87e commit 47bfcd0
2 files changed
Lines changed: 212 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
34 | 56 | | |
35 | 57 | | |
36 | 58 | | |
| |||
201 | 223 | | |
202 | 224 | | |
203 | 225 | | |
| 226 | + | |
204 | 227 | | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
205 | 238 | | |
206 | 239 | | |
207 | 240 | | |
208 | 241 | | |
209 | 242 | | |
210 | 243 | | |
211 | | - | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
212 | 260 | | |
213 | 261 | | |
| 262 | + | |
214 | 263 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8463 | 8463 | | |
8464 | 8464 | | |
8465 | 8465 | | |
| 8466 | + | |
| 8467 | + | |
| 8468 | + | |
| 8469 | + | |
| 8470 | + | |
| 8471 | + | |
| 8472 | + | |
| 8473 | + | |
| 8474 | + | |
| 8475 | + | |
| 8476 | + | |
| 8477 | + | |
| 8478 | + | |
| 8479 | + | |
| 8480 | + | |
| 8481 | + | |
| 8482 | + | |
| 8483 | + | |
| 8484 | + | |
| 8485 | + | |
| 8486 | + | |
| 8487 | + | |
| 8488 | + | |
| 8489 | + | |
| 8490 | + | |
| 8491 | + | |
| 8492 | + | |
| 8493 | + | |
| 8494 | + | |
| 8495 | + | |
8466 | 8496 | | |
8467 | 8497 | | |
8468 | 8498 | | |
| |||
10994 | 11024 | | |
10995 | 11025 | | |
10996 | 11026 | | |
10997 | | - | |
| 11027 | + | |
| 11028 | + | |
| 11029 | + | |
| 11030 | + | |
| 11031 | + | |
| 11032 | + | |
| 11033 | + | |
| 11034 | + | |
| 11035 | + | |
| 11036 | + | |
| 11037 | + | |
| 11038 | + | |
| 11039 | + | |
| 11040 | + | |
| 11041 | + | |
| 11042 | + | |
| 11043 | + | |
| 11044 | + | |
| 11045 | + | |
| 11046 | + | |
| 11047 | + | |
| 11048 | + | |
| 11049 | + | |
| 11050 | + | |
| 11051 | + | |
| 11052 | + | |
| 11053 | + | |
| 11054 | + | |
| 11055 | + | |
| 11056 | + | |
| 11057 | + | |
| 11058 | + | |
| 11059 | + | |
| 11060 | + | |
| 11061 | + | |
| 11062 | + | |
| 11063 | + | |
| 11064 | + | |
| 11065 | + | |
| 11066 | + | |
| 11067 | + | |
| 11068 | + | |
| 11069 | + | |
| 11070 | + | |
| 11071 | + | |
| 11072 | + | |
| 11073 | + | |
| 11074 | + | |
| 11075 | + | |
| 11076 | + | |
| 11077 | + | |
| 11078 | + | |
10998 | 11079 | | |
10999 | 11080 | | |
11000 | 11081 | | |
| |||
12941 | 13022 | | |
12942 | 13023 | | |
12943 | 13024 | | |
12944 | | - | |
12945 | | - | |
12946 | | - | |
12947 | | - | |
12948 | | - | |
12949 | | - | |
| 13025 | + | |
| 13026 | + | |
| 13027 | + | |
| 13028 | + | |
| 13029 | + | |
| 13030 | + | |
| 13031 | + | |
| 13032 | + | |
| 13033 | + | |
| 13034 | + | |
12950 | 13035 | | |
12951 | 13036 | | |
12952 | 13037 | | |
| |||
21294 | 21379 | | |
21295 | 21380 | | |
21296 | 21381 | | |
| 21382 | + | |
| 21383 | + | |
| 21384 | + | |
| 21385 | + | |
| 21386 | + | |
| 21387 | + | |
| 21388 | + | |
| 21389 | + | |
| 21390 | + | |
| 21391 | + | |
| 21392 | + | |
| 21393 | + | |
| 21394 | + | |
| 21395 | + | |
| 21396 | + | |
| 21397 | + | |
| 21398 | + | |
| 21399 | + | |
| 21400 | + | |
| 21401 | + | |
| 21402 | + | |
| 21403 | + | |
| 21404 | + | |
| 21405 | + | |
| 21406 | + | |
| 21407 | + | |
| 21408 | + | |
| 21409 | + | |
| 21410 | + | |
| 21411 | + | |
| 21412 | + | |
| 21413 | + | |
| 21414 | + | |
| 21415 | + | |
| 21416 | + | |
| 21417 | + | |
| 21418 | + | |
| 21419 | + | |
| 21420 | + | |
| 21421 | + | |
| 21422 | + | |
| 21423 | + | |
| 21424 | + | |
| 21425 | + | |
| 21426 | + | |
| 21427 | + | |
| 21428 | + | |
| 21429 | + | |
| 21430 | + | |
| 21431 | + | |
| 21432 | + | |
| 21433 | + | |
| 21434 | + | |
| 21435 | + | |
| 21436 | + | |
| 21437 | + | |
| 21438 | + | |
| 21439 | + | |
| 21440 | + | |
| 21441 | + | |
| 21442 | + | |
| 21443 | + | |
| 21444 | + | |
| 21445 | + | |
| 21446 | + | |
| 21447 | + | |
| 21448 | + | |
| 21449 | + | |
| 21450 | + | |
| 21451 | + | |
21297 | 21452 | | |
21298 | 21453 | | |
21299 | 21454 | | |
| |||
0 commit comments