test(rest): give the import-template parity case a measured per-shape budget - #21481
Merged
objectstack-fleet[bot] merged 1 commit intoOct 2, 2026
Merged
Conversation
… budget The parity case in import-template-route.test.ts loops over 20 default shapes. The loop is 98% of the case under every load measured, and the engine boot is warm by the time the case runs. So the case gets an explicit timeout of 1000ms per shape (20000ms for 20 shapes), with the measured timings and the arithmetic in a comment above it. No assertion changes. Claude-Session: https://claude.ai/code/session_016GiHYRmLSNWTfbX9gVQkpz Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
📓 Docs Drift CheckNothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
This was referenced Oct 2, 2026
objectstack-fleet
Bot
deleted the
claude/issue-21428-import-template-flake
branch
October 2, 2026 23:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #21428
Clause-②: no
The parity case in
packages/rest/src/import-template-route.test.ts, "for every shape of default the engine reads", ran past vitest's default 5000ms twice on loaded Test Core shards. I measured where the time goes. The engine boot costs 5-6ms. The loop over the 20 default shapes is 98% of the case under every load I tried. So, as the triage ruling says for the loop case, the case now has an explicit timeout with a stated, measured budget: 1000ms per shape, which is 20000ms for the 20 shapes today. The diff is this one test file (+31 / -1). No assertion changes.Premise check (done first, on
origin/main)6210f887and on4c8363f4, this branch's base. The file was last changed at88b484e0.premise_still_valid: true.boot()is warm: 5-6ms idle. The cold first-use cost goes to the file's first case instead (see Acceptance notes).Where the time goes
I added phase timers (
performance.now()) to a throwaway copy of the file. The copy was never committed and is deleted. Each run ran the whole file, so the parity case ran after the 25 cases above it, as it does in CI. The box has 4 vCPU and was shared with other jobs. The "busy loops" are CPU-boundnode -e 'for(;;){}'processes. Their PIDs were recorded, and a trap killed them.Within the loop, each shape does three things, each about a third of the loop: build the template through the real export route, parse the workbook, and import one row through the real import door. Idle totals were 312-337ms, 303-390ms and 289-344ms.
engine.findwas 4-5ms. The slowest single shape took 71-100ms idle and 671-912ms at 24 busy loops.All of that is the work the case asserts on, and boot plus schema sync is 2% of the case. So moving the boot into a
beforeAllwould remove about 6ms from a case that takes 7-8s under load. It would not fix this.The fix
const PARITY_MS_PER_SHAPE = 1_000;, and the case's timeout isObject.keys(PARITY).length * PARITY_MS_PER_SHAPE. The budget is per shape because the loop is the cost. If someone adds a shape, the budget grows with it and the headroom stays the same.Before / after: CI's budget, under load
This reproduces CI's signature on the unmodified file. The measurement uses 24 busy loops on 4 vCPU and 4 interleaved pairs. Each pair runs BEFORE first and then AFTER, with the same command and the same load. Interleaving puts the shared box's drift on both sides.
git hash-object9412aad7, the same as the4c8363f4blob), run beside the committed file.bc10b1eb.vitest run --project local --maxWorkers=2 --reporter=verbose FILE, run inpackages/rest.Test timed out in 5000msat 5048-5160ms, every runbc10b1eb)When idle, the committed file at
bc10b1ebruns 37 passed, and the parity case takes 1064ms.The case's own timings did not change, and they could not. The fix changes the window and leaves the cost alone, because the cost is the asserted work.
Precedents read
a690c494, which landed for Flaky on Test Core (6/6): the legacy-boolean case of dev-plugin-tenancy-posture.test.ts times out at 5000ms — the file header declares this exact hazard #19631, removed an unbounded, load-dependent draw from the clocked window.80c29a14, which landed for Flaky on Test Core (5/6): the first case of packages/client auth-login-register-envelope.test.ts times out at 5000ms — it pays a cold sqlite-wasm + ObjectQL + better-auth scrypt sign-up inside vitest's default timeout #20242, moved a one-time cold load out of every clocked window to module scope. It argued against both abeforeAlland a per-case timeout, on the grounds that "every budget the cost is moved INTO can be exhausted by a heavier shard".Verification, at
bc10b1ebpnpm turbo run build --filter='@objectstack/rest^...' --concurrency=2, 24 / 24 tasks.pnpm --filter @objectstack/rest exec vitest run --maxWorkers=2 src/import-template-route.test.ts:Tests 37 passed (37).pnpm --filter @objectstack/rest test:Test Files 256 passed (256),Tests 4848 passed | 322 skipped (5170).pnpm --filter @objectstack/rest typecheck: exit 0, andcheck:test-typecheck: OK, 0 files / 0 errors.tsconfig.jsonexcludes*.test.ts.tsc -p tsconfig.test.json --listFilesincludes this file, so the typecheck does cover it.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack, with no paths, derived 54 commands for this diff. All 54 ran and exited 0.--ranreports54 derived famil(ies) accounted for — 54 run, 0 NOT-MEASURED. Two of them needed a prerequisite first:check-plugin-teardown-shape.mjs --self-testrefused on the shallow clone, because its pinned fixture commit was missing. After fetching that commit, it exited 0.check:dual-build-cjs-loadsanswered PREREQUISITE NOT MET because there was nodist/. Afterpnpm turbo run build --filter='!@objectstack/docs'(71 of 72 tasks were cache hits), it exited 0.pnpm lintis a narrowing, with three pieces of evidence:ESLint#isPathIgnoredanswersfalsefor the one changed file. The computed config for it carriesparserOptionsecmaVersion/sourceTypeonly, with noprojectand noprojectService.eslint --no-inline-config --format jsonon the changed file reports 1 file, 0 errors and 0 warnings.eslint.config.mjsitself states. The diff adds no export and touches no other file, so it cannot change the verdict on any untouched file.Changeset
skip-changeset. The diff is one*.test.tsfile.@objectstack/rest'sfiles[]isdist,README.mdandCHANGELOG.md. Afterpnpm --filter @objectstack/rest build,PARITY_MS_PER_SHAPEand the test-only stringspty_listand the case's title have 0 hits across all three. The positive controlanswerImportTemplatehas 6 hits indist/. No published byte changes.Acceptance notes
Generated by Claude Code