Skip to content

test(plugin-auth): the last three #23184 test-budget rows (org-create-posture-gate, implicit-account-linking, dev-admin-seed-credential-gate) - #24626

Merged
objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/issue-23184-plugin-auth-test-budget
Oct 19, 2026
Merged

objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/issue-23184-plugin-auth-test-budget

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #23184
Clause-②: no

The three @objectstack/plugin-auth rows that were still open on #23184. Every other row on the card has landed (#23243, #23300, #23697, #23711, #23721, #23748), so this PR closes the card. Tests only: no source file, option or export changes, and nothing is published (skip-changeset).

The readings (A1)

Each reading is the vitest module line (✓ src/FILE.test.ts (N tests) MS), the number the hourly battery's per-file timing reads. Base and head ran interleaved, three rounds, under scripts/pm/os-verify-lock.sh, on a shared box. Base is the file at de4352460, run under a temporary name. Head is 6c63f499e (the merge of main after it touches none of these files).

file base runs (ms) head runs (ms) median base → head card CI reading → projected
org-create-posture-gate.test.ts 4756 / 4856 / 4317 1309 / 1175 / 1112 4756 → 1175 (−75 %) 11.6 s → 2.9 s
implicit-account-linking.test.ts 3365 / 3538 / 3449 1051 / 1324 / 1288 3449 → 1288 (−63 %) 11.7 s → 4.4 s
dev-admin-seed-credential-gate.test.ts 4824 / 4560 / 4504 2741 / 2740 / 2753 4560 → 2741 (−40 %) 10.3 s → 6.2 s

The projection scales the card's CI reading by head ÷ base. That is the card's own worst CI/local ratio applied to the head median (2.44, 3.39 and 2.26). No file declares a budget header: all three come in under the 10 s line.

Where the time went (per-case probe at base, scrypt calls counted and timed):

  • org-create-posture-gate: the slowest phase was per-test. 25 sign-ups each ran one production-parameter scrypt derivation under a fresh salt, 3.5 s of 4.9 s. The first case also loaded better-auth's vendor modules inside the clock, about 0.8 s.
  • implicit-account-linking: also per-test. 14 sign-ups took 1.9 s of 3.8 s, and the first manager's vendor load took about 0.75 s.
  • dev-admin-seed-credential-gate: per-test again. A sqlite boot (init() plus DDL for 30 objects, about 70 ms) ran in each of 15 cases. There were 12 derivations (1.8 s), and case ⓪ paid about 1.1 s of vendor load.

CI readings (A2): NOT MEASURED. Two sources were tried. The test-core-timings-K-of-6 artifacts of the hourly runs (latest 39265242003) and the Test Core job logs both redirect to productionresultssa13.blob.core.windows.net, which answers 403 from this container. The Test Core check-run annotations name only the slowest file of a package that is over its ceiling. The open refresh #24467 carries no per-file readings ("Regression readings: NOT MEASURED"). So the card's 2026-10-13 readings stay the CI numbers, and the pin is the next measured refresh's list.

What changed

  • All three files: vendor modules load at module top. AuthManager reaches better-auth through a lazy await import(), so the first case paid the vendor's module evaluation inside its clock. Side-effect imports at module top follow AGENTS.md ("clocked windows measure behaviour, never loading"). They copy the shape admin-impersonate-endpoint.test.ts already uses, and each list is the set of modules a probe saw that file's managers load.

    • better-auth's one load-time read is NODE_ENV, which its env module keeps on first evaluation. Measured inside vitest: that module is already evaluated when the ./auth-manager import runs. It is reached via ./objectql-adapter.js → better-auth/adapters. The vendor reads isTest() = true, isDevelopment() = false, even after a case sets NODE_ENV=development. So moving the rest of the load changes no captured value, and dev-admin-seed-credential-gate gets the move too. An earlier note on this card said that could not happen; it does not hold.
  • org-create-posture-gate: one founder sign-up per file. It goes through the real /sign-up/email route in a beforeAll, with both knobs unset. runGuidedWorkspaceCreation copies its tables (structuredClone) onto each case's own engine, builds the case's own manager under the case's own env and tenancy service, and presents that sign-up's cookie. No case is about the sign-up: each one is about what organization/create and /auth/config answer under a scenario. The two cases that sign up inline still do. Derivations: 25 → 3.

  • implicit-account-linking: one sign-up per file, copied per case. signUp gives a case's first sign-up on a still-empty engine a copy of one real sign-up's tables, with the user row's address set to the case's own, and answers that sign-up's cookies. Two kinds of sign-up still run live, as before:

    • one on an engine that already holds a user;
    • one under a secondaryStorage manager, where the session goes to the cache.

    makeManager records which engine a manager runs over. Derivations: 14 → 2.

  • dev-admin-seed-credential-gate: pooled engine, scrypt memo. It reuses the landed __tests__/sqlite-engine-pool.testkit.ts: one booted engine per file, emptied before each case, refused if its schema changed or it holds an organization row. It also reuses __tests__/scrypt-memo.testkit.ts: a sign-in after the seed, and the second seed run, re-derive the same password against the same salt and are answered from the first derivation's bytes. Every fresh salt is still derived for real. Derivations: 12 → 9. Engine boots: 15 → 1.

Every it, describe and expect line of the base files is present unchanged in the head, compared line by line:

file expect( lines it( lines describe( lines
org-create 53/53 13/13 3/3
implicit 83/83, plus one 23/23 3/3
dev-admin 50/50 15/15 1/1

The one added expect asserts the shared sign-up's own 200, the same check signUp already made. In org-create, expect(cookie).not.toBe('') is the same line; it now reads the copied sign-up's cookie. The test lists match too: 25/25, 23/23 and 15/15 pass at both base and head.

Equivalence and isolation (measured once, nothing kept)

  • Equivalence probe (a temporary test file, deleted, never committed). For every arrangement the head copies, it compared a live sign-up (the base arrangement) with the copy (the head arrangement), normalising ids, tokens, hashes and timestamps.
    • implicit-account-linking: all 13 addresses, including the accountLinking opt-out manager. The copied tables equal a live sign-up of that address on that manager, and get-session with the copied cookie answers 200 for that user and address.
    • org-create-posture-gate: all 23 scenarios runGuidedWorkspaceCreation runs. A sign-up under the scenario's own env and tenancy service gives the same tables before organization/create, then the same status, body, three /auth/config flags and tables after it.
    • 36 of 36 equal. No copied row carries a fake-engine sequence id (row_N): 0 in all 36. So a copied table cannot collide with an id the case's engine assigns later.
  • Leak mutations, each through scripts/ablation-replace.mjs (anchor hit 1 → 0, blob changed, restored blob == HEAD, git diff HEAD empty):
    • Dropping structuredClone in implicit-account-linking turns the file red: 12 failed / 23.
    • Disabling the pool's row reset in sqlite-engine-pool.testkit.ts turns dev-admin-seed-credential-gate red: 13 failed / 15.
    • Dropping structuredClone in org-create-posture-gate leaves the file green (25/25). Its assertions read sys_organization, which each case's own engine creates, so a leak into the sign-up's tables is invisible to them. So isolation there was measured directly. A snapshot of the shared sign-up's tables taken after the beforeAll is byte-identical after the last case with the copy, 4 rows before and after. Without the copy the snapshot changes: 4 → 15 rows. The same snapshot in implicit-account-linking is identical with the copy and changed without it.
  • Shuffled order: --sequence.shuffle with seeds 1, 2 and 3, all three files green (25/25, 23/23, 15/15). The verbose order shows different first cases per seed.

Tests and gates, at head 54067dc35 (after merging main at 5a871e4e4)

  • The three files at the final head, one more interleaved run on the module line: org-create 1274 ms, implicit 1234 ms, dev-admin 2492 ms. Results: 25/25, 23/23 and 15/15.
  • Whole package: all 165 @objectstack/plugin-auth test files ran in 8 chunks with vitest run --maxWorkers=2, each under the lock. Every chunk exited 0 (VERDICT command-exit 0): 165/165 files passed, with 3165 tests passed and 10 skipped. The skips were already there at base.
  • Typecheck: pnpm --filter @objectstack/plugin-auth run typecheck gave VERDICT command-exit 0. check:test-typecheck reports OK and the debt ledger is unchanged (10 files / 88 errors / 23 signatures). tsc -p tsconfig.test.json --listFilesOnly lists all three files, and none of them is in the ledger.
  • ESLint on the three files (--no-inline-config): exit 0, no findings.
  • Derived gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack, with no paths, at 54067dc35 gave 55 commands. All 55 exited 0, each exit captured before any pipe. check:dual-build-cjs-loads first exited 3 (PREREQUISITE NOT MET, no workspace dist/). It exited 0 when re-run after a full turbo run build (77/77, VERDICT command-exit 0). dispatch-gates --ran reports 55 derived, 55 run, 0 NOT-MEASURED, 0 UNRUN.
  • CI families run on this PR's check-runs. test:repo was not run locally (CI-owned; the diff touches no repo-project test).
  • No budget header was added, so measure-test-shard-timings --self-test is not owed.

Acceptance notes

  • org-create-posture-gate's assertions cannot see a write that leaks into tables its cases do not read. The copy holds by construction and by the snapshot above. This is an observation about the file's reach, not a defect.
  • Each of these files spends about 30 to 35 s in vitest transform and import on this box before its first case, at base and at head. That is outside the module line the budget reads.

Generated by Claude Code

…at module top, pooled engine, one founder sign-up)

Claude-Session: https://claude.ai/code/session_01LR1cD9JqgkeqY7k9rrZH8s
Co-authored-by: Claude <noreply@anthropic.com>
…m one copied sign-up

Claude-Session: https://claude.ai/code/session_01LR1cD9JqgkeqY7k9rrZH8s
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json c3954057fe100fe1e500a2a8e5f88f95764e679e → packageMentionDocs.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对

ACCEPT · #23184 · domain:services seat 2 · os-warren · session_01LR1cD9JqgkeqY7k9rrZH8s · 2026-10-19T17:10Z · head 54067dc35a, the review of record for claim 6269637863.

  1. Form: draft, base main. The first line is Fixes #23184, and no other closing keyword sits beside an issue number. Clause-②: no. Tests only: three plugin-auth test files, +195/−31, and skip-changeset (nothing published moves).
  2. Premise, measured:
    • Interleaved module-line medians, base → head: org-create-posture-gate 4756 → 1175 ms, implicit-account-linking 3449 → 1288 ms, dev-admin-seed-credential-gate 4560 → 2741 ms.
    • The causes were per-test scrypt derivations (25→3, 14→2, 12→9), repeated sqlite boots (15→1) and the vendor load. A shared sign-up is copied per case, and the landed engine pool is reused.
    • Projected against the card's CI readings by the card's own worst CI/local ratio: 2.9 / 4.4 / 6.2 s, all under 10 s, so no budget header is needed.
  3. Gates:
    • The seat's diff read: 0 expect( lines removed and 1 added (the shared sign-up's own 200 check). The dev's sorted comparison found every it / describe line unchanged.
    • Isolation was measured by a leak mutation per file and by three shuffle seeds, all green.
    • Local: the package's 165 files pass (3165 tests); typecheck exit 0, ledger unchanged; the 55 derived families exited 0 (--ran: 0 NOT-MEASURED).
    • CI on 54067dc35a at this read: 15 success, 7 skipped, 15 in progress, 0 failed. Landing waits for all of them.
  4. Accepted deviations:
    • The CI per-file readings (the card's A2) are NOT MEASURED: the timing artifacts and job logs answer 403 from this container. The projection above stands in for them, and the next measured shard-timings refresh is the confirming reading.
    • One measurement round was cut by its own timeout while it waited in the lock queue. Only paired rounds are reported.
    • The classifier refused a read of the container's proxy notes. It was not retried or worked around.
  5. Out of scope: three observations stay in the PR's Acceptance notes (carrier: none). An earlier diagnosis on this card about better-auth's NODE_ENV capture does not hold. org-create-posture-gate's assertions cannot see writes to tables they do not read. About 30 s of transform and import per file falls outside the module line the budget reads.
  6. Landing: ordinary queue landing. When every check on the current head is green, the seat sends pr_ready + automerge_enable through the relay, then confirms MERGED and clears the card. If the next measured shard-timings refresh shows a row back over 10 s, that row reopens on test budget: 171 test files in 20 packages run over 10 s of wall in the hourly full battery (refresh PR #23141, #22949 ruling E item 3) — one card per owning lane, split by triage #23151's family, not here.

Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 19, 2026 17:29
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 973917c Oct 19, 2026
45 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-23184-plugin-auth-test-budget branch October 19, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test budget (domain:services): 18 test file(s) over 10 s of wall in the hourly full battery (Part of #23151)

2 participants