[rafaelha]

A harness for agents that work on Tsim's code, built from what slowed down the last 87 agent sessions on this repository.

What past sessions showed

• At least 6 out-of-memory incidents, each of which killed other sessions; limits stated in prompts did not hold.
• About 21 h of tool time in sleep and polling loops waiting on jobs.
• About 540 one-off timing and comparison scripts, with repeated corrections to how they measured (threads, postselection, baselines, cold numba cache).
• 40 commands killed their own shell with pkill -f.
• test_the_calling_thread_keeps_parallel_kernels_after_run_tasks failed in 49 runs and was deselected by hand 36 times.

What this adds

tsim-dev CLI (dev/, installed by uv sync as an editable path dependency of the dev group; not part of the wheel). Output is short, details go to .tsim-dev/, and exit codes mean 0 pass, 1 the code failed or got slower, 2 the harness could not run, 3 still running.

Command	What it does
check	pre-commit on the changed files, then the full suite
test --changed	tests of the changed files
bench --ref origin/main	interleaved A/B in fresh processes, one pinned thread, warm numba cache; "slower" only when every round agrees; circuits from tsim-bench, postselected as their paper does (--no-postselect to turn off), --suite quick/full/hard/corpus, --max-width K --width-of tsim/clifft
bench --side .:threads=10 --side .:threads=1	the same A/B between sides with their own threads or strategy; a side is ., a revision, a checkout or a Python executable
bench --width --ref origin/main	active widths only, compiled and not sampled; rows print as circuits finish on both sides (100 of 190 in 9 s), --timeout is the deadline for the whole run; any wider width exits 1. Shot benchmarks also compare the two sides' widths and check that a side's width does not change between runs
corpus fetch/list	fetch the pinned tsim-bench, list circuits with arXiv id, width and mask size
inspect	qubits, T count, active width and stats, without sampling
profile	py-spy, split into compile and sampling, with kernel time charged to the calling function
run, job start/wait/stop	any command under systemd memory, CPU and time caps

Benchmark circuits come from tsim-bench at a pinned commit (dev/tsim_dev/corpus.py), fetched into ~/.cache/tsim-dev on first use; --corpus-dir or TSIM_BENCH_DIR point at a local checkout. tsim-bench holds 190 circuits from 28 papers; for each one a circuits.toml records the arXiv paper (the original one for circuits that benchmark papers collected), the sampler, and the postselection the paper's protocol uses, with the reason (see its BENCHMARKS.md). Its python -m tsim_bench check verifies that no masked detector fires without noise (183 circuits of width ≤ 20 pass; 7 are too wide to sample). Suites: quick (8 circuits), full (32, widths up to 16, including tricycle CCZ factories up to width 16) and hard (7 tricycle/BLP factories, widths 16–26). The timing worker lives there too, so other backends can reuse it.

Developer skills in .claude/skills/ (.agents/skills links to it): shipping-tsim-changes, benchmarking-tsim, profiling-tsim, testing-tsim-correctness. They are not under skills/, so they are not published to users.

.claude/settings.json: a PreToolUse hook that blocks pkill -f and background Python without a memory cap; read-only git/gh and tsim-dev are allowed; push, PR creation and merge ask.

AGENTS.md: points to the CLI and the skills; rules for heavy jobs, performance and pull requests updated.

Test fix: the kernels test used a 4,096-element prange, which numba's TBB layer runs on one thread. It now uses 2^22 elements.

Verification

• uv run tsim-dev check: lint passes, 2606 tests pass in 82 s (on a TBB host, where the kernels test used to fail).
• Null A/B, this branch against origin/main (identical src/), quick suite with postselection, one thread, 5 rounds, AMD Ryzen 9 9900X 12-Core Processor. kept is the accepted fraction:

circuit                         k  shots/s   kept  compile   shots/s vs ref (min, max)      compile vs ref
inject_cultivate_d3_p0.001      4    29.5M  0.687     0.02   1.00x [1.00, 1.01] same    0.99x same
inject_cultivate_d5_p0.001      6    14.3M  0.144     0.13   1.00x [1.00, 1.00] same    1.00x same
end2end_d3_dsurface15_p0.001    4     925k  0.538     1.82   1.00x [0.99, 1.01] same    1.00x same
hook_inject_T_d15_pd5_p0.001    1     823k  0.667     2.06   1.00x [1.00, 1.00] same    1.01x same
distillation_color_d5_Z         3    12.8M 0.0797     0.02   1.00x [1.00, 1.00] same    0.98x same
coherent_d3_r3                  4    18.9M      -     0.01   1.00x [0.99, 1.00] same    0.98x same
surface_d7_r7                   0    11.3M      -     0.11   1.00x [1.00, 1.00] same    1.00x same
qv_q10                         10    10.6M      -     0.14   1.01x [1.00, 1.02] same    1.00x same

• --suite full: all 32 circuits run without errors (one round, no reference).
• --side .:threads=10 --side .:threads=1, 2 rounds: 7.8x on inject_cultivate_d5_p0.001, 3.3x on coherent_d5_r5, 2.1x on qv_q10.
• bench --width --ref origin/main over the whole corpus: 190 circuits, 0 errors, all the same width as main; 180 rows within 64 s, all in 512 s (the 15-to-1 d25 circuit compiles for 240 s).
• Jobs: a 200 MB cap, a 3 s time limit and job stop were each reported correctly.
• The hook is covered by test/unit/test_dev_cli.py, with the verdict logic, the test mapping and the skill frontmatter.

Not included

• GPU runs and comparisons with Clifft, SymFT and Magna stay in tsim-bench.
• The fuzz harnesses and the exact oracle are still on local branches.
• The hook and permission rules were tested as scripts, not in a live Claude Code session.
• A review workflow (risk tiers, bot-comment gate) follows in a stacked pull request.

🤖 Generated with Claude Code

[github-actions]

☂️ Code Coverage

current status: ✅

Overall Coverage

Statements	Covered	Coverage	Threshold	Status
4897	4266	87%	0%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 44541ca by action🐍

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-10T15:05:44.194204Z	e34f504	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e34f504e2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".