Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills
50 changes: 50 additions & 0 deletions .claude/hooks/guard_bash.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""PreToolUse hook for Bash: block two commands that have taken sessions down.

Exit code 2 blocks the command and shows stderr to the agent. Everything else
passes. Keep this list short: it is for rules that must hold every time.
"""

import json
import re
import sys

COMPUTE = re.compile(r"\b(python3?|pytest|uv run)\b")
CAPPED = re.compile(r"\b(tsim-dev|systemd-run)\b")
DETACHED = re.compile(r"\bnohup\b|\bsetsid\b|&\s*(disown\b|$)", re.MULTILINE)


def verdict(tool_input: dict) -> str | None:
"""Return why the command is blocked, or None to let it run."""
command = tool_input.get("command", "")
if re.search(r"\bpkill\b[^|;&\n]*\s(-\w*f|--full\b)", command):
return (
Comment thread
rafaelha marked this conversation as resolved.
Resolved
"`pkill -f` also matches this shell's own command line and kills it. "
"Stop a job with `uv run tsim-dev job stop NAME`, or kill by PID."
)
background = tool_input.get("run_in_background") or DETACHED.search(command)
if background and COMPUTE.search(command) and not CAPPED.search(command):
return (
"Background compute needs a hard memory cap; uncapped jobs have used "
"55 GB and killed every session on this machine. Start it with "
"`uv run tsim-dev job start NAME -- <command>` and follow it with "
"`uv run tsim-dev job wait NAME`."
)
return None


def main() -> int:
"""Read the hook payload from stdin and decide."""
try:
payload = json.load(sys.stdin)
except ValueError:
return 0
reason = verdict(payload.get("tool_input") or {})
if reason:
print(reason, file=sys.stderr)
return 2
return 0


if __name__ == "__main__":
sys.exit(main())
35 changes: 35 additions & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"permissions": {
"allow": [
"Bash(uv run tsim-dev *)",
"Bash(uv sync)",
"Bash(git status *)",
"Bash(git diff *)",
"Bash(git log *)",
"Bash(git show *)",
"Bash(gh pr view *)",
"Bash(gh pr diff *)",
"Bash(gh pr checks *)",
"Bash(gh issue view *)",
"Bash(gh run view *)"
],
"ask": [
"Bash(git push *)",
"Bash(gh pr create *)",
"Bash(gh pr merge *)"
]
},
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "python3 \"$CLAUDE_PROJECT_DIR/.claude/hooks/guard_bash.py\""
}
]
}
]
}
}
138 changes: 138 additions & 0 deletions .claude/skills/benchmarking-tsim/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
name: benchmarking-tsim
description: Measures Tsim throughput and compile time with `tsim-dev bench` on the tsim-bench circuits and compares a change against a reference commit or checkout. Use when asked to benchmark Tsim, to check a change for performance regressions, to produce before and after numbers for a pull request, to run the tsim-bench corpus, or to add a benchmark circuit.
metadata:
internal: true
---

# Benchmarking Tsim

Use `uv run tsim-dev bench`. Do not write timing scripts: hand-written ones have
measured the wrong thing often enough (compile time included, a cold numba cache,
several threads, a busy machine) that their numbers are not accepted in reviews.

## Check a change against main

```bash
git fetch origin main
uv run tsim-dev bench --ref origin/main # quick suite, about 4 minutes
```

Use `origin/main`, not `main`: the local branch is often behind. `--ref` also takes
any commit, or the path of another checkout that has a `.venv`
(`--ref ../tsim-other-worktree`).

```
circuit k shots/s kept compile shots/s vs ref (min, max) compile vs ref
inject_cultivate_d5_p0.001 6 13.9M 0.141 0.13 1.20x [1.19, 1.21] faster 0.96x same
```

- `kept` is the fraction of shots that survive postselection (`-` when there is none).
- `k` is the active width of the compiled sampler. With `--ref`, a row that ends in
`WIDER: k 6 -> 7` or `NARROWER: ...` means the two sides compiled different widths;
`wider` fails the run. A width that changes between runs of one side is an error.

- `shots/s vs ref` is new over reference, so above 1 is faster. The bracket is the
smallest and largest ratio over the rounds.
- A verdict of `faster` or `slower` needs the mean outside the tolerance (5%) and
every round on the same side of 1. `noisy` means the rounds disagree: run again
with `--rounds 9 --min-seconds 1` before you conclude anything.
- `kept-differs`: the two sides kept different fractions of shots (beyond binomial
noise), so they did not sample the same thing and the rate is not compared. It
counts as a failure: a change to postselection or to the sampled distribution.
- Exit code 0: nothing slower. 1: at least one `slower` in rate or compile time, a
`kept-differs`, or a wider active width. 2: a circuit could not be measured.
- The JSON under `.tsim-dev/bench/` has every round.

## Active width only (`--width`)

For changes to compilation, scheduling or the plan, compare widths without sampling:

```bash
uv run tsim-dev bench --width --ref origin/main # whole corpus, compile only
uv run tsim-dev bench --width --suite full --ref origin/main
```

Both sides compile every circuit once, at the same time, in `--jobs` workers (default
4). A row is printed as soon as a circuit is done on both sides, small circuits first,
with an `[n/N]` count, so the first rows appear within seconds; the row shows the width
next to the one recorded in tsim-bench. The verdict is `wider`, `narrower`, `same` or
`unknown`; any `wider` exits 1. `--case-timeout` (300 s) stops one slow compile;
`--timeout` (1800 s here) is the deadline for the whole run, after which the circuits
not yet compiled are reported as errors. A circuit over Tsim's limit shows `>=N`: N is the width
in Tsim's refusal, a lower bound. Widths do not depend on the machine, so this needs
no rounds or quiet machine, but it says nothing about speed: a change that keeps the
width can still be slower. Compile times are shown for each side.

## Compare settings: threads, strategy

`--side REF[:key=value,...]`, given twice, replaces `--ref` when the two sides differ in
more than the checkout. REF is `.` (this checkout), a git revision, a checkout path, or
a Python executable (an environment built elsewhere). Keys: `threads=N`,
`strategy=NAME`. The first side is "new".

```bash
uv run tsim-dev bench --side .:threads=10 --side .:threads=1 # 10 cores vs 1
uv run tsim-dev bench --side .:threads=4 --side origin/main:threads=4
```

- Compile time is compared too; when the sides differ only in threads, read the rate
column.

## What the command controls for you

- One thread, pinned to a core (`--threads N` to change; say so when you report).
- Each measurement is a fresh process with a warm numba cache, under a memory cap.
Kernels loaded from the cache have behaved differently from freshly compiled
ones, and the cached case is what users get.
- Compile time and the first call are excluded from shots/s and reported apart.
- The two checkouts alternate round by round, so drift affects both.
- Circuits come from tsim-bench (https://github.com/rafaelha/tsim-bench) at the commit
pinned in `dev/tsim_dev/corpus.py`, fetched into `~/.cache/tsim-dev` on first use.
Each circuit's manifest there records its arXiv paper, its sampler and how the paper
postselects; the benchmark postselects that way by default. `--no-postselect`
samples every shot instead. shots/s counts requested shots, kept or not.

## What you must do

- Run one benchmark at a time, and nothing else heavy (tests, other agents' jobs)
next to it. Heed the busy-machine warning.
- A change must hold across families. Run `--suite full` before claiming a speedup,
and report the circuits that got slower as well as the ones that got faster.
- For a single circuit: `uv run tsim-dev bench inject_cultivate_d5_p0.001 --ref origin/main`
(a file stem or a full id from `tsim-dev corpus list`), or a path to any `.stim` file
with `--sampler` and `--postselect all|none`.
- Postselection changes what is measured: a change that speeds up rejected shots shows
only with it on. Report which you used, and run both when the change touches the
postselection path.
- Paste the table in the pull request, with the CPU model (`lscpu | grep 'Model name'`)
and the thread count.

## Suites and the whole corpus

Suites are defined in tsim-bench's `suites.toml`: `quick` (the default, a few minutes),
`full` (one or two circuits per family, widths up to about 16) and `hard` (widths 16 to
26, such as the tricycle CCZ factories; minutes per circuit). `--suite corpus` is every
circuit, over 190 from 28 papers, so run it as a job:

```bash
uv run tsim-dev corpus list --max-width 12 # what would run
uv run tsim-dev job start corpus --timeout 14400 -- \
uv run tsim-dev bench --suite corpus --max-width 12 --ref origin/main
uv run tsim-dev job wait corpus # repeat until it reports the end
```

`--max-width K` skips circuits whose recorded active width is larger than K, and those
with no recorded width; `--width-of clifft` (or another backend in tsim-bench's
`k_max.json`) filters on that backend's width instead, for backend work that targets a
different regime. `--only SUBSTRING` keeps circuits whose id contains it.

## Limits

- Tsim on the CPU only. Comparisons with other simulators (Clifft, SymFT, Magna) are
run from the tsim-bench repository, which pins each simulator's environment.
- To add a circuit or change a suite, change tsim-bench: put the file in its paper's
folder, add an entry to that folder's `circuits.toml` (arXiv id, sampler, postselection
and the reason for it, see its `BENCHMARKS.md`), run `python -m tsim_bench check`
there, then bump `REV` in `dev/tsim_dev/corpus.py`. Until the commit is pushed, point
the harness at the checkout with `--corpus-dir` or `TSIM_BENCH_DIR`.
63 changes: 63 additions & 0 deletions .claude/skills/profiling-tsim/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: profiling-tsim
description: Finds where Tsim spends compile and sampling time with `tsim-dev profile` and `tsim-dev inspect`, and turns a profile into an optimization plan. Use when asked why a circuit is slow, where time goes, what to optimize next, or before starting performance work on Tsim.
metadata:
internal: true
---

# Profiling Tsim

## 1. Size the circuit first

```bash
uv run tsim-dev inspect cultivation_d5 # or a path to any .stim file
```

It prints qubits, T count, detectors, `active_width` (k) and `SamplerStats` without
sampling. Cost per shot grows like 2^k. Do not sample or profile above k ≈ 20 without
a smaller `--batch-size` and a memory estimate (2^k × 16 B × batch).

## 2. Profile one run

```bash
uv run tsim-dev profile cultivation_d5 --seconds 5
```

The run is a fresh process, one pinned thread, warm numba cache, under a memory cap.
The output splits samples by phase (`compile`, `sample`) and prints three tables:

- **innermost Tsim frame (kernel callers):** read this first. Jitted numba code has
no symbols, so time inside a kernel is charged to the Python function that called
it. On cultivation_d5 at 1.0.0 this shows 82% in `sample_words`
(`noise/channels.py`) and 14% in `_run_chunk` (`statevector/shots.py`): noise
sampling, not the state vector, is the cost there.
- **innermost frame:** separates `[jitted numba code]` from numpy, libc (memcpy,
malloc) and Python overhead. A large non-jitted share means work outside kernels.
- **Tsim frames, inclusive:** the call tree, for compile time in particular.

The folded stacks are saved for flame graphs (`speedscope <file>` or
`flamegraph.pl`).

Profile at least two families before choosing a target; the hot spot differs
between circuits with mostly direct detectors (cultivation, surface codes), wide
quantum components (quantum volume) and many rotations (coherent noise).

## 3. Inside a kernel

The profile stops at the kernel boundary. To compare variants of a kernel, change
it and use `uv run tsim-dev bench <circuit> --ref origin/main`: that is the
measurement that counts. Micro-benchmarks of a kernel alone have misled before
(cache effects, struct passing by value), so confirm any gain end to end.

Hardware counters (`perf stat`) need `/proc/sys/kernel/perf_event_paranoid` at 1 or
lower; ask the user rather than changing it.

## 4. Turn it into a plan

- Write down the target (phase, function, share of time) and the circuits it
affects, before changing code.
- Estimate the ceiling: removing a function that takes 14% gives at most 1.16x.
- Prefer one general change over special cases for particular circuits, and drop
ideas whose gain is small compared with the code they add.
- After the change, profile again to confirm the share moved, then follow the
`benchmarking-tsim` skill for the numbers that go in the pull request.
84 changes: 84 additions & 0 deletions .claude/skills/shipping-tsim-changes/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
name: shipping-tsim-changes
description: Takes a change to Tsim's code from a branch to a reviewed local commit, with the checks, evidence and git rules this repository uses, scaled to what the change touches. Use when starting a feature, bug fix or optimization in the Tsim repository, when updating an existing pull request, or before saying that a change is done.
metadata:
internal: true
---

# Shipping a change to Tsim

## 1. Start

**New work.** Work in a worktree on a new branch off the newest main, not in the main
checkout: `git fetch origin main && git worktree add -b <type>/<topic> ../tsim-<topic>
origin/main`, then `uv sync` there. Stack on another branch only when the change needs it.

**Updating an existing pull request** (review comments, a follow-up, a fix the user asks
for on that PR). Keep its branch: find it with `gh pr view <n> --json headRefName`, use
the worktree that already has it (`git worktree list`) or add one for that branch, and
commit on top. Do not start a new branch or a new PR for it. Rebase only when the user
asks or the branch conflicts with its base; a rebase rewrites a pushed branch, so say
that it needs a force-push.

**Acceptance criteria.** Infer them from the task and the code, and state them in a line
or two before you start: the behavior, the test that shows it, and for performance work
the circuits and the gain that matters. Ask only when they are unclear, for example an
unstated performance target, two plausible behaviors, or a change to public API.

## 2. Build

- For a bug, write the failing test first and watch it fail.
- `uv run tsim-dev test --changed` while iterating.
- Anything long or memory-hungry goes through `uv run tsim-dev run -- <command>` or
`uv run tsim-dev job start NAME -- <command>`; see AGENTS.md.

## 3. Verify

Run what applies and keep the output. A claim without the command and its result is not
evidence.

| Change | Required |
|---|---|
| Any | `uv run tsim-dev check` prints PASS for lint and tests |
| Could affect compile or sampling cost (see below) | `uv run tsim-dev bench --ref origin/main`; any regression investigated and explained |
| Aims to make Tsim faster | the same with `--suite full`; see the `benchmarking-tsim` skill |
| Could change the active width (graph, simplification, scheduling, plan) | `uv run tsim-dev bench --width --ref origin/main`; any `wider` explained |
| Bug fix | the new test fails on `origin/main` and passes on the branch |
| User-visible | a `CHANGELOG.md` entry under `## [Unreleased]` |
| Behavior the user skills describe | `skills/` updated; `test/unit/test_skills.py` passes |

**When to benchmark.** Code on the compile path (parsing, the ZX graph and its
simplification, components, the plan, scheduling, lowering to the action tape) or the
sampling path (samplers, noise, the state-vector executor, numba kernels), and changes
to dependencies or numba settings, can change performance. Docs, tests, type hints,
error messages, printing and code that runs once per call outside those paths do not.
When unsure, run the quick suite: it takes a few minutes.

**A regression is a finding, not a veto.** If the benchmark reports `slower` (exit 1),
do not stop at the exit code and do not hide it:

1. Confirm it: rerun the affected circuits with `--rounds 9 --min-seconds 1`.
2. Find the cause: `tsim-dev profile` on one affected circuit, on both checkouts.
3. Remove it if the change does not need it. If it does, for example a correctness fix
that has to do more work, keep it and say so: which circuits, by how much, and why.
A wrong result is never the faster option.

Then read your own diff (`git diff origin/main --stat`, then the diff). Remove anything
the task did not need: special cases for one benchmark, leftover debug code, unrelated
reformatting. Ask whether a simpler change would do.

## 4. Commit and review

- Commit locally with a Conventional Commits message.
- Do not push, open a pull request, force-push or merge unless the user asked for that
step in this conversation. Some branches are not meant to be public yet. When updating
an existing PR, pushing to its branch still needs the user's go-ahead.
- For a review, give a fresh subagent the diff and the task, not your conclusions, and
ask for correctness findings only. Fix what it finds, then run step 3 again.

## 5. Report

Say in a few sentences what changed, the verification commands you ran with their
results, and what you did not verify. For performance-relevant changes include the bench
table, the CPU model and the thread count, and the explanation of any regression. Say
plainly if something failed or was skipped.
Loading
Loading