[rwkeyes]

Problem and result

docs/PERF-NEXT.md listed "attribute the ~330 ms fixed per-request cost" as the next experiment. Ran it: a ladder of
unique prompts against two arms, with the engine's own prefill_ms and park_ms read out of its log and TTFT taken
from the first content delta.

The split: TTFT = prefill + ~21 ms of server in both modes. The cache-on state adds +212 ms to the prefill
intercept (434 against 222); parking is ~101 ms and happens after the answer in both modes; --prompt-cache 0 is
58 ms cheaper than the router's cache-off path at 75 tokens; and ~165 ms inside prefill remains unexplained by
any configuration lever tried.

Related discussion

none — measurement and documentation.

Approach

1. The ladder (strata-intercept-probe.py): unique salted prompts at 4…2,048 target tokens (74…1,608 real), two
   samples each, max_tokens 8, streaming, TTFT from the first delta.content — not the first SSE byte, which the
   server sends before the prompt is read (the first attempt measured 2.6 ms and was thrown away).
2. Two arms on one engine build: the deployed conv_route: on and shadow (classify and log, never switch), so
   the cache-on path stock always runs is measured on the same card, not inferred.
3. The cause sweep: fixed 75-token prompts, six samples per arm, --spec 0 and --prompt-cache 0, plus a repeated
   default arm for drift (base 382 ms, base2 382 ms).
4. The page gains section 6 with the table, the six findings, and the artifacts link; section 4's untried list is
   re-ranked (the attribution item becomes "an engine-side phase timer", plus a new idea about dropping cache slots in
   a one-shot mode), and section 2 records that --spec 0 is refused on a native IQ pack.

Validation

Passed:

• Both fits are from ≥128-token legs (linear regime), and the intercept reproduces as TTFT intercept − 21 ms
  server in both arms.
• The mechanism is visible in the data, not assumed: in the shadow arm every request has a ~101 ms park; in the
  on arm exactly three do — the first three of the run — and the fourth onward has none, which is batch_after 4
  switching the cache off.
• The --prompt-cache 0 comparison is same-build, same prompts, 6 samples each, with a repeated default arm
  bounding drift at 0.5 ms.
• --spec 0 failure captured: engine exit 2 with "it needs --native SHARD1, --spec T (T >= 2) and --prefill
  CHUNK" — recorded rather than retried.
• Production: every window restored strata-b70 with :8095 http=200 and engine 0.1.58 in the log, checked after
  the last one.
• Links and rendering: no broken relative link; GitHub's Markdown API renders five tables.

Failed:

• The first ladder run measured the wrong thing: TTFT from the first SSE byte read 2.6 ms because the stream opens
  before the prompt is read. Fixed to the first delta.content, both arms re-run; the byte-prefix JSONs are kept
  beside the good ones so the mistake and its cause are auditable.
• The --spec 0 arm produced nothing for the same reason it was supposed to answer a question: the engine refuses
  that flag on an IQ pack, so the arm never started. Re-run unfiltered to capture the refusal.

Not run:

• No phase timer inside the engine, so the remaining ~165 ms is bounded, not named.
• No repeat-traffic (cache-hit) ladder: the row-cache question from §5 stays open for that shape.

Limits

• One card, one model, one session per arm; the fits are for prompt sizes ≥128 tokens, and the 74–120-token points sit
  slightly above the line (300–340 ms measured against a 280–315 ms fit) because the engine's fixed cost does not
  decay smoothly at small sizes.
• --prompt-cache 0 is a restart-class setting. The 58 ms is evidence that the slots cost something, not a proposal to
  change the deployed config.