[rwkeyes]

Problem and result

The ranked list's first item was "ship conv-route-batch-after 1–2", on a single-shot measurement of −21 % median TTFT.
The missing half was what a smaller run length costs a conversation. Ran it: one traffic shape — a repeated 3k prompt,
a two-turn conversation, six unique single-shot prompts, then the conversation's next turn and the repeat — against two
card windows that differ only in --conv-route-batch-after.

1 buys 0.52 s across six single-shot requests and costs 1.88 s on the next conversation turn. Break-even is ~21
single-shots per conversation re-open. 4 stands; 1 is only right for a pure one-shot queue.

Finding it also exposed a dead setting: conv_route_batch_after in the config was ignored by the server (it read only
the flag, and the launcher passes no flag), so both arms were byte-identical until the flag was passed by hand. Fixed in
PR #16.

Related discussion

• [Docs] File the performance inventory and close the PLE read-path knobs #14 — the fixed-cost split (what the cache-on state costs per request)
• serve: honour conv_route_batch_after from the config, not only the flag #16 — the config-key fix, unmerged because serve/ must stay byte-identical here

Approach

1. One shape, both arms. strata-mixed-probe.py runs R1 (3k prompt), C1/C2 (two turns), S1–S6 (unique
   single-shots), C3 (third turn), R2 (the 3k prompt again); a step that jumps from ~0.3 s to seconds is a re-prefill.
2. Server flags, not config keys. The first sweep set the value in the config and produced identical results in both
   arms — the flag is what the server reads. Re-ran with SRV_ARGS='--conv-route-batch-after 1'; the inert runs are
   kept beside the good ones.
3. The page gains section 7 with the step table, the five findings, and the decision; the ranked item closes as "keep
   4"; the closed table records both numbers; the method rules renumber to section 8.

Validation

Passed:

• The difference is in the steps that should differ, and only there: S2–S4 drop by 0.17 / 0.18 / 0.13 s and S5–S6
  are equal (0.328 against 0.327) in both arms — the flip happens either way by the fifth request; C2 flips from 0.587
  to 2.465; C3 and R2 behave the same in both arms.
• The reproduction is clean: C2 at 2.465 s matches the 2026-11-09 arm's 2.472 s, on a different day, a different
  window and a fresh engine start. Those older artifacts are shipped under mixed/2026-11-09/.
• Every window restored production: :8095 http=200, engine 0.1.58, checked after each.
• Rendering: six tables on the page, no broken relative link.

Failed:

• The first sweep was inert — the config key it varied is not read by the server. Caught because the arms were
  byte-identical, which no real setting difference would produce. Fixed the sweep (server flag), and filed the setting
  itself as PR serve: honour conv_route_batch_after from the config, not only the flag #16.

Not run:

• No run length between 1 and 4 (2 and 3 would interpolate, not surprise), and no time-based decay — the router counts
  requests, not seconds.

Limits

• One card, one model, one 3k-token conversation per arm, one run each. The single-shot saving is measured on six
  requests; the conversation cost is one number from one turn, though it reproduces a week-old measurement to 7 ms.
• The 3k context is what makes the re-prefill 1.9 s; a 30k agent context would make the same mistake far more expensive,
  which strengthens the conclusion rather than weakening it.