[rwkeyes]

Problem and result

Item 1 of the ranked list was "defer the slot refill past the first token", worth 72–188 ms of TTFT. I built it as a
flag (STRATA_REFILL_DEFER, one if in the serve prompt path, default unchanged) and measured it. It is refuted, and
badly — it saves the 72–188 ms and costs two orders of magnitude of decode:

real tokens	refill kept	refill deferred	decode, kept	decode, deferred
126	398 ms	2,945 ms	92.0 tok/s	30.9 tok/s
456	539 ms	3,126 ms	91.8 tok/s	0.5 tok/s
1,608	1,468 ms	3,953 ms	74.3 tok/s	0.4 tok/s

Mechanism, from the per-sample sequence: at 126 tokens requests 1–2 are normal (0.553, 0.537 s) and request 3 is
3.069 s. The collapse starts once the cache holds the previous prompt's experts. The borrowed slots are the decode
working set, so the refill is a prefetch for the tokens about to be generated; skipping it converts one 72–188 ms
transfer per request into a streamed expert fetch per token. 0.32 ms/slot buys 92 tok/s over 0.5.

So the refill is not dead weight, and "when" was the wrong question. "How much" survives: only the experts the first
decode positions actually route are worth pre-loading — item 2 of §4 now.

Related discussion

• [Docs] The ~165 ms is the expert-slot refill: 0.319 ms per lent slot, on the TTFT path #19 — §9, where the refill was named and this lever proposed

Approach

1. Patched the serve prompt path with an env-gated skip (STRATA_REFILL_DEFER=1), default OFF, so both arms run on one
   binary and the default path is untouched.
2. Rebuilt incrementally (ninja strata) and ran the same ladder on both arms, six samples per size, with the trace on.
3. Compared TTFT, the refill line, and decode tok/s, then read the per-sample sequence to see when the damage starts
   rather than only that it happened.

Validation

Passed:

• The control arm reproduces the pre-patch binary: 398 / 539 / 1,468 ms against §9's 401 / 540 / 1,463 ms on the
  earlier build, so the flag alone changed nothing.
• The refill line disappears exactly where predicted: 71.7 and 82.5 and 188.2 ms in the kept arm, absent from the
  deferred arm at 456 and 1,608 tokens.
• Decode is measured independently from the engine's own generated in X ms (Y tok/s) line, not inferred from TTFT.
• The default is unchanged: env-gated, and the arm without the flag matches the published numbers.
• Production untouched: every window restored :8095 (http=200, engine 0.1.58).

Failed:

• First build did nothing: ninja: no work to do. build-sycl-aot's CMAKE_HOME_DIRECTORY is sycl/, so it
  compiles sycl/src/program/generate.cpp, not src/program/generate.cpp. The portable copy was reverted from backup
  and the SYCL copy patched; the patch ships as a diff.
• First A/B run died on BEFORE: unbound variable — an unescaped $ expanded by the outer shell inside the window's
  bash -c; fixed the way the trace driver already does it.

Not run:

• The "how much" question (a reduced refill set) — that is now the open item, not measured here.

Limits

• One build, one card, one model, 6 samples per size; the deferred arm's numbers are so far from the kept arm's that
  sampling error is not the explanation.
• The deferred arm's decode figures at 456 and 1,608 tokens (0.4–0.5 tok/s) come from 8-token generations; a longer
  generation would be needed to characterise the collapse, which is not worth measuring given the conclusion.