Skip to content

[Docs] MTP is shared: say what the fork lets you tune instead - #10

Merged
rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/mtp-tuning
Nov 13, 2026
Merged

rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/mtp-tuning

Conversation

@rwkeyes

@rwkeyes rwkeyes commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Problem and result

The MTP bullet read as if guess-then-check were the fork's advantage, and the paragraph that was meant to correct that
named only two of the fork's three drafting settings. Checking the two servers line by line also turned up one item I
had implied was ours and isn't.

  • Both arms of the 0.2.13 comparison started with --spec 6 --spec-min-p 0.5 and sent no per-request override, so
    drafting contributes nothing to the measured gain. That is now on the page, with the settings.
  • The real difference is control over the drafter, not a different drafter, and it is now a four-row table.
  • spec_min_p per request is stock behaviour: upstream v0.2.13's server already forwards
    strata_tune{pcie_frac, spec_min_p, prefill_pipe, aux_cpus}. The fork's additions are spec (depth, clamped to
    2…startup), lookup (prompt lookup on/off), the spec_window policy, and the TUNE / POST /v1/tune channel that
    changes settings globally without a restart.

Related discussion

none — documentation only.

Approach

Replaced the paragraph under the three engine decisions with: the shared-behaviour statement including the exact
drafting settings both arms used, a four-row table of stock vs fork around drafting, and one line saying none of it was
switched on for the numbers below (spec_window 0, GuideLLM sends no strata_tune). The glossary's MTP entry now
carries the same three additions and the note that spec_min_p per request is on both sides.

Validation

Passed:

  • Both servers read, not assumed. grep -n 'strata_tune' serve/server.py in the two 0.2.13 trees: the stock tree
    forwards pcie_frac, spec_min_p, prefill_pipe, aux_cpus; the fork adds spec (with the comment "clamps it to
    2..--spec … a deeper window than the engine was started with is refused there") and lookup. spec_window exists
    only in the fork (set_spec_window, 0 = off).
  • The measured settings come from the two config files the arms ran: identical --spec 6 --spec-min-p 0.5 --mtp … --mtp-window 8192.
  • Rendering: the new table renders as 3 columns × 5 rows, and the lookup=0|1 cell stays one cell (escaped pipe).
    No broken relative links; all anchors resolve.

Failed:

none observed.

Not run:

  • No measurement of the tuning controls themselves; the page still makes no speed claim about them.

Limits

  • The table describes what each server accepts. It does not claim a measured benefit for any of the fork's four
    additions, and says that spec_window was off for the comparison.

The bullet read as if guess-then-check were the fork's, and the
clarifying paragraph named only two of the fork's three drafting
settings. Both are fixed, and one of the three turned out to be
upstream's already.

Both arms in the 0.2.13 comparison started with --spec 6 --spec-min-p 0.5
and sent no per-request override, so drafting contributes nothing to the
measured gain. That is now stated with the settings, and a small table
gives the real difference - control, not a different drafter:

  draft depth      stock: --spec at startup          fork: also spec=N per request, 2..startup
  prompt lookup    stock: --suffix-draft at startup   fork: also lookup=0|1 per request
  draft confidence stock: spec_min_p per request      fork: the same, plus live global retune
  window policy    stock: none                        fork: spec_window (off in the measurement)

spec_min_p per request is stock behaviour: upstream's server forwards
strata_tune{pcie_frac, spec_min_p, prefill_pipe, aux_cpus} in v0.2.13.
The fork's additions are `spec`, `lookup`, the window policy and the
TUNE/POST /v1/tune channel that changes settings globally without a
restart - checked in both serve/server.py files, not assumed.

The glossary's MTP entry now matches.
@rwkeyes
rwkeyes merged commit 8a61e43 into dirigo/0.2.12-verified Nov 13, 2026
3 checks passed
@rwkeyes
rwkeyes deleted the docs/mtp-tuning branch November 13, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant