Skip to content

[Docs] Explain the 0.2.13 improvement instead of tabulating it - #5

Merged
rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/explain-0213-gain
Nov 13, 2026
Merged

rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/explain-0213-gain

Conversation

@rwkeyes

@rwkeyes rwkeyes commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Problem and result

The measured block on the front page tabulated the comparison but did not explain it: a reader got four percentages
and a parity row and had to assemble the claim themselves, while the strongest parts of the measurement (the 210 ms
absolute gap, the extra answers per window, the 24-token extraction where stock returns nothing) were either a single
crammed row or absent from the page entirely.

Before: "First token, 1 stream — 739 ms — 949 ms — −22.1 %", with the mechanism in one dense sentence and
four caveats in the surrounding paragraph. After: the claim first, the evidence under it, the two framings of the
same number shown once and explained, and a short "what this does not claim".

Related discussion

none — documentation only.

Approach

  1. Claim first. A "short version" paragraph states that the first token arrives 11–22 % sooner and that four
    streams finish three more answers per 45 s window, and that nothing is traded for it because the engine is at
    parity on every leg.
  2. Absolute next to relative. The table gains the absolute gap (210 / 541 / 812 ms) beside the percentage, and a
    fifth column giving the same measurement the other way round ("stock needs 28.4 % longer"). One sentence says they
    are one number and which to quote, so nobody can accuse the page of picking whichever direction flatters it.
  3. Throughput out of the latency table. Finished requests and output tokens per fixed window get their own table
    with the granularity stated, instead of a row that mixed requests and percentages.
  4. Mechanism as a contrast. "Why the gap exists" now compares what each server does at a fresh prompt (stock
    opens the cache and pays the pool work; the fork recognises one-shot traffic, serves with the cache off, switches
    back on for a conversation), which also explains why the gap is zero at eight streams.
  5. The other measured win. The 24-token extraction — stock returns '', the fork returns 2024-06-09 with zero
    reasoning characters — and the green suite (1,537 tests against stock's 1,512 with one failure) are now on the
    page, cited to the report.
  6. Caveats compressed. "What this does not claim" keeps three sentences; the positional-drift bound moved into
    the Measured-on block, next to the hardware and the date.

Validation

Passed:

  • Every figure is unchanged and still traces to the report. 739/2,622/6,688/9,641 ms against 949/3,163/7,500/9,659;
    the absolute gaps were computed from those medians (949−739=210, 3163−2622=541, 7500−6688=812); the reciprocal
    percentages (28.4 / 20.6 / 12.1 %) are 100·(stock/fork − 1) of the same numbers, printed here to one decimal.
  • Throughput and extraction were already verified against guidellm/*.json and the probe output in
    dirigo-results/0.2.13-port/README.md.
  • Links and anchors: 0 broken relative links; the new in-page anchor and the report link resolve; both new tables
    have consistent column counts (4 and 3 data columns).
  • Rendering: GitHub's Markdown API returns 9 tables, 2 images and 3 blockquotes for the page, with the new tables
    parsing as tables.

Failed:

none observed.

Not run:

  • No new measurement and no build; this is a documentation change over data measured 2026-11-09.

Limits

  • One pass per arm. The drift bound (2–4 %) is quoted from the companion run and is the reason the one-stream
    request count is reported but not claimed.
  • The claim is scoped to unique-prompt traffic; the page says where it does not apply rather than generalising it.

The measured block listed numbers a reader had to assemble into a claim.
It now says the claim first, then backs it:

- a "short version" paragraph: first token 11-22 % sooner, three more
  answers per 45 s window at four streams, and nothing traded for it
  because the engine is at parity on all eight engine-level legs;
- the table gains an absolute column (210 / 541 / 812 ms) next to the
  percentage, and a second column showing the same measurement read the
  other way ("stock needs 28.4 % longer"), with one sentence saying they
  are one number and which to quote;
- a separate table for work done in the fixed window, so throughput is no
  longer a row crammed into the latency table;
- "why the gap exists" now contrasts the two servers behaviour at a fresh
  prompt instead of naming the router;
- "the same config also changes answers": the 24-token extraction probe
  (stock returns '', the fork returns 2024-06-09) and the green suite,
  which were only in the report before;
- "what this does not claim" kept to three sentences, with the drift bound
  moved into the Measured-on block where the numbers are cited.

The chart's numbers are unchanged. The same two-way framing is applied to
the headline table in dirigo-results/0.2.13-port/README.md, so the front
page and the evidence say the same thing in the same words.
@rwkeyes
rwkeyes merged commit df05e44 into dirigo/0.2.12-verified Nov 13, 2026
3 checks passed
@rwkeyes
rwkeyes deleted the docs/explain-0213-gain branch November 13, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant