Skip to content

[Docs] Explain the engine's performance on the front page, early - #2

Merged
rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/why-it-is-fast
Nov 13, 2026
Merged

rwkeyes merged 1 commit into
dirigo/0.2.12-verifiedfrom
docs/why-it-is-fast

Conversation

@rwkeyes

@rwkeyes rwkeyes commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Problem and result

The front page explained what this fork measured before it explained why the engine is fast at all, and it left
the impression that the fork's server levers are where the speed comes from. They are not: the performance is
upstream Strata's (Niko1221's engine), and it is the most important thing about the project.

Before: the reader met "release 0.2.12 verified snapshot" and the fork's table of changes before any statement
about what the engine does faster than the alternatives. After: a Why it is fast section sits directly below
what Strata is — four plain-language decisions plus the measured release-by-release record — and the fork's own
delta is explicitly described as smaller and server-side.

Related discussion

none — documentation only.

Approach

Four sections inside one heading, ordered so a newcomer can stop after the first three bullets:

  • The four decisions: only the experts a word needs are computed (24,576 specialists, ~10 per word, each
    free GB holds ~700 more, CPU works in parallel with the card); prompts read in 8,192-token pieces;
    the 28.8 GB lookup table on the SSD; guess-then-check (2.4–3.2 tokens per pass, answers 1.6–1.8× sooner).
  • What upstream's releases changed, one row per release, each measured against the release before it:
    0.1.13 prompt chunks on an RTX 5070 (572 → 1,290 tok/s Q2_0), 0.2.0 zero-copy and I/O prefetch (1.7/2.6/2.9×
    decode), 0.2.4 exact CPU rows (4.08 → 5.11 tok/s), 0.2.7 resident-expert routing (+15% story, +37% code on
    an RTX 3060; +22% prompt and +31% decode on an R9700), and layer-major prefill on two cards (first word after
    a 32K prompt 141.8 s → 27.7 s).
  • A "Measured on" block in the repository's house style, stating that each row is one machine, one model and
    one workload.
  • One end-to-end row from the card this branch verified (1,404 tok/s prompt at 4K, 80.8 tok/s decode), with
    a pointer to DIRIGO.md, and one sentence naming the fork's own delta as server-side and smaller.

Every figure is upstream's published measurement, not a new one: the rows come from the release notes in
docs/ and the engine history table in docs/DETAILS.md.

Validation

Passed:

  • Every number traced. All 32 figures in the new section were grepped back to docs/release-notes-*.md and
    docs/DETAILS.md; none is estimated or rounded beyond what the source states (1.7 / 2.6 / 2.9× from
    1.70 / 2.56 / 2.88x, the rest verbatim).
  • Links and anchors: 0 broken relative links in the README; both cross-page anchors (DIRIGO.md speed
    section, #what-this-fork-changes) resolve against their headings.
  • Rendering: through GitHub's Markdown API the section renders as one 3-column table with 6 data rows and
    the "Measured on" blockquote, as intended.

Failed:

none observed.

Not run:

  • Not re-measured. These are the upstream project's numbers on the hardware each row names; nothing was run
    on this branch's machine for this change. What stays unverified here is whether any figure still holds on a
    current release — they are quoted as release history, not as a current claim.
  • No build or serve, as this is a documentation change.

Limits

  • The section is Qwen3.8-Flash-Next-specific where it names expert counts and the PLE table size, because that
    is the model this branch serves.
  • Upstream is now at 0.2.13; the table stops at 0.2.7 because that is the last release note set carried in
    docs/. Extending it means taking 0.2.8–0.2.13 from the GitHub release pages.

The front page explained what this fork measured before it explained why the
engine is fast at all, and left the reader with the impression that the fork's
server levers were where the speed comes from. They are not: the performance is
upstream Strata's, and it is the reason the project is interesting.

New section, placed directly after what Strata is and before the routes:

- the four decisions that make it fast, in plain words (only the experts a word
  needs; prompts read in 8,192-token pieces; the 28.8 GB table on the SSD;
  guess-then-check), with the numbers upstream states for each;
- a release-by-release table of what actually changed performance, each row
  against the release before it: 0.1.13 prompt chunks (+125% prompt on an
  RTX 5070), 0.2.0 zero-copy and I/O prefetch (1.7-2.9x decode), 0.2.4 exact CPU
  rows (+25% decode), 0.2.7 resident-expert routing (+15%/+37%), R9700 (+22%
  prompt, +31% decode), layer-major prefill on two cards (141.8 s -> 27.7 s to
  first word after a 32K prompt);
- one end-to-end row from this branch's own verified card, and an explicit
  statement that the fork's delta is smaller and server-side.
@rwkeyes
rwkeyes merged commit 6d73d7d into dirigo/0.2.12-verified Nov 13, 2026
3 checks passed
@rwkeyes
rwkeyes deleted the docs/why-it-is-fast branch November 13, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant