Repository navigation
[Docs] Explain the engine's performance on the front page, early - #2
Merged
Merged
Conversation
The front page explained what this fork measured before it explained why the engine is fast at all, and left the reader with the impression that the fork's server levers were where the speed comes from. They are not: the performance is upstream Strata's, and it is the reason the project is interesting. New section, placed directly after what Strata is and before the routes: - the four decisions that make it fast, in plain words (only the experts a word needs; prompts read in 8,192-token pieces; the 28.8 GB table on the SSD; guess-then-check), with the numbers upstream states for each; - a release-by-release table of what actually changed performance, each row against the release before it: 0.1.13 prompt chunks (+125% prompt on an RTX 5070), 0.2.0 zero-copy and I/O prefetch (1.7-2.9x decode), 0.2.4 exact CPU rows (+25% decode), 0.2.7 resident-expert routing (+15%/+37%), R9700 (+22% prompt, +31% decode), layer-major prefill on two cards (141.8 s -> 27.7 s to first word after a 32K prompt); - one end-to-end row from this branch's own verified card, and an explicit statement that the fork's delta is smaller and server-side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
The front page explained what this fork measured before it explained why the engine is fast at all, and it left
the impression that the fork's server levers are where the speed comes from. They are not: the performance is
upstream Strata's (Niko1221's engine), and it is the most important thing about the project.
Before: the reader met "release 0.2.12 verified snapshot" and the fork's table of changes before any statement
about what the engine does faster than the alternatives. After: a Why it is fast section sits directly below
what Strata is — four plain-language decisions plus the measured release-by-release record — and the fork's own
delta is explicitly described as smaller and server-side.
Related discussion
none — documentation only.
Approach
Four sections inside one heading, ordered so a newcomer can stop after the first three bullets:
free GB holds ~700 more, CPU works in parallel with the card); prompts read in 8,192-token pieces;
the 28.8 GB lookup table on the SSD; guess-then-check (2.4–3.2 tokens per pass, answers 1.6–1.8× sooner).
0.1.13 prompt chunks on an RTX 5070 (572 → 1,290 tok/s Q2_0), 0.2.0 zero-copy and I/O prefetch (1.7/2.6/2.9×
decode), 0.2.4 exact CPU rows (4.08 → 5.11 tok/s), 0.2.7 resident-expert routing (+15% story, +37% code on
an RTX 3060; +22% prompt and +31% decode on an R9700), and layer-major prefill on two cards (first word after
a 32K prompt 141.8 s → 27.7 s).
one workload.
a pointer to
DIRIGO.md, and one sentence naming the fork's own delta as server-side and smaller.Every figure is upstream's published measurement, not a new one: the rows come from the release notes in
docs/and the engine history table indocs/DETAILS.md.Validation
Passed:
docs/release-notes-*.mdanddocs/DETAILS.md; none is estimated or rounded beyond what the source states (1.7 / 2.6 / 2.9×from1.70 / 2.56 / 2.88x, the rest verbatim).DIRIGO.mdspeedsection,
#what-this-fork-changes) resolve against their headings.the "Measured on" blockquote, as intended.
Failed:
none observed.
Not run:
on this branch's machine for this change. What stays unverified here is whether any figure still holds on a
current release — they are quoted as release history, not as a current claim.
Limits
is the model this branch serves.
docs/. Extending it means taking 0.2.8–0.2.13 from the GitHub release pages.