Repository navigation
[Docs] Say which stream count a reader is actually in - #12
Merged
Merged
Conversation
The table has rows for 1, 2, 4 and 8 streams and said nothing about which of them happens to a person. Now, right under the table: - one stream is one person or one agent loop - ask, read, ask again - and that is where the 22 % applies; - two to four is a few agents or tools on one box, still 17 % and 11 %; - eight is a queue (batch job, eval harness, team). Both servers ran the tests at --batch 0, one request at a time, which is why the two lines meet there; for real concurrency the engine takes --batch/--slots and the numbers would be a different experiment, linked to BATCHING.md. The "what it does not show" line about eight streams now says the real reason: the measured configuration serves one request at a time, so multi-slot serving is untested here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
The comparison table shows 1, 2, 4 and 8 streams and never says which of those a reader will actually be. Without that
mapping the 22 % headline is abstract, and the equal result at eight streams reads as a weakness rather than as a
property of the test rig.
Related discussion
none — documentation only.
Approach
A short block under the table maps the rows to situations: one stream = one person or one agent loop, where the 22 %
applies and which is most desktop use; two to four = a few agents or tools on one box, 17 % and 11 %; eight = a queue
(batch job, evaluation harness, team), where both servers ran
--batch 0— one request at a time — so eight clientsmostly wait and the lines meet.
It also says what to do if you need concurrency: the engine takes
--batch/--slots(2–8 in the usual case, or--parallelin the Terminal), which is a different experiment from the one measured, linked todocs/BATCHING.md.The "what it does not show" line now carries the real reason for the eight-stream tie —
--batch 0, one request at atime — instead of the vaguer "queue-bound".
Validation
Passed:
--batch 0.docs/BATCHING.md("By default Strata serves one request at a time … withparallel:N (the engine's
--batch N, also spelled--slots N)") anddocs/AI_SETUP.md(--parallel N, 1–8).BATCHING.mdlink resolves, GitHub's Markdown API rendersthe new list.
Failed:
none observed.
Not run:
--batch 2–8numbers would be a different experiment.Limits
changes the picture and is out of scope here.