Repository navigation
serve: honour conv_route_batch_after from the config, not only the flag - #16
Merged
Merged
Conversation
The launcher writes "conv_route_batch_after" into the run config and starts the server without --conv-route-batch-after, while the server read only the flag (argparse default 4). Every deployment therefore ran 4 whatever the config said. Measured on 2026-11-14: two mixed-traffic windows whose configs differed only in that key (4 against 1) produced identical results on all eleven steps - a conversation's second turn 0.587 s in one arm and 0.555 s in the other, the single-shot steps matching to 0.02 s. Passing the flag by hand reproduced the 2026-11-09 finding at once (C2 2.465 s, a re-prefill). resolve_batch_after() in serve/conv_router.py resolves the value: explicit flag, then the config key, then 4; malformed values fall back rather than raise. The flag's argparse default becomes None so an absent flag is distinguishable from an explicit 4. A test pins all four cases. Not for dirigo/0.2.12-verified: that branch keeps serve/ byte-identical to the deployed snapshot by design, and the deployment is unaffected today (its config says 4). Land this with the next tree bump.
This was referenced Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
conv_route_batch_afterin the run config was ignored.serve/server.pyreada.conv_route_batch_after— the--conv-route-batch-afterflag, default 4 — whilebin/strata-b70-serve.shwrites the key into the config JSON andstarts the server with only
--engine,--configand--port. A deployment that set 1 or 2 in the config got 4, andnothing reported the mismatch.
The fix resolves the value in one place: explicit flag → config key → 4, with malformed values falling back
instead of raising. The flag's argparse default becomes
Noneso "absent" is distinguishable from "explicitly 4".How it was found
A mixed-traffic sweep set
conv_route_batch_afterin two configs (4 and 1) and got results identical to 0.02 s on alleleven steps. Passing
--conv-route-batch-after 1by hand reproduced the 2026-11-09 result immediately — theconversation's second turn re-prefilled at 2.465 s against 0.587 s. So the config key was inert, not the router.
Validation
Passed:
python -m pytest serve/test_conv_router.py -q→ 12 passed, including the newtest_the_config_key_is_honoured_when_the_flag_is_absent, which pins: config 2 → 2, config"3"→ 3, flag 1 beatsconfig 2 → 1, no flag and no key → 4, and
0/-1/True/"off"/[1]/{}→ 4 rather than an exception.import serve.serverin the fix worktree → ok, andresolve_batch_after(None, {"conv_route_batch_after": 2})→ 2at the real import path, so the new name resolves where the server uses it.
python -m pytest serve/test_server.py -q→ 376 passed, 526 subtests passed in 110 s, on the fix branch in athrowaway worktree (the default branch's tree was never touched).
default=Noneon the flag, andbatch_after=resolve_batch_after(...)at the call site.Failed:
-k "conv_route or batch_after"onserve/test_server.pydeselects all 376 tests — the wiring has no test there,which is why the key could rot unnoticed. The new test lives with the router, where the decision does. (An earlier
attempt at the same run returned pytest exit 4 because
--timeoutis not installed in this venv, and exit 5 from thefiltered run; neither was a test failure.)
Merge note — do not merge into
dirigo/0.2.12-verifiedThat branch keeps
src/,include/,sycl/,metal/,serve/,setup.pyandCMakeLists.txtbyte-identical tothe deployed snapshot so a re-clone of v0.2.12 plus the carry still reproduces the tree hash (
DIRIGO.md). Merging thiswould break that check. The deployment is not affected today — its launcher config says 4, which is what it gets — so
this should land with the next tree bump.