Repository navigation
server: tool-call readers search for an absent opener once - #882
Conversation
envelopes and closeCall looked for a DSML opener from each <tool_call> they passed, and hideInto looked for each kind of call opener from each call it passed, so an opener the reply lacked was searched for to its end every time. Each now keeps where an opener is next and searches again only once it has read past it.
|
Thank you, @jishnuvenugopal, for fixing the missing-opener rescans in parsing, closeCall and streaming hiding together. At fe4299b, each cached match remains valid until the cursor passes it, and each call starts with fresh caches. We found no changed parsing or hiding decision. Please scope the linear-time claim to the scans this change fixes. The parser and reply-text integration checks can then cover mixed and incomplete markup. Posted by Ash's AI Assistant Synthena |
jishnuvenugopal
changed the title
|
Thank you for checking the cache lifetimes. Agreed: this only stops the rescans for the DSML opener in |
… against the old search #881's putBack is exact only for replacements without a NUL, which json.quote guarantees for its one caller; its doc line now says so, and the sort line says the cost: one pass over the text, then M log M for M marks. The two timing tests from #882 and #883 now name what they measure (each opener, and each key and value end, searched once) instead of claiming that all tool parsing is linear. A new test parses 20,000 random GLM calls (keys without values, values without closers, repeated keys, stray tags, whitespace) and checks each against the same pairs found by #883's parent loop, written out plainly: the arguments must be identical.
…VIEW names the refused GLM pair requests rightly One line each for #881, #882, #883, #889, #890, #709, #848, #892, #867, #868, #871 and #873, crediting their authors. ZIG-PREVIEW said a two-Mac GLM pair refuses "temperature-1" requests; it refuses any temperature above 0 (backend.zig and engine.zig both test temperature > 0).
|
Thank you, @jishnuvenugopal, for caching each tool-call opener search. #882 is included in 1.0.12 as merge 99505af, at fe4299b. The opener results are reused during parsing, closing and hiding calls. Our 084406f renames the timing test to describe the cached searches. An unmatched namespaced opener can still rescan its suffix, so the claim is limited to the searches this change removes. The macOS suite passes 2,968 of 2,978 tests with 10 skipped; Linux passes 1,368 of 1,375 with 7 skipped. Additional mixed DSML and namespaced differential cases remain owed on our side. Posted by Ash's AI Assistant Synthena |
…n after every call
… against the old search ashhart#881's putBack is exact only for replacements without a NUL, which json.quote guarantees for its one caller; its doc line now says so, and the sort line says the cost: one pass over the text, then M log M for M marks. The two timing tests from ashhart#882 and ashhart#883 now name what they measure (each opener, and each key and value end, searched once) instead of claiming that all tool parsing is linear. A new test parses 20,000 random GLM calls (keys without values, values without closers, repeated keys, stray tags, whitespace) and checks each against the same pairs found by ashhart#883's parent loop, written out plainly: the arguments must be identical.
…VIEW names the refused GLM pair requests rightly One line each for ashhart#881, ashhart#882, ashhart#883, ashhart#889, ashhart#890, ashhart#709, ashhart#848, ashhart#892, ashhart#867, ashhart#868, ashhart#871 and ashhart#873, crediting their authors. ZIG-PREVIEW said a two-Mac GLM pair refuses "temperature-1" requests; it refuses any temperature above 0 (backend.zig and engine.zig both test temperature > 0).
What this changes
envelopesandcloseCall(zig/src/server/tool_parse.zig) looked for a DSML opener from each<tool_call>theypassed, and
hideInto(zig/src/server/reply_text.zig) looked for each kind of call opener from each call it passed.An opener the reply does not contain was searched for to the end of the reply every time. Each now keeps where an
opener is next and searches again only once it has read past that place. The searches only move forward, so they
find what they found before. Only these scans change: a namespaced
<ns:tool_call>opener with no closer is stillsearched past once per opener (
envelopes, tool_parse.zig:75), and the envelope sort is unchanged.Verified outcome
server reads the whole answer again after every new piece of text. A 128 KiB reply of calls took 51 ms to parse at
the end, and 54 ms for the check after each streamed piece.
over after every call.
unchanged: 50,000 varied replies give byte-identical results.
Receipt
Environment:
mainat c7137bb, Zig 0.17.0, Apple M4, macOS 27.0.1. Host code: no model or GPU.Exactness: n/a. No engine change; the reply's tokens and text are the same.
Decode speed before and after: n/a.
bench_openai.py's prompts offer no tools, and these readers only run forrequests that offer tools.
Prompt speed before and after: n/a, for the same reason.
Speed on
<tool_call>{"name":"lookup","arguments":{"n":1}}</tool_call>repeated, in ReleaseSafe as the serverships, fastest of three, c7137bb then this branch:
parsecloseCallhideToolCallsSame output: a local fuzz harness (not part of this change) ran 50,000 edited replies through
parse,closeCall,hideToolCallsandfinishedProseon c7137bb and on this branch. A digest of everything they returned, call idsleft out, is equal (4c8dc88dffd56f19).
Tests run:
a reply of many calls reads in linear time(tool_parse_test.zig):parse,closeCallandhideToolCallson 400 JSON calls and on 3,200. 8x the calls may take at most 24x the time (or under 20 ms), thebound fuzz-cortheon's linear-time test uses. It fails on c7137bb and passes on this branch.
zig build test test-golden -Dcpu=apple_m1, uncached: 164 of 164 steps, 1982 of 1990 tests pass, 8 skip,0 fail (c7137bb: 1981 of 1989, 8 skip); the one more is the new test.
zig build fuzz-cortheonpasses.zig fmt --check build.zig zigis clean. Lean: 527 problems on the base and 527 on this branch, 0 new(
lean_new.py).Not run, and why: no CUDA host here. These readers are server code that both backends use, and their tests need no
GPU.
Checklist
change.)
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Noengine change; not re-run.)
arithmetic changes.)