[abhinav-pola]

TL;DR

The Responses client now sends input as a streamed JSON array, one item at a time, instead of the SDK serializing the whole request into one string. On long tau3 conversations that string was enough to OOM trial pods.

What changed?

• send runs input through the SDK's own InputsUnion$outboundSchema up front and passes the SDK input: []. The fetcher then puts the real items back as a streamed body built with effect Stream: fromIterable → rechunk(1) → JSON.stringify per item → intersperseAffixes → encodeText → toReadableStream, sent with duplex: "half".
• Everything else is unchanged: the other fields, headers, extraBody merging, retries and timeouts. If the input fails the SDK schema, the old path runs, so validation errors look the same as before.

Why?

Some tau3 banking trials crashed with JavaScript heap out of memory. In those trials a provider returned a single turn with thousands of parallel KB_search calls, each returning about 13KB of documents. A 7,221-call turn adds 95M characters to the conversation.

The SDK serializes the next request with JSON.stringify, which produces a 98M-character string. That string takes 2 bytes per character because the docs contain non-latin1 characters, and it is UTF-8 encoded on top. These copies come on top of the conversation itself and push the heap past the pod's ~507MB. All 5 epochs share one heap (in_flight: 5), so smaller turns can add up to an OOM too.

This replaces #151 as the memory fix: episodes run exactly as before, and nothing is truncated or ended early.

How to test

bun test src/providers/responses-client.test.ts

The new test sends a request with multi-MB items through the real layer. It checks that the body arrives in more than one chunk and parses to exactly the JSON the SDK would have sent.

Manual repro (not committed): the real makeOpenRouterModelLayer against a local HTTP sink, with real tau3 retrieval over the 698-doc corpus, under node --max-old-space-size=507.

Case	Before	After
1 episode, 7,221 KB_search calls	heap OOM (signal 6), max RSS 1.2GB	OK, heap 307MB, max RSS 758MB
5 concurrent episodes, 2,500 calls each	heap OOM (signal 6)	OK, max RSS 775MB

Benchmark impact

None expected. The wire JSON is the same; only the key order differs, with input first. Trials that used to crash now finish.

Reviewer focus

• streamInputItems in src/providers/responses-client.ts: the body uses chunked transfer encoding, with no Content-Length.
• A conversation that is itself larger than the heap can still OOM. This removes the serialization copies, not the conversation.

Checklist

• Tests cover changed behavior
• Public API or configuration changes are backward compatible, or the break is documented
• Benchmark changes document dataset provenance and licensing
• No credentials, private results, or restricted dataset contents are included
• Documentation is updated where needed

Link to Devin session: https://openrouter.devinenterprise.com/sessions/72a3f13c847645c5b06306d88cec089d
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/72a3f13c847645c5b06306d88cec089d?variant=devin
Requested by: @abhinav-pola

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

• Disable automatic comment, CI, and merge conflict monitoring

---

Original prompt from Abhinav

SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1791572889.759909]: @Devin move the tool call fuzz dataset to the public repo and sync it back to the subtree
</latest_message>

=== BEGIN THREAD HISTORY (in #agents-benchmarks) ===
Abhinav Pola (U090K0G7JF3) [ts=1791572889.759909]: @Devin move the tool call fuzz dataset to the public repo and sync it back to the subtree
=== END THREAD HISTORY ===
Channel ID: C0BAKP8P5C3
Thread URL: https://openrouter.slack.com/archives/C0BAKP8P5C3/p1791572889759909?thread_ts=1791572889.759909&amp;cid=C0BAKP8P5C3

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.