[LukasParke]

Stack 2/3: callModel streamTimeout, armed for every model request

Linear: DEV-723. Base: #136 (watchdog core and typed errors). Next: stall retries (#138). Ports OpenRouterTeam/typescript-sdk#773, re-implemented for the agent engine rather than copied.

callModel(client, {
  model, input, tools,
  streamTimeout: { firstContentMs: 15_000, contentIntervalMs: 30_000 },
});

Design

• One dispatch point. The engine has five betaResponsesSend sites: initial, tool-round follow-up, final (allowFinalResponse), empty-output retry, and unsent-results resume. All five now go through a new sendModelRequest(), so deadlines re-arm for every request. That matches per-step firstChunkMs in vercel/ai.

• Abort composition. With a deadline set, each request gets its own AbortController. dispatchRequestOptions(turnSignal) composes it with the existing pieces:

  • the run signal
  • the caller's RequestOptions.signal
  • the replicated timeoutMs

  On a stall the watchdog aborts that controller. That tears down the hung fetch, and every consumer of the turn rejects with StreamStalledError. The watchdog also receives the composed signal and disarms when it aborts, so a user abort or timeoutMs rejects with its own reason.

• Client-only field. streamTimeout is stripped from the API request on both resolution paths: clientOnlyFields in async-params.ts and the destructure in resolveRequestForContext.

• Off by default. Without streamTimeout, dispatchRequestOptions is byte-for-byte the old behavior. There is no extra signal and no wrapper.

• No unhandled rejections. A consumer that breaks out of a tool-enabled getter while a later turn stalls used to leave executionPromise rejecting unhandled. This adds the same one-line void executionPromise.catch(() => {}) as fix(agent): prevent unhandled rejection when aborting stream #122, with the same text so the two merge cleanly in either order. A regression test fails without it.

• The SDK's idempotent ToolEventBroadcaster.complete() was not needed. Every agent getter in the tool loop awaits executionPromise after draining the broadcaster, so a follow-up stall still reaches all concurrent consumers. A test covers this, as Perry suggested on feat: callModel per-turn stall timeouts (DEV-723 2/5) typescript-sdk#773.

Review-thread pitfalls

• Cumulative timeoutMs across SDK retries (Devin and Perry on feat: stream watchdog core for stalled-stream detection (DEV-723 1/5) typescript-sdk#770 and feat: callModel per-turn stall timeouts (DEV-723 2/5) typescript-sdk#773). Supplying any signal disables the SDK's per-attempt AbortSignal.timeout. The replicated timeout then spans every retryConfig attempt of one request.
  • The agent already had this behavior whenever the run signal was set. With this PR, streamTimeout alone also triggers it.
  • It is not changed here. It is documented as an open question below.
  • The ?? vs || difference for timeoutMs: 0 also predates this PR and is left as is.
• Reaching into client._options.timeoutMs: existing code, unchanged.

Naming

The SDK called the option timeout. This PR uses streamTimeout. CallModelInput maps every ResponsesRequest key, so a bare timeout would shadow any future API field of that name. It is also easy to confuse with RequestOptions.timeoutMs and toolTimeoutMs. Renaming is a one-line change if reviewers prefer parity.

Tests

call-model-stream-timeout.test.ts (12 tests, fake timers). betaResponsesSend is mocked, and each mocked body errors when its request signal aborts, as fetch does. It covers:

• a metadata-only stall, asserting that the request signal is aborted with the stall and the body is torn down
• a healthy stream leaving no timers behind
• the option off by default (no signal, no watchdog, a 10-minute gap is fine)
• a mid-generation stall with retryable: false
• five concurrent consumers rejecting with the same error
• a run-signal abort reported as itself
• RequestOptions preserved
• streamTimeout never sent to the API (static and async-resolved requests)
• in the tool loop:
  • per-request re-arm (turn 1 passes, turn 2 stalls, separate signals)
  • total time over the deadline being fine when every turn is within it
  • concurrent consumers rejecting on a follow-up stall
  • an early consumer exit not leaving an unhandled rejection

Port notes (Python / Go bots)

Carry:

• the streamTimeout option and its client-only stripping
• the single dispatch helper, so every request site re-arms the deadlines
• the per-request cancel token, composed with the run cancellation and the per-request timeout
• the rule that an abort disarms the watchdog
• the "early consumer exit must not leak an unobserved failure" rule (in Python: a background task exception that is never retrieved)

The AbortSignal.any details and the SDK RequestOptions quirk are TypeScript-specific, but the Go and Python ports should check whether their own SDK disables per-attempt timeouts when a cancel token is supplied.

Open questions

1. Keep timeoutMs cumulative across SDK retryConfig attempts (existing behavior), or move to per-attempt? A per-attempt fix needs either an SDK hook or not passing signal at all.
2. Option name: streamTimeout (this PR) or timeout (SDK parity)?

Verification

CI only runs on PRs into main, so this stacked PR shows no checks. Everything below was run locally on this branch:
pnpm install --frozen-lockfile, pnpm typecheck, pnpm lint, pnpm test (agent 1328, mcp 12, coverage ratchet holds), sentrux gate (no degradation).

Once this stack lands, OpenRouterTeam/typescript-sdk#770, OpenRouterTeam/typescript-sdk#773, OpenRouterTeam/typescript-sdk#774 and OpenRouterTeam/typescript-sdk#775 can be closed.

🤖 Generated with Claude Code