From e030ddb26cbdd1d66d7e05ab48e1907a7c8f1f0f Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 18 Sep 2026 03:42:55 +0900 Subject: [PATCH 1/2] fix(web-search): replay executed bridge searches to the destination The web-search passthrough bridge runs an intercepted web_search proxy-side and shows the caller a hosted web_search_call cell. The caller replays that cell on every later turn, so the destination received an item type it never produced, carrying a query and sources but no result, and usually searched again. Record each executed search in a process-local memo scoped to the upstream destination and keyed by the cell id, then restore the destination's own function_call and function_call_output in the cell's place before the next turn's first leg is dispatched. The memo stores exactly what a continuation leg would have sent, so a replayed turn and a continued turn show the destination one consistent conversation. A miss leaves the replayed item untouched: no second search is billed and no result text is invented. Closes #4587 --- .../010_roadmap.md | 131 ++++++++ scripts/test-layout/layout.json | 1 + src/adapters/openai-responses/passthrough.ts | 11 +- .../openai-responses/tool-output-recovery.ts | 75 +++++ src/responses/bridge-search-replay-cache.ts | 152 +++++++++ src/server/responses/passthrough-delivery.ts | 5 + src/web-search/passthrough-bridge.ts | 27 +- structure/runtime.md | 17 +- tests/fixtures/test-layout-expected.json | 1 + .../web-search-bridge-replay.test.ts | 295 ++++++++++++++++++ 10 files changed, 706 insertions(+), 9 deletions(-) create mode 100644 devlog/_plan/260918_ld_search_bridge_and_thinking_replay/010_roadmap.md create mode 100644 src/responses/bridge-search-replay-cache.ts create mode 100644 tests/web-search/web-search-bridge-replay.test.ts diff --git a/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/010_roadmap.md b/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/010_roadmap.md new file mode 100644 index 0000000000..06a6d55444 --- /dev/null +++ b/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/010_roadmap.md @@ -0,0 +1,131 @@ +# LD — bridged web-search replay and Anthropic thinking replay + +Delivery lane R3-LD. The completion bar for this lane is not "one turn +succeeded" but "the next turn inherits exactly the same result". Local test +execution is prohibited here: every claim below is backed by source reading and +by hosted CI at an exact head. + +## Units + +| Unit | Item | Kind | Write scope | +|---|---|---|---| +| U1 | #4587 bridged hosted `web_search` result is not replayed to the destination | new implementation | `src/responses/bridge-search-replay-cache.ts` (new), `src/web-search/passthrough-bridge.ts`, `src/adapters/openai-responses/passthrough.ts`, `src/adapters/openai-responses/tool-output-recovery.ts`, `structure/` owner doc, one new test file plus its two layout entries | +| U2 | #4429 key-auth Responses gateway echoes hosted `web_search` as a client `function_call` | re-judge at HEAD, closure recommendation | none — assessment only | +| U3 | #3719 Anthropic thinking/`redacted_thinking` replay through proxy-auth translation | verify at HEAD, closure recommendation | none — assessment only | +| U4 | #3952, #4783, #4900 | review only, scoped | none | + +U1 is the only unit that writes code. U2 and U3 are re-judged against current +source because both issue bodies predate the commits that changed the answer; +the host owns every close decision. + +## U1 — #4587 + +### What is actually broken + +`src/web-search/passthrough-bridge.ts` intercepts the destination's +`function_call` named `web_search`, runs the search proxy-side, and shows the +caller a hosted `web_search_call` cell whose id is a proxy-minted +`ws_`. Two paths reach that state: + +- `endAfterSearch` (a leg mixing the search with a client-executed tool call) + ends the turn on that leg, so no continuation carries the result upstream. +- The ordinary continuation path does re-POST `function_call` + + `function_call_output` to the destination through `appendBridgeSearchTurn`, + but only inside that turn. + +In both cases the caller's own history now holds a hosted `web_search_call` +item. On the next turn it replays that item, and the destination receives an +item type it never produced, with no result text and no matching +`function_call`/`function_call_output` pair. The observable effect is a wasted +round trip: the model usually searches again. + +### Where the fix has to live + +Not in the bridge. By the time the bridge wraps a turn, that turn's first leg +is already on the wire, so the rewrite must happen before dispatch. The +existing pre-dispatch rewrites of outbound `input` are +`backfillWebSearchQueries` and `repairOrphanedInputItems` in +`src/adapters/openai-responses/tool-output-recovery.ts`, applied from +`src/adapters/openai-responses/passthrough.ts`. The restore joins them there. + +### The memo + +A process-local, bounded, expiring memo records what the bridge executed: + +- Key: the destination identity (the existing salted digest of the provider + base URL from `src/responses/reasoning-replay-cache.ts`) plus the synthesized + cell item id. The cell id is a v4 UUID this proxy mints, so it cannot collide + across conversations; the destination scope is what stops one provider's + executed call from being replayed into another provider's conversation. +- Value: the destination's `call_id`, its original item id, the original + arguments text, and the executed result text. +- Bounds: entry count, total bytes, and TTL, following the discipline already + established by the reasoning replay cache. Result text lives in memory only + and is never logged, serialized, or exported. + +### The invariant that matters most + +**A memo miss is a no-op.** If the cell id is unknown, expired, or was recorded +against a different destination, the replayed `web_search_call` item is left +exactly as it is. The lane must never re-run the search to recover a lost +result, and must never synthesize result text. Re-running would bill a second +search the caller did not ask for and would answer the model with a different +search than the one its history claims; synthesizing would put words in the +destination's own mouth. Both are the easiest wrong fix available here, and +neither is permitted. + +### Regression pins + +1. A replayed `web_search_call` whose id is in the memo becomes the + destination's `function_call` followed by its `function_call_output`, in + that order, at the item's original position. +2. A replayed `web_search_call` with no memo entry is byte-identical to the + input item. +3. An entry recorded against one destination is not restored for another. +4. An expired entry behaves exactly like a miss. +5. Providers without `webSearchBridge.enabled` allocate nothing and their + outbound body keeps its original object identity. + +## U2 — #4429 + +The issue asked for two things: a non-Ollama executor so a key-auth Responses +gateway can arm the bridge, and the intercepted call executed proxy-side with +the conversation continued upstream. Both shipped. The reporter's remaining +concern — the destination never learning the result — is #4587 and is U1 here. +The hosted/client distinction the issue turns on is `isWebSearchCallItem` +versus `isClientExecutedItem` in the bridge: a namespaced `ns__web_search` is +treated as a client-owned tool and is never intercepted, which is the boundary +that keeps the undeclared-tool guard's authority intact. + +Closure recommendation and the evidence for it are recorded in `020`. + +## U3 — #3719 + +The issue body states that the inbound translator drops assistant `thinking` +and `redacted_thinking` blocks. That is no longer true at HEAD: +`src/claude/inbound.ts` encodes the Anthropic signature and the opaque +redacted payloads into bounded `ocxr1` envelopes, and +`src/adapters/anthropic.ts` replays them as `redacted_thinking` blocks +followed by a signed `thinking` block. + +This lane verifies that path and keeps it separate from the different question +of carrying signature data to other providers. The gate that enforces the +separation is `isLikelyRealAnthropicThinkingSignature`: a block is replayed +only when its signature looks like a real upstream-issued one, so a proxy-minted +continuity value or another provider's opaque blob is dropped rather than +forwarded as an Anthropic signature. Manufacturing a signature is out of scope +and stays that way. + +The remainder of #3719 is measurement — a controlled cache creation/read +comparison across continuation turns — which this lane cannot produce under the +no-local-execution rule. + +## Operating constraints + +- No local test, typecheck, install, or GUI build. Verification is source + reading plus hosted CI at the exact final head. +- No merging, no direct pushes to `dev`, no closing issues or pull requests. + The lane ends with an open PR and exact-head CI evidence, or with a written + closure recommendation handed to the host. +- No flake management: no widened timeouts, added retries, platform skips, or + masking. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index adebdb38bb..e96e059e31 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1426,6 +1426,7 @@ "warmup.test.ts": "codex-integration", "web-search-anthropic.test.ts": "web-search", "web-search-backend-union.test.ts": "web-search", + "web-search-bridge-replay.test.ts": "web-search", "web-search-candidates.test.ts": "web-search", "web-search-parse.test.ts": "web-search", "web-search-passthrough-bridge.test.ts": "web-search", diff --git a/src/adapters/openai-responses/passthrough.ts b/src/adapters/openai-responses/passthrough.ts index 435e928b00..1184ac6a1c 100644 --- a/src/adapters/openai-responses/passthrough.ts +++ b/src/adapters/openai-responses/passthrough.ts @@ -36,7 +36,8 @@ import { scrubOcxCompactionItems, stripCanonicalOnlyToolFields, stripCanonicalOn import { stripCanonicalForwardPromptCacheOptions, stripDeprecatedPromptCacheRetention } from "./prompt-cache"; import { isPlainObject } from "./internal"; import { normalizeToolSchemas, promoteClientLoadedTools, stripUnsupportedHostedTools } from "./tool-schema"; -import { annotateEmptyResponsesToolOutputs, backfillWebSearchQueries, normalizeResponsesToolResultAdjacency, repairOrphanedInputItems, repairOversizedReplayCallIds, repairUnidentifiedToolOutputItems } from "./tool-output-recovery"; +import { annotateEmptyResponsesToolOutputs, backfillWebSearchQueries, normalizeResponsesToolResultAdjacency, repairOrphanedInputItems, repairOversizedReplayCallIds, repairUnidentifiedToolOutputItems, restoreBridgedWebSearchCalls } from "./tool-output-recovery"; +import { bridgeSearchReplayScope } from "../../responses/bridge-search-replay-cache"; import { applyTierDecisionToResponsesBody, normalizeCanonicalForwardContinuationEnvelope, normalizeCanonicalForwardPromptEnvelope, stripCanonicalForwardSamplingParams, stripPreviousResponseId, stripStatefulResponsesParams, stripUnsupportedForwardParams } from "./canonical-forward"; import { normalizeImageGenClientTools, preferConfiguredHostedTools } from "./image-gen"; import { stripMuseSparkUnsupportedWebSearchFields, stripOpenAiOnlyWebSearchFields } from "./web-search"; @@ -321,6 +322,14 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): outBody = repairOversizedReplayCallIds(outBody); } outBody = stripUnsupportedReasoningSummaryDelivery(outBody, parsed.modelId); + // #4587: on a bridged provider, hand the destination back the search call and result the + // proxy executed on its behalf, in place of the hosted cell the caller replays. Scoped to + // this destination and recorded by the bridge itself, so a provider without the opt-in + // computes no identity and keeps the body reference it already had. This runs before the + // query backfill below because a restored cell is no longer a web_search_call to repair. + if (provider.webSearchBridge?.enabled === true) { + outBody = restoreBridgedWebSearchCalls(outBody, bridgeSearchReplayScope(provider.baseUrl)); + } // Repair stored history from before the bridge emitted both keys, in either // direction: a conversation that already recorded a web_search_call replays it // every turn, and a strict parser rejects the whole request over the missing key — diff --git a/src/adapters/openai-responses/tool-output-recovery.ts b/src/adapters/openai-responses/tool-output-recovery.ts index ec67cbaee9..bd05234cbb 100644 --- a/src/adapters/openai-responses/tool-output-recovery.ts +++ b/src/adapters/openai-responses/tool-output-recovery.ts @@ -1,6 +1,7 @@ import { createHash } from "node:crypto"; import { EMPTY_TOOL_OUTPUT_ANNOTATION, isWhitespaceOnlyTextPartArray } from "../empty-tool-output-annotation"; import { isPlainObject } from "./internal"; +import { peekBridgeSearchReplay } from "../../responses/bridge-search-replay-cache"; const MAX_RESPONSES_CALL_ID_LENGTH = 64; @@ -265,6 +266,80 @@ export function backfillWebSearchQueries(body: unknown): unknown { return changed ? { ...body, input } : body; } +/** + * Give a bridged destination back its own search call and result (issue #4587). + * + * When `providers..webSearchBridge` is armed, the proxy intercepts the destination's + * `function_call` named `web_search`, runs the search, and shows the CALLER a hosted + * `web_search_call` cell. The caller stores that cell and replays it on every later turn, so the + * destination receives an item type it never produced, carrying a query and sources but no result. + * It typically responds by searching again. + * + * This restores the exchange the destination actually had: the cell becomes the destination's own + * `function_call`, immediately followed by the `function_call_output` the bridge produced for + * it, in the cell's original position. It runs before the first leg of the next turn is + * dispatched, which is the only place it can run — by the time the bridge wraps a turn, that + * turn's first leg is already on the wire. + * + * Three things it deliberately does not do: + * - It never re-runs a search. A missing memo entry means the result is gone, and paying for a + * second search would answer the model with a different search than its history claims. + * - It never invents result text. A miss leaves the item exactly as the caller sent it, which is + * the behaviour every unbridged conversation already has. + * - It never restores a call id the body already carries. If the history somehow holds that + * `function_call` too, emitting a second one would be a duplicate the upstream must reject. + * + * Entries are scoped to the upstream destination, so a history replayed against a different + * provider cannot resurrect a call that provider never made. Callers pass `undefined` for any + * provider without the bridge armed, and the common path then returns the original reference. + */ +export function restoreBridgedWebSearchCalls(body: unknown, destinationScope: string | undefined): unknown { + if (destinationScope === undefined) return body; + if (!isPlainObject(body) || !Array.isArray(body.input)) return body; + const input = body.input; + + // Cheap pre-check: nothing to do for a conversation that carries no hosted search cell at all, + // which is every turn before the model's first bridged search. + let hasCell = false; + for (const item of input) { + if (isPlainObject(item) && item.type === "web_search_call" && typeof item.id === "string") { + hasCell = true; + break; + } + } + if (!hasCell) return body; + + const occupiedCallIds = new Set(); + for (const item of input) { + if (isPlainObject(item) && typeof item.call_id === "string") occupiedCallIds.add(item.call_id); + } + + let changed = false; + const restored: unknown[] = []; + for (const item of input) { + if (isPlainObject(item) && item.type === "web_search_call" && typeof item.id === "string") { + const memo = peekBridgeSearchReplay(destinationScope, item.id); + if (memo && !occupiedCallIds.has(memo.callId)) { + changed = true; + occupiedCallIds.add(memo.callId); + restored.push({ + type: "function_call", + ...(memo.sourceItemId ? { id: memo.sourceItemId } : {}), + call_id: memo.callId, + name: memo.name, + // The bridge records the complete arguments text from the call's own done frame; the + // empty-object fallback matches what a continuation leg would have sent. + arguments: memo.argumentsText.length > 0 ? memo.argumentsText : "{}", + }); + restored.push({ type: "function_call_output", call_id: memo.callId, output: memo.output }); + continue; + } + } + restored.push(item); + } + return changed ? { ...body, input: restored } : body; +} + export function repairOrphanedInputItems(body: unknown, dropReasoning: boolean, synthesizeMissingCallOutputs = false): unknown { if (!isPlainObject(body) || !Array.isArray(body.input)) return body; const input = body.input; diff --git a/src/responses/bridge-search-replay-cache.ts b/src/responses/bridge-search-replay-cache.ts new file mode 100644 index 0000000000..6de24cdce3 --- /dev/null +++ b/src/responses/bridge-search-replay-cache.ts @@ -0,0 +1,152 @@ +/** + * Process-local memo pairing a bridged hosted `web_search` cell with the destination's own + * call and the result the proxy executed for it (issue #4587). + * + * The web-search passthrough bridge (`src/web-search/passthrough-bridge.ts`) intercepts the + * destination's `function_call` named `web_search`, runs the search itself, and shows the + * caller a hosted `web_search_call` cell under a proxy-minted `ws_` id. The caller + * stores that cell and replays it on every later turn. The destination, which never produced a + * `web_search_call` in its life, then sees an unknown item type carrying a query and sources + * but no result — so it usually just searches again. + * + * This memo is what lets the pre-dispatch rewrite in the Responses adapter put the destination's + * own `function_call` and `function_call_output` back in that item's place. It records exactly + * what `appendBridgeSearchTurn` would have written onto a continuation leg, so a replayed turn + * and a continued turn show the destination the same conversation. + * + * Scope. Entries are keyed by the upstream destination in addition to the cell id. The cell id is + * a v4 UUID minted here, so it cannot collide across conversations, but an unscoped key would let + * a history replayed against a DIFFERENT provider resurrect a call that provider never made. + * + * Bounds and privacy. Result text is web content the caller already received, but it is still + * request-derived data: it lives in memory only, is never logged, serialized, or exported, and is + * bounded by entry count, total bytes, and TTL so a long-lived proxy cannot grow without limit. + * + * A miss is deliberately indistinguishable from "no entry": the caller leaves the replayed item + * alone. Neither re-running the search nor inventing a result is an acceptable recovery. + */ + +import { reasoningReplayDestinationIdentity } from "./reasoning-replay-cache"; + +const MAX_ENTRIES = 64; +const MAX_TOTAL_BYTES = 512 * 1024; +const TTL_MS = 60 * 60 * 1000; + +export interface BridgeSearchReplayEntry { + /** The destination's own call id, as it appeared on the intercepted item. */ + callId: string; + /** The destination's own item id, replayed when the upstream supplied one. */ + sourceItemId?: string; + /** The tool name the destination called, recorded rather than assumed. */ + name: string; + /** The intercepted call's complete arguments text. */ + argumentsText: string; + /** The tool result the bridge produced for that call. */ + output: string; +} + +interface StoredEntry { + entry: BridgeSearchReplayEntry; + bytes: number; + at: number; +} + +const entries = new Map(); +let totalBytes = 0; +let clockForTests: (() => number) | null = null; + +const now = (): number => clockForTests?.() ?? Date.now(); + +/** + * Identify the upstream destination a bridged search belongs to. + * + * Reuses the salted process-local destination digest the reasoning replay cache already defines, + * so both stores agree on what "the same upstream" means and neither invents a second notion of + * destination identity. + */ +export function bridgeSearchReplayScope(baseUrl: string | undefined): string | undefined { + return reasoningReplayDestinationIdentity(baseUrl); +} + +function keyFor(scope: string, cellItemId: string): string { + return scope + "\u0000" + cellItemId; +} + +function drop(key: string, stored: StoredEntry): void { + entries.delete(key); + totalBytes -= stored.bytes; +} + +function sweep(at: number): void { + for (const [key, stored] of entries) { + if (at - stored.at >= TTL_MS) drop(key, stored); + } + // Map iteration is insertion-ordered, so the oldest surviving entry is always the first one. + while (entries.size > MAX_ENTRIES || totalBytes > MAX_TOTAL_BYTES) { + const oldest = entries.entries().next(); + if (oldest.done) { + totalBytes = 0; + return; + } + drop(oldest.value[0], oldest.value[1]); + } +} + +/** + * Record one executed bridged search. + * + * An entry with no call id is not recorded: the restore would have to emit a `function_call` + * without one, which is not a valid item and could not be paired with its output anyway. + */ +export function rememberBridgeSearchReplay( + scope: string | undefined, + cellItemId: string, + entry: BridgeSearchReplayEntry, +): void { + if (!scope || cellItemId.length === 0 || entry.callId.length === 0) return; + const key = keyFor(scope, cellItemId); + const existing = entries.get(key); + if (existing) drop(key, existing); + const bytes = 2 * ( + key.length + + entry.callId.length + + (entry.sourceItemId?.length ?? 0) + + entry.name.length + + entry.argumentsText.length + + entry.output.length + ); + // A single oversized result is refused outright rather than evicting the whole store for it. + if (bytes > MAX_TOTAL_BYTES) return; + entries.set(key, { entry: { ...entry }, bytes, at: now() }); + totalBytes += bytes; + sweep(now()); +} + +/** + * Look up one recorded search without consuming it. + * + * The same cell is replayed on every subsequent turn of the conversation, so a consuming read + * would restore the pair once and then silently stop. Expiry stays absolute: a conversation that + * outlives the TTL degrades to today's behaviour (the hosted cell replays unchanged) rather than + * pinning entries open for as long as anyone keeps talking. + */ +export function peekBridgeSearchReplay( + scope: string | undefined, + cellItemId: string, +): BridgeSearchReplayEntry | undefined { + if (!scope || cellItemId.length === 0) return undefined; + const key = keyFor(scope, cellItemId); + const stored = entries.get(key); + if (!stored) return undefined; + if (now() - stored.at >= TTL_MS) { + drop(key, stored); + return undefined; + } + return stored.entry; +} + +export function clearBridgeSearchReplayCacheForTests(clock?: (() => number) | null): void { + entries.clear(); + totalBytes = 0; + clockForTests = clock ?? null; +} diff --git a/src/server/responses/passthrough-delivery.ts b/src/server/responses/passthrough-delivery.ts index dbc3b15133..a01bf13ca4 100644 --- a/src/server/responses/passthrough-delivery.ts +++ b/src/server/responses/passthrough-delivery.ts @@ -45,6 +45,7 @@ import { createPassthroughWebSearchBridgeStream, createPassthroughWebSearchBridgeExecutor, } from "../../web-search/passthrough-bridge"; +import { bridgeSearchReplayScope } from "../../responses/bridge-search-replay-cache"; import { fetchWithHeaderTimeout, providerFetch } from "./fetch-helpers"; import { providerApiKeySelectionIsCurrent } from "../../providers/api-key-selection"; import { requiresVisionPreprocessing } from "../../vision"; @@ -402,6 +403,10 @@ export async function deliverPassthroughResponse( describeImages: requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName), sidecar: config.webSearchSidecar, }), + // Scope the executed-search memo to this exact upstream (#4587). The Responses adapter + // derives the same scope from the same base URL before the NEXT turn is dispatched, so + // a replayed hosted cell can be turned back into the destination's own call and result. + destinationScope: bridgeSearchReplayScope(route.provider.baseUrl), // Appending a search result can push the continuation past the ceiling the first leg // was admitted under, so the same limit is re-applied before every later send. checkOutboundBody: (continuationBody: string) => { diff --git a/src/web-search/passthrough-bridge.ts b/src/web-search/passthrough-bridge.ts index 6a1a4b2f9c..4de220fceb 100644 --- a/src/web-search/passthrough-bridge.ts +++ b/src/web-search/passthrough-bridge.ts @@ -75,6 +75,7 @@ import { } from "./sidecar-providers"; import { providerDestinationConfigError } from "../lib/destination-policy"; import { redactSecretString } from "../lib/redact"; +import { rememberBridgeSearchReplay } from "../responses/bridge-search-replay-cache"; /** Canonical Ollama Cloud origin. The only origin the "ollama" backend derives on its own. */ export const OLLAMA_CLOUD_ORIGIN = "https://ollama.com"; @@ -377,6 +378,11 @@ export interface PassthroughWebSearchBridgeStreamOptions { /** Sends one continuation leg and resolves with its response. */ send: (body: string) => Promise; execute: PassthroughWebSearchBridgeExecutor; + /** + * Destination identity for the executed-search memo (#4587). When absent nothing is recorded, + * and the next turn replays the hosted cell exactly as it does today. + */ + destinationScope?: string; /** * Re-applies the caller's outbound body ceiling to a continuation body. Returns a refusal * message when the extended body may not be sent, or undefined when it is admitted. @@ -1094,11 +1100,22 @@ async function* bridgeStreamBlocks( outcome = await options.execute(queries, options.signal); } yield* emit(state.searchEndFrames(call, queries, outcome)); - turns.push({ - call, - // The model needs a readable result either way; an executor error is reported as the - // tool result rather than as a turn failure, so it can still answer without the search. - output: outcome.error ? "Web search failed: " + outcome.error : outcome.text, + // The model needs a readable result either way; an executor error is reported as the + // tool result rather than as a turn failure, so it can still answer without the search. + const output = outcome.error ? "Web search failed: " + outcome.error : outcome.text; + turns.push({ call, output }); + // Record what a continuation leg WOULD put on the wire, whether or not this leg sends one + // (#4587). The caller keeps the hosted cell and replays it next turn; the pre-dispatch + // rewrite in the Responses adapter uses this to hand the destination back its own call and + // result instead of an item type it never produced. Recording the same text that + // appendBridgeSearchTurn would append is what keeps a replayed turn and a continued turn + // showing the destination one consistent conversation. + rememberBridgeSearchReplay(options.destinationScope, call.cellItemId, { + callId: call.callId, + sourceItemId: call.sourceItemId, + name: WEB_SEARCH_TOOL_NAME, + argumentsText: call.argumentsText, + output, }); } diff --git a/structure/runtime.md b/structure/runtime.md index 94c8ad9fc8..00131f381c 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -307,9 +307,20 @@ disarmed rather than falling through to another paid search. A leg that mixes an `web_search` call with another client-executed tool ends the turn on that leg: the intercepted searches run, their hosted cells complete, the held client calls are released for the caller to execute, and the leg's own terminal closes the turn with no continuation sent upstream. The -destination therefore never receives the executed search result — the caller replays the hosted -`web_search_call` cell, which carries the query and sources but no result text, so the -destination's own `function_call`/`function_call_output` pair is not reconstructed. A leg whose +destination therefore does not receive that search result during the turn. It gets it on the next +one: every search the bridge executes is recorded in `src/responses/bridge-search-replay-cache.ts` +under the hosted cell's proxy-minted id, scoped to the upstream destination and bounded by entry +count, total bytes, and a one-hour TTL. When the caller replays that cell, +`restoreBridgedWebSearchCalls` in `src/adapters/openai-responses/tool-output-recovery.ts` puts the +destination's own `function_call` and the executed `function_call_output` back in the cell's +position before the next turn's first leg is dispatched, recording exactly the text +`appendBridgeSearchTurn` would have sent on a continuation leg so a replayed turn and a continued +turn show the destination one consistent conversation. The rewrite runs only for a provider with +`webSearchBridge.enabled`, and a miss — unknown id, expired entry, a different destination, or a +`call_id` the body already carries — leaves the replayed item untouched. Re-running the search or +synthesizing result text is not a permitted recovery. +`tests/web-search/web-search-bridge-replay.test.ts` pins the restore and each of those refusals. +A leg whose upstream terminal is `response.failed` or `response.incomplete` runs no search at all and closes any cell it opened rather than leaving it in progress. Assistant text is not treated as a search instruction. diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 5ba90d98db..3f87179c7e 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1254,6 +1254,7 @@ "warmup.test.ts": "codex-integration", "web-search-anthropic.test.ts": "web-search", "web-search-backend-union.test.ts": "web-search", + "web-search-bridge-replay.test.ts": "web-search", "web-search-candidates.test.ts": "web-search", "web-search-parse.test.ts": "web-search", "web-search-passthrough-bridge.test.ts": "web-search", diff --git a/tests/web-search/web-search-bridge-replay.test.ts b/tests/web-search/web-search-bridge-replay.test.ts new file mode 100644 index 0000000000..3d5a6aab28 --- /dev/null +++ b/tests/web-search/web-search-bridge-replay.test.ts @@ -0,0 +1,295 @@ +/** + * #4587: the bridge shows the CALLER a hosted web_search_call cell for a search it ran + * proxy-side. The caller replays that cell on the next turn, so the destination — which never + * produced a web_search_call — saw an unknown item type carrying a query and no result, and + * usually just searched again. + * + * These pin the repair and, just as importantly, its refusals: a miss must leave the replayed + * item alone rather than re-running the search or inventing a result. + */ +import { afterEach, describe, expect, test } from "bun:test"; +import { + createPassthroughWebSearchBridgeStream, + type PassthroughWebSearchBridgePlan, +} from "../../src/web-search/passthrough-bridge"; +import { restoreBridgedWebSearchCalls } from "../../src/adapters/openai-responses/tool-output-recovery"; +import { + bridgeSearchReplayScope, + clearBridgeSearchReplayCacheForTests, + rememberBridgeSearchReplay, +} from "../../src/responses/bridge-search-replay-cache"; +import { createResponsesPassthroughAdapter as createResponsesPassthroughAdapterProduction } from "../../src/adapters/openai-responses"; +import { withTestTranslatorBudget } from "../helpers/translator-budget"; +import type { OcxProviderConfig } from "../../src/types"; + +const createResponsesPassthroughAdapter = (...args: Parameters) => + withTestTranslatorBudget(createResponsesPassthroughAdapterProduction(...args)); + +const GATEWAY_BASE_URL = "https://gateway.internal/v1"; +const OTHER_BASE_URL = "https://other-gateway.internal/v1"; + +function frame(type: string, payload: Record): string { + return "event: " + type + "\ndata: " + JSON.stringify({ type, ...payload }); +} + +function sseBody(...blocks: string[]): string { + return blocks.concat("data: [DONE]").join("\n\n") + "\n\n"; +} + +function streamFromText(text: string): ReadableStream { + const chunk = new TextEncoder().encode(text); + let sent = false; + return new ReadableStream({ + pull(controller) { + if (sent) { + controller.close(); + return; + } + sent = true; + controller.enqueue(chunk); + }, + }); +} + +function clientEvents(body: string): Record[] { + return body + .split(/\r?\n/) + .filter(line => line.startsWith("data:")) + .map(line => line.slice(5).trim()) + .filter(payload => payload.length > 0 && payload !== "[DONE]") + .map(payload => JSON.parse(payload) as Record); +} + +const plan: PassthroughWebSearchBridgePlan = { + backend: "ollama", + endpoint: "https://ollama.com/api/web_search", + maxSearches: 3, + timeoutMs: 60_000, +}; + +const searchCall = { + type: "function_call", + id: "fc_1", + call_id: "call_1", + name: "web_search", + arguments: "{\"query\":\"opencodex release\"}", +}; + +const clientCall = { + type: "function_call", + id: "fc_2", + call_id: "call_2", + name: "exec", + arguments: "{\"cmd\":\"ls\"}", +}; + +/** The reported shape: one bridged search and one client-executed call on the same leg. */ +const mixedLeg = sseBody( + frame("response.output_item.added", { output_index: 0, item: { ...searchCall, arguments: "" } }), + frame("response.output_item.done", { output_index: 0, item: searchCall }), + frame("response.output_item.added", { output_index: 1, item: { ...clientCall, arguments: "" } }), + frame("response.output_item.done", { output_index: 1, item: clientCall }), + frame("response.completed", { + response: { id: "resp_1", status: "completed", output: [searchCall, clientCall] }, + }), +); + +const initialBody = JSON.stringify({ + model: "glm-4.7", + stream: true, + input: [{ role: "user", content: [{ type: "input_text", text: "what is the latest release?" }] }], + tools: [{ type: "web_search" }], +}); + +/** + * Run one mixed leg through the real bridge stream and return the hosted cell id the client + * would store. Reading the id off the emitted stream, rather than asserting a literal, is what + * proves the memo is keyed on the same id the caller actually replays. + */ +async function runBridgedMixedLeg(baseUrl: string, result = "opencodex 2.50.0 shipped"): Promise { + const stream = createPassthroughWebSearchBridgeStream({ + plan, + firstLeg: streamFromText(mixedLeg), + requestBody: initialBody, + send: async () => { + throw new Error("a mixed leg must not send a continuation"); + }, + execute: async () => ({ text: result, sources: [{ url: "https://example.test/rel", title: "Releases" }] }), + destinationScope: bridgeSearchReplayScope(baseUrl), + }); + const body = await new Response(stream).text(); + const added = clientEvents(body).find(event => + event.type === "response.output_item.added" + && (event.item as { type?: string } | undefined)?.type === "web_search_call"); + const cellId = (added?.item as { id?: string } | undefined)?.id; + expect(typeof cellId).toBe("string"); + return cellId as string; +} + +/** The next turn as the caller sends it: the hosted cell, then the client's own tool result. */ +function nextTurnBody(cellId: string): Record { + return { + model: "glm-4.7", + input: [ + { role: "user", content: [{ type: "input_text", text: "what is the latest release?" }] }, + { + type: "web_search_call", + id: cellId, + status: "completed", + action: { type: "search", query: "opencodex release", queries: ["opencodex release"] }, + }, + { type: "function_call", id: "fc_2", call_id: "call_2", name: "exec", arguments: "{}" }, + { type: "function_call_output", call_id: "call_2", output: "ok" }, + ], + }; +} + +afterEach(() => { + clearBridgeSearchReplayCacheForTests(); +}); + +describe("bridged web_search replay to the destination", () => { + test("the next turn carries the destination's own call and the executed result", async () => { + const cellId = await runBridgedMixedLeg(GATEWAY_BASE_URL); + + const restored = restoreBridgedWebSearchCalls( + nextTurnBody(cellId), + bridgeSearchReplayScope(GATEWAY_BASE_URL), + ) as { input: Record[] }; + + // The item type the destination never produced is gone, replaced in place by the exchange + // it actually had: its own call, immediately followed by the result the proxy executed. + expect(restored.input.some(item => item.type === "web_search_call")).toBe(false); + const call = restored.input[1]; + const output = restored.input[2]; + expect(call).toEqual({ + type: "function_call", + id: "fc_1", + call_id: "call_1", + name: "web_search", + arguments: "{\"query\":\"opencodex release\"}", + }); + expect(output).toEqual({ + type: "function_call_output", + call_id: "call_1", + output: "opencodex 2.50.0 shipped", + }); + // The client's own call and result are untouched and still adjacent. + expect(restored.input[3]).toEqual({ type: "function_call", id: "fc_2", call_id: "call_2", name: "exec", arguments: "{}" }); + expect(restored.input[4]).toEqual({ type: "function_call_output", call_id: "call_2", output: "ok" }); + }); + + test("an executor failure replays as the same tool result the destination would have seen", async () => { + const stream = createPassthroughWebSearchBridgeStream({ + plan, + firstLeg: streamFromText(mixedLeg), + requestBody: initialBody, + send: async () => { + throw new Error("a mixed leg must not send a continuation"); + }, + execute: async () => ({ text: "", sources: [], error: "backend refused" }), + destinationScope: bridgeSearchReplayScope(GATEWAY_BASE_URL), + }); + const body = await new Response(stream).text(); + const added = clientEvents(body).find(event => + event.type === "response.output_item.added" + && (event.item as { type?: string } | undefined)?.type === "web_search_call"); + const cellId = (added?.item as { id?: string }).id as string; + + const restored = restoreBridgedWebSearchCalls( + nextTurnBody(cellId), + bridgeSearchReplayScope(GATEWAY_BASE_URL), + ) as { input: Record[] }; + expect(restored.input[2]).toEqual({ + type: "function_call_output", + call_id: "call_1", + output: "Web search failed: backend refused", + }); + }); + + test("a cell this proxy never executed is left exactly as the caller sent it", () => { + const body = nextTurnBody("ws_never-recorded"); + const restored = restoreBridgedWebSearchCalls(body, bridgeSearchReplayScope(GATEWAY_BASE_URL)); + // Same reference: a miss allocates nothing and invents nothing. + expect(restored).toBe(body); + }); + + test("a search recorded for one destination is not replayed into another", async () => { + const cellId = await runBridgedMixedLeg(GATEWAY_BASE_URL); + const body = nextTurnBody(cellId); + expect(restoreBridgedWebSearchCalls(body, bridgeSearchReplayScope(OTHER_BASE_URL))).toBe(body); + }); + + test("an expired entry behaves exactly like a miss", async () => { + let clockMs = 1_000; + clearBridgeSearchReplayCacheForTests(() => clockMs); + const cellId = await runBridgedMixedLeg(GATEWAY_BASE_URL); + const body = nextTurnBody(cellId); + // Still inside the TTL. + expect(restoreBridgedWebSearchCalls(body, bridgeSearchReplayScope(GATEWAY_BASE_URL))).not.toBe(body); + clockMs += 61 * 60 * 1000; + expect(restoreBridgedWebSearchCalls(body, bridgeSearchReplayScope(GATEWAY_BASE_URL))).toBe(body); + }); + + test("a call id the body already carries is never duplicated", () => { + const scope = bridgeSearchReplayScope(GATEWAY_BASE_URL); + rememberBridgeSearchReplay(scope, "ws_dup", { + callId: "call_2", + name: "web_search", + argumentsText: "{}", + output: "result", + }); + const body = nextTurnBody("ws_dup"); + expect(restoreBridgedWebSearchCalls(body, scope)).toBe(body); + }); + + test("an unbridged provider is never given a scope to restore from", () => { + const scope = bridgeSearchReplayScope(GATEWAY_BASE_URL); + rememberBridgeSearchReplay(scope, "ws_unbridged", { + callId: "call_1", + name: "web_search", + argumentsText: "{}", + output: "result", + }); + const body = nextTurnBody("ws_unbridged"); + expect(restoreBridgedWebSearchCalls(body, undefined)).toBe(body); + }); +}); + +describe("the Responses passthrough adapter", () => { + function providerFixture(bridged: boolean): OcxProviderConfig { + return { + adapter: "openai-responses", + baseUrl: GATEWAY_BASE_URL, + authMode: "key", + apiKey: "fixture-key", + ...(bridged ? { webSearchBridge: { enabled: true, backend: "ollama" } } : {}), + } as OcxProviderConfig; + } + + function outboundInput(provider: OcxProviderConfig, cellId: string): Record[] { + const request = createResponsesPassthroughAdapter(provider).buildRequest({ + modelId: "glm-4.7", + context: { messages: [] }, + stream: true, + options: {}, + _rawBody: nextTurnBody(cellId), + }, { headers: new Headers() }); + return (JSON.parse(request.body) as { input: Record[] }).input; + } + + test("restores the pair on the wire for a bridged provider", async () => { + const cellId = await runBridgedMixedLeg(GATEWAY_BASE_URL); + const input = outboundInput(providerFixture(true), cellId); + expect(input.some(item => item.type === "web_search_call")).toBe(false); + expect(input[1]).toMatchObject({ type: "function_call", call_id: "call_1", name: "web_search" }); + expect(input[2]).toMatchObject({ type: "function_call_output", call_id: "call_1", output: "opencodex 2.50.0 shipped" }); + }); + + test("leaves the hosted cell alone when the provider has not opted in", async () => { + const cellId = await runBridgedMixedLeg(GATEWAY_BASE_URL); + const input = outboundInput(providerFixture(false), cellId); + expect(input.some(item => item.type === "web_search_call" && item.id === cellId)).toBe(true); + expect(input.some(item => item.call_id === "call_1")).toBe(false); + }); +}); From 5447329b263e3559bb9eb74e78bb80f482feab44 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 18 Sep 2026 03:45:56 +0900 Subject: [PATCH 2/2] docs(devlog): record the lane's closure evidence for #4429 and #3719 Both issue bodies predate the commits that changed the answer, so each claim is re-judged against the current source with the delivering commit named. --- .../020_closure_evidence.md | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 devlog/_plan/260918_ld_search_bridge_and_thinking_replay/020_closure_evidence.md diff --git a/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/020_closure_evidence.md b/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/020_closure_evidence.md new file mode 100644 index 0000000000..4e9b5ddf85 --- /dev/null +++ b/devlog/_plan/260918_ld_search_bridge_and_thinking_replay/020_closure_evidence.md @@ -0,0 +1,90 @@ +# LD — closure evidence for #4429 and #3719 + +Both issue bodies predate the commits that changed the answer. Each claim below was re-judged +against `origin/dev` at `2f025814f3`, by reading the current source rather than the issue text. +The host owns every close decision; this unit only supplies the evidence. + +## #4429 — key-auth Responses gateway echoes hosted `web_search` as a client `function_call` + +The report asked for two things, and named the shape of the fix itself: ship a non-Ollama executor +for `webSearchBridge` by reusing the existing sidecar executors, then have the intercepted call run +proxy-side and continue the conversation upstream so the caller sees a hosted `web_search_call` +cell. + +| Ask | State at HEAD | Evidence | +|---|---|---| +| A non-Ollama key-auth gateway can arm the bridge | Delivered | `5b707d3a5c` | +| The intercepted call runs proxy-side and the conversation continues upstream | Delivered | `024e43ddf1`, `3557ada7de` | +| A leg mixing the search with a client-executed tool does not kill the turn | Delivered | `2e6a0316b9` (#4586) | +| The destination learns the executed result on the next turn | #4587, addressed by this lane's PR | — | + +`planPassthroughWebSearchBridge` no longer returns early for a non-`ollama` backend. For +`openai`, `anthropic`, `xai`, `gemini` and `exa` it plans on the presence of that backend's own +resolved credential and never derives an Ollama Cloud origin, so an internal gateway on an +arbitrary base URL arms exactly as the reporter asked. The credential boundary #3761 called for +survives: only the `ollama` backend spends the passthrough provider's own API key, and a backend +whose credential is missing stays disarmed instead of falling through to a different paid search. + +The hosted/client distinction the issue turns on is `isWebSearchCallItem` against +`isClientExecutedItem`. A namespaced `ns__web_search` is a tool identity the client declared and +executes itself, so it is never intercepted, and `web_search` is never added to the +undeclared-tool guard's allowed names — that would authorize a call nobody can execute rather than +removing it. The guard's authority over every other tool a destination emits is unchanged. + +**Recommendation.** Closable once #4587 lands. One thing in the report is not resolved and is not +a code defect: whether the reporter's gateway auto-executes Kimi's builtin `$web_search` server +side and merely echoes the call. That needs a live probe the reporter offered to run. If the host +wants it tracked, it is a question to the reporter on the existing thread, not a separate defect. + +## #3719 — Anthropic thinking replay through proxy-auth translation + +The body states that the inbound translator drops assistant `thinking` and `redacted_thinking` +blocks on replay. That is false at HEAD. + +| Checkbox | State at HEAD | Evidence | +|---|---|---| +| Implement Anthropic-to-Anthropic replay fidelity, preserving upstream signatures and opaque redacted blocks | Delivered | `4a59dbc2b7`, `58fcb0961d` | +| Verify a multi-turn thinking/tool-result exchange with both block types | Covered by regression tests | `tests/adapters/anthropic/anthropic-thinking-signature.test.ts` | +| Compare cache creation/read usage across controlled continuation turns | Not established | — | +| Document native passthrough eligibility separately from translated-route cache support | Delivered | `docs-site/src/content/docs/guides/claude-code.md` | + +`src/claude/inbound.ts` encodes the Anthropic signature as `{sig}` and each opaque +`redacted_thinking` payload as `{red:[...]}` inside a bounded `ocxr1` envelope carried in +`encrypted_content`. `src/adapters/anthropic.ts` replays the redacted blocks verbatim first, in +the original stream order, then the signed `thinking` block. + +The separation this lane was asked to preserve is real and enforced in code, not by convention. +`isLikelyRealAnthropicThinkingSignature` gates the outbound `thinking` block, so a value that does +not look like an upstream-issued signature is dropped rather than sent as one. Two specific +leaks are refused rather than generalized: the inbound translator rejects an `ocxr1` envelope +carrying `sig` that arrives in an Anthropic `signature` field, because proxy-minted reasoning +continuity must never be replayed as an Anthropic signature; and a native OpenAI-encrypted blob +has no `ocxr1` prefix, so the decoder returns null and it keeps its placeholder rather than being +laundered into a signature. Carrying signature data to a different provider stays out of scope. + +**Recommendation.** Only the third checkbox is unproven, and it is a measurement rather than an +implementation: a controlled cache creation/read comparison across continuation turns with a +stable model, credential scope, prompt prefix, tool set and retention setting. It needs live +Anthropic traffic, which this lane cannot produce under the no-local-execution rule, and neither +the per-turn cache-miss claim nor its attribution to dropped replay blocks was ever reproduced — +in #3646 or here. The host's options are to close #3719 on the implemented and documented scope +and treat the unreproduced cache claim as not established, or to keep it open solely as a +measurement task. The public guide already states the honest position: replay preserves non-hidden +signed blocks and opaque redacted blocks on the intended Anthropic adapter, and that this does not +establish live Anthropic acceptance or cache-hit improvements. No documentation change is needed +for a close. + +## Pull requests reviewed, not carried + +- **#3952** mixes several changes. The part that touches this lane is a single guarded rewrite in + the Responses passthrough: when `provider.modelSuffixBracketStrip` is set, the outbound `model` + has its bracketed suffix stripped, detached before the write so the caller's raw body is not + mutated. That much is sound and independent of the freeform-tool and Moonshot work in the same + branch. It should be judged per change, not as one verdict. +- **#4783** targets `main` and is a draft. Within this lane's scope it adds an `openai-apikey` + bridge backend and routes the bridge's reasoning setting through `resolveSidecarReasoning` + instead of reading `sidecar.reasoning` directly. The backend addition is the same generalization + #4429 asked for, so it overlaps that area; it needs to be retargeted to `dev` before any of it + can be judged on merit. +- **#4900** belongs to another lane's Cursor work. Not duplicated here. Nothing in this lane's + history-replay scope depends on it.