Skip to content

Empty/truncated LLM responses persisted as successful completions #3

Description

@grave0x

Problem

LLM responses are deserialized with unwrap_or("") and never validated:

  • Anthropic: crates/llm/src/lib.rs:210body["content"][0]["text"].as_str().unwrap_or("")
  • OpenAI-compatible: crates/llm/src/lib.rs:298body["choices"][0]["message"]["content"].as_str().unwrap_or("")

A truncated/empty/aborted response (connection died mid-turn, upstream stream ended without a finish frame, API returned an unexpected shape) is persisted as a successful assistant message and the prompt returns status: "completed" (crates/server/src/handler/session.rs:271-279).

This mirrors upstream anomalyco/opencode#40061 (closes #39968): when an SSE connection dies without a finish frame, the AI SDK synthesizes finishReason "other"/undefined which opencode mapped to benign "unknown", so truncated output was persisted as success and opencode run exited 0. Upstream now maps "other"/undefined → "error" and fails the turn while keeping partial output.

Impact

  • Silent data loss: user believes the model answered; content is empty/partial with no signal.
  • Breaks automation/exit-code semantics ("run completed" when it didn't).

Spec / Acceptance criteria

  1. Provider complete() validates the response shape before returning Ok:
    • Anthropic: require content array with a text block; empty content → Err("empty completion").
    • OpenAI-compatible: require choices[0].message.content non-empty; also honor choices[0].finish_reason — map length/content_filter/missing to an error (or explicit flag).
  2. Session handler distinguishes "completed" vs "error": on Err, respond status: "error" with the message (keep any partial content in the session as-is, but do not mark the turn completed).
  3. Test: stub a provider returning {"choices":[{"message":{"content":""},"finish_reason":"stop"}]} → prompt fails with a clear error, not "completed".

Reference: upstream anomalyco/opencode#40061 (closes #39968); our code: crates/llm/src/lib.rs:209-215, 298-305, crates/server/src/handler/session.rs:269-291.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions