Skip to content

LLM clients have no timeouts + blocking call in async handler — prompt can hang forever #1

Description

@grave0x

Problem

All LLM HTTP clients are created with no timeout: reqwest::blocking::Client::new() in crates/llm/src/lib.rs:182 (Anthropic), :227 (OpenAI), :251 (OpenAI-compatible). A blackholed connection (established but never delivering data — e.g. the IPv6-vs-IPv4 path upstream root-caused in anomalyco/opencode#39859) hangs forever.

Worse: provider.complete() is a blocking call invoked directly inside an async tokio handler at crates/server/src/handler/session.rs:270. A hung provider stalls the entire server handler (no spawn_blocking, no timeout), not just one request.

This mirrors upstream anomalyco/opencode#40069 ("apply default header timeout to all providers", closes #39859) — upstream applied a 300s header timeout at the fetch layer to every provider because only OpenAI had one.

Impact

  • POST /api/session/:id/prompt can hang indefinitely when a provider connection blackholes.
  • No error is ever surfaced; operator must kill the process.

Spec / Acceptance criteria

  1. All provider clients (AnthropicProvider, OpenAIDirectProvider, OpenAICompatibleProvider) get explicit timeouts:
    • connect timeout ~30s, header timeout ~300s (config-overridable, false disables — keep ??-style semantics for an optional timeout_ms).
  2. Run complete() off the async executor: tokio::task::spawn_blocking with a timeout wrapper in the session handler (crates/server/src/handler/session.rs:262-286).
  3. On timeout/error: return a structured error response (status: "error" + message) instead of hanging or silently falling back to admitted.
  4. Test: mock a server that accepts TCP but never responds → prompt must fail within bounded time.

Reference: upstream anomalyco/opencode#40069 (closes #39859); our code: crates/llm/src/lib.rs, crates/server/src/handler/session.rs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions