openhands: a new backend, driven through its maintained agent-server (1.49.2)

[ZixiaoL]

OpenHands V1 as a backend, driven through openhands-agent-server 1.49.2 (MIT) — the REST/WebSocket
interface its vendor maintains, not the deprecated CLI. PyPI openhands is OpenHands/openhands-cli,
whose README opens with "This project is no longer actively maintained" and whose last release is
1.16.0 of 2026-05-08; the agent server released 1.49.2 on 2026-09-17.

The shape

One server process per turn, so the one-process-per-turn contract every other backend keeps is kept
here: the runner reads NDJSON off a driver's stdout and cancel is a process-group kill. Cold start
to a serving /alive is 3.3–4.1 s, measured, and a conversation created by one process is read back
intact by a different process over the same on-disk store — which is what makes the per-turn shape
work at all, and what this product needs, since sandboxes are recycled between turns.
runner/openhands_driver.py follows dsh_driver.py: same header, same {"m", "p"} line protocol.

Measured properties the design rests on, each pinned by a test:

• the agent is frozen at the conversation's first creation, so the conversation id is a uuid5
  over the tool policy and the declared MCP servers — a second create with a different tool list
  leaves the persisted agent as it was
• the credential is never persisted: base_state.json holds the LLM spec with api_key: None,
  so the key rides the environment
• the answer can arrive as a FinishAction rather than a trailing assistant message
• disabling a tool is enforced by omission from the spec, not by a refusal

Columns

column	result
vercel, 49 of the 50 catalog ids	239 of 245 scenarios
google, the eight gemini ids	40 of 40
custom-harness	passes: own skill, own script, own tool policy, and a declared MCP server

Vercel's six: qwen3.8-27b missed a recall after a switch and after a recycle (the model). The other
four were mistral-medium-3.5, which Vercel refuses with Assistant message must have either content or tool_calls about a message whose content is right there — isolated against the live endpoint,
the identical request answers 200 with content: "M1" and 400 with content: [{"type":"text",…}].
The relay stringifies assistant content for that route once the provider names it, the tool-role
twin one role over, and the re-run passed five of five.

The defect this PR spent the most on

The google column was 29 of 40 until the last commit. Every gemini follow-up failed after
145–212 s with no reason of any kind — eleven scenarios across five models, never a first turn:

AttributeError: 'PromptTokensDetailsWrapper' object has no attribute 'cache_creation_tokens'
  openhands/sdk/llm/utils/telemetry.py:259, _cache_buckets

Neither the provider nor the request shape. From litellm 1.95.0 the wrapper's __setattr__ mirrors
an assignment between cache_write_tokens and cache_creation_tokens, putting both names into
model_fields_set, and litellm then drops the unset attribute from __dict__ as a
construction-cost optimisation. The SDK's telemetry uses "cache_creation_tokens" in details.model_fields_set as its existence test. Each side is self-consistent; together they are
not, and three lines reproduce it with no agent, no provider and no network:

>>> PromptTokensDetailsWrapper(cached_tokens=123).model_fields_set
{'cache_creation_tokens', 'cached_tokens', 'cache_write_tokens'}
>>> hasattr(_, 'cache_creation_tokens')
False

1.49.2 is the newest SDK, so there is nothing to upgrade to; it asks only for litellm>=1.93.0, so
litellm joins the same single pip invocation pinned at 1.94.3, and the install asserts the
defect is absent rather than asserting a version — a future bump fails the image instead of
failing every follow-up on a provider that reports prompt caching. The guard compares both pins,
because a volume built before this holds the right agent-server and the wrong litellm.

Why Vercel never saw it: it reports cache_creation_input_tokens on every response, so litellm
sets cache_creation_tokens rather than leaving it None and the attribute genuinely exists.
Measured through litellm 1.101.0 on the real streaming path, vercel/mistral-medium-3.5 and
vercel/gpt-5.4 both answer hasattr=True. That also rules the defect out as a second cause of the
mistral failures above.

The defect is intermittent — Google populates prompt_tokens_details only sometimes, and a turn
without it passes on the broken pin too (captured from inside _cache_buckets:
Usage(prompt_tokens=7410, …, prompt_tokens_details=None), on a follow-up that passed). So the
single-model A/B is supporting evidence and the column is what carries the claim: 29 of 40 before,
40 of 40 after, same ids, same image otherwise.

Why the record said nothing, and what now happens instead

event_service._run_and_publish publishes an error event only for an exception that is not a
ConversationRunError, assuming the run already emitted its own — and an exception raised out of
the run's error handling is exactly the case where nobody did. The status flips to error and the
WebSocket carries no reason at all; the sentence existed only in the server's log, and the record
said the turn ended error eleven times.

The driver now falls back to that log, taking the exception lines rather than the file's last
lines: a turn that dies in seconds and is retried to exhaustion ends with SIGTERM … Shutting down,
and the first version of this fallback put exactly that into the record, which reads like a reason
and is worse than silence. It reports both ends of the chain, since the root cause names the defect
and the wrapper names the operation. COLUMNS is set so the sentence survives rich's wrapping in
one piece. Verified end to end against the broken pin: the record now reads
agent-server said: AttributeError: 'PromptTokensDetailsWrapper' object has no attribute 'cache_creation_tokens' … ConversationRunError: …. Five tests pin it, three of them for defects
this fallback had.

docs/harness-verification.md point 6 gains the case it did not cover: a harness that reports a
failure nowhere.

Also here

• tmux in the image: openhands-tools declares libtmux, which drives the real binary; upstream's
  own image installs it, and without it the server dies at start.
• TMUX_TMPDIR is set short. The server defaults it inside the working directory, and a workspace
  here is /data/workspaces/hsess<32 hex>, so the socket landed at 106 characters against the
  108-byte sun_path limit; tmux answered File name too long, the agent retried the tool it could
  not start, and a 10 s turn became 235 s and then 4,000 s. Four wrong guesses preceded it; what
  ended the investigation was sending the server's stdout to a file instead of DEVNULL.
• The relay route is guarded on the key like every other builder's.
• openhands is opt-in, not in the default HR_BACKENDS: its environment is ~666 MB, the aider
  precedent and well above the 300 MB line.

Verification

All 17 registration points. tsc clean. Python suite 976 passed against origin/main's 944,
with the same 23 environment failures on both, so no regression. docs/support-matrix.md carries no
openhands rows, matching kimi, the most recent merged harness; the record is in
docs/support-matrix-notes.md.

🤖 Generated with Claude Code

[vercel]

@ZixiaoL is attempting to deploy a commit to the Future HR Team on Vercel.

A member of the Team first needs to authorize it.