Repository navigation
Revert LLM_DISABLE_STREAMING=true in strix.yml once contextual-orchestrator fixes the stale invalid_stream_options gate #1446
Description
Incident
Since Strix moved to contextual-orchestrator's orchestrator/free pool today (#1434), every Strix scan org-wide has been failing closed with openai.BadRequestError: 400 invalid_stream_options — stream_options.include_usage=true is not supported with tools or response_format. Example: https://github.com/ContextualWisdomLab/.github/actions/runs/33307905354/job/99247611184
Root cause (confirmed by reading actual source, not guessing):
- strix-agent 1.5.3 hardcodes
ModelSettings(include_usage=True)for every turn (strix/core/inputs.py::make_model_settings()), with no supported env var, CLI flag, or config key to disable just that flag while keeping streaming. Verified against strix v1.5.3 and openai-agents v0.19.4 source directly. contextual_orchestrator/server.py(around line 6583-6589) unconditionally rejectsstream=true+stream_options.include_usage=truewhenevertoolsorresponse_formatare present — a check introduced in commite7618a3alongside an unrelated fix, justified only by a test docstring ("Structured passthrough cannot emit usage SSE") that is factually wrong:_chat_response_sse_chunksalready builds a genuineusage_source: "reported"SSE chunk from the provider's realusagefield, and the tool-loop path (proxy_completion,orchestrator.py) already forces the upstream call to non-streaming and already captures real per-call usage — the honest-usage plumbing this check thinks doesn't exist already exists and already runs on this exact path.
Mitigation shipped now
This PR/commit sets LLM_DISABLE_STREAMING: "true" in the Strix job env in .github/workflows/strix.yml. This is a genuine, documented strix env var (strix/config/models.py::_NonStreamingModel) that makes every turn a single non-streaming call, so stream_options is never sent and the gateway's check never fires. This restores the required Strix gate org-wide immediately, at the cost of Strix losing real-time SSE streaming (all turns become synchronous).
What must happen before this is reverted
- A PR on
ContextualWisdomLab/contextual-orchestratorthat removes or narrows the stale guard atserver.py(~line 6583-6589), per evidence that both the tool-loop and non-tool-loop branches already source real, honest usage from the provider response (never fabricated) — no orchestrator.py change is required, the plumbing already works. - That PR flips the two tests currently pinning the rejection (
tests/test_chat_tools_passthrough_controls_http_honesty.py::test_http_structured_stream_usage_fails_closed_before_execution,tests/test_stream_options_null_flags_noop_http_honesty.py::test_http_chat_structured_streams_include_usage) to assert a realusage_source: "reported"SSE chunk, adds a positive honesty-invariant test, and corrects the now-staleCHANGELOG.mdentry describing the old rejection as intentional. - Once that PR is merged and deployed, verify with a manual Strix
workflow_dispatchrun (or a direct request reproducing Strix's exact request shape) showing 200 +usage_source: "reported", then removeLLM_DISABLE_STREAMING: "true"fromstrix.ymlhere so Strix regains real streaming.
Do not let this mitigation quietly become permanent — it trades away working infrastructure for no architectural reason once the real fix lands.
Activity