fix(combos): reserve output headroom before fallback - #4664
RHODIZSECURITY wants to merge 3 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
✅ Deterministic PR hygiene checks passed. |
⏳ DRAFT
What to do
Review readiness checklist
2/4 boxes ticked. This PR stays in draft until every box above is ticked. Hygiene✅ Deterministic PR hygiene checks passed. |
|
Post-open live validation on the affected HomeLab path (no prompt/session contents included): after restarting the hotfixed OpenCodex 2.55.0, one real |
리뷰 · 우선순위 67 / 80설명 이 PR은 콤보(fallback 사다리)가 너무 작은 컨텍스트 창을 가진 후보에게 이미 바이트를 보낸 뒤 고친 핵심은 두 갈래입니다. 첫째, 테스트도 의도에 맞춰 있습니다. 유닛은 100k+32k 예약으로 128k 타깃 스킵, 90k는 통과, 창 미지는 통과, 출력 한도 없으면 느슨한 계약 유지, Spark 128k/32k resolve를 고정합니다. E2E는 작은 타깃 hit=0, 큰 타깃 hit=1, 논리 콤보 200을 증명합니다. 라이브 후속 댓글(Opus/Sol 429 → Spark 로컬 413 9ms → 큰 창 폴백 200)도 같은 이야기를 현장 숫자로 뒷받침합니다. 지금
메인테이너의 판단이 필요한 지점
너의 추천 이 댓글은 grok-bot이 작성했습니다 |
Co-authored-by: RHODIZ IT <info.rhodiz@gmail.com>
0242862 to
d823a00
Compare
Keep every assertion registered under the existing suite isolation and cleanup hooks. Static source inspection only; tests not executed on this connected host. Co-authored-by: RHODIZ IT <info.rhodiz@gmail.com>
lidge-jun
added
the
landed-via-maintainer
…un#4664) [skip ci] A combo could route a large turn onto a fallback whose total context window cannot hold the input plus the output allowance the caller asked for. That target answers 200, emits a few hundred tokens and stops on finish_reason: length, which the Anthropic surface renders as "response exceeded the output token maximum" naming a limit the model never approached. Raising CLAUDE_CODE_MAX_OUTPUT_TOKENS only changes the number in that message. By the time it happens, output has committed and no later target may be tried. Admit a combo child against both budgets before dispatch. When the caller declared max_output_tokens, require estimated input <= input ceiling AND estimated input + min(declared output, target output ceiling) <= context window, and refuse locally with 413 input_admission_refused before any upstream bytes are sent. Combo policy already treats that local code as a safe hop, so the ladder selects a larger-context target without replaying committed output. The two budgets are checked separately on purpose. resolveInputCeiling already answers "how much input may this target take", and modelMaxInputTokens can tighten it below the window; charging the output reserve against that tightened number would count the reserve twice and skip a target that fits. The window is what input and output actually share, so the reserve belongs there. Reserving min(declared, target ceiling) rather than a fixed slice is what makes this catch the reported case: the common industry reservation of min(max_output, 20k) leaves 100k + 20k inside a 128k window, so the turn is admitted and fails upstream anyway. Canonical native slugs that the narrower pinned table does not carry now resolve their window from the generated in-tree bundle. That table gap is why the gate was completely inert on the route where this was observed. The bundle is compiled in, not a catalog read, so this adds no I/O, and explicit provider and operator caps may only narrow the result. It deliberately covers slugs retired from the picker, because a retired slug is still dispatchable when an operator names it explicitly in a combo target, which is exactly that configuration. Scope stays narrow. Direct and single-target requests keep the deliberately loose 2.5x pathological-input gate, because they have nowhere to hop. Compaction turns stay exempt. Unknown context and a caller that declared no output allowance both remain fail-open, so no limits are invented for custom providers. Closes lidge-jun#4664 Co-authored-by: RHODIZ IT <info.rhodiz@gmail.com>
2 participants
Root cause
A combo can route a large Claude Code turn onto a fallback whose total context window is too small to satisfy the request plus the caller's requested output allowance. If that target starts producing output and then returns
finish_reason: length, OpenCodex maps it to Anthropicstop_reason: max_tokens. Claude Code then renders its genericClaude's response exceeded the 64000 output token maximumerror even though the model emitted only hundreds of output tokens.This was reproduced on a live RHODIZ SMC session with
combo/rhodiz-coding-primary. The failing Spark attempts repeatedly ended at exactly 127,559 total tokens, e.g. ~126,981 input + 578 output and ~127,102 input + 457 output. That is a ~128k total-context boundary, not a 64k generated-output event. RaisingCLAUDE_CODE_MAX_OUTPUT_TOKENSwould therefore only change the number in the client error and would not fix the cause.There was a second hole: canonical
gpt-5.3-codex-sparkhas 128k context metadata in the generated OpenAI/OpenAI-Codex bundle, but the narrower native override map did not contain Spark, soresolveInputCeiling()returned unknown on the canonical route and the existing pre-dispatch admission gate was inert for this exact model.Fix
max_output_tokens, require:estimated input + min(requested output, target output ceiling) <= target context window.413 input_admission_refusedbefore any upstream bytes are sent. Existing combo policy already treats that local code as a safe hop, so the ladder can select a larger-context target without replaying committed output.For canonical Spark, the gate now resolves 128,000 context / 32,000 max output. A ~100k input with a 64k requested output allowance is skipped before dispatch because 100k + 32k cannot fit in 128k.
Safety boundary
This does not replay after output. It prevents dispatch to a known-incompatible combo target before output can commit. Unknown capability remains fail-open, so the change does not invent limits for custom providers. Single-target semantics are unchanged.
Verification
Exact head
024286241b17b65a4b25064fd988d5223ca55a2e, based ondev@aa91958e3b050084e1edc07dcd66b05ef6eac604when opened:bun x tsc --noEmit— PASS;git diff --check— PASS.New E2E proves the first 128k/32k target receives zero upstream hits, the larger-context second target receives exactly one hit, and the logical combo request returns 200.
The same code is locally hotfixed on the affected HomeLab OpenCodex 2.55.0; installed-source probing confirms the observed Spark-shaped request is preflight-refused with
estimatedTokens=100000,ceiling=128000,requiredOutputHeadroom=32000.No credentials, private prompt contents, or session identifiers are included here.
Review readiness checklist
devat opening time.