anthropic: cache-aware top-level output_config.effort carrier - #2473
Merged
Merged
Conversation
Adopt Anthropic's GA effort parameter as the top-level reasoning effort carrier, composed with per-turn thinking budget_tokens (documented best practice for Opus 4.5+/5). Cache-aware hysteresis: a top-level effort change does not preserve cached prefixes, so the carrier attaches only after a level stabilizes across two consecutive requests: - Per-turn adaptive-effort oscillation never touches the carrier (apply/restore flips per request; only stable resting levels qualify) - User switches (/effort, config) re-establish after one suppressed call — one deliberate rewrite instead of churn - Endpoint rejection latch (mirrors isThinkingError: 4xx status gate + anchored phrases): Chat and stream retries once without output_config, latch keeps effort semantics on budget_tokens for the session - CloneWithModel inherits the capability latch, resets the stability window; SetReasoningEffort accepts xhigh/max (carrier-only levels) SDK note: v1.68 typed params cannot express the per-message output_config marker for cache-preserving mid-conversation switching; re-establishments rewrite the prefix once (documented in buildParams). Tests: hysteresis matrix (stabilization, oscillation/alternation, switch re-establishment, rejection latch), OutputConfig emission gating, isEffortError true/false cases, clone latch/window semantics. Co-Authored-By: ggcode <noreply@ggcode.dev> Co-Authored-By: ggcode <noreply@ggcode.dev>
topcheer
commented
Sep 17, 2026
topcheer
left a comment
Owner
Author
There was a problem hiding this comment.
复审通过 ✅(两重点核销;首轮 CI 失败=mcp SSE 时序 flake,rerun 绿——本地 3 次复跑过+包域零交集)
① KV-cache 前缀影响——hysteresis 设计正确:body 引用的官方语义"top-level effort 不 preserve prefix"(注释同样写明)→ 解法是双连续稳定才挂载体:adaptive-effort 的每轮 set+restore 振荡永远不稳定→永不触载体(budget_tokens 逐轮调制 cache-neutral);用户显式切换第二次调用重建载体=一次有意的 cache rewrite 而非风暴。effort off 重置窗口(振荡防泄漏)。与 #2445 前缀稳定性约定一致:稳定优先,变更刻意。
② 旧网关拒载回退:isEffortError 锚定短语+4xx 参数类状态码限定(镜像 isThinkingError 形态);CAS latch 单次降级重试(Chat+ChatStream 双路径,Stream 仅未发射时);latch 持久+clone 继承(endpoint 级能力记忆)但会话窗口重置(clone 各自 cache prefix 重新学习)——边界划分准确。
SDK 边界诚实:per-message marker 不可表达(v1.68)→重建走 top-level,注释留 flip 点。
5 测试。CI rerun 绿。可合并。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Concept
Anthropic 2026 GA effort parameter lives at top-level
output_config.effort(not inside thinking blocks). Official guidance: top-level effort changes do preserve the prompt cache prefix and combining with extended thinking is the recommended pattern (effort steers overall token envelope, budget_tokens caps explicit thinking). SDK v1.68 exposesMessageNewParams.OutputConfig(GA); per-message markers are not expressible yet.Gap
ggcode maps effort only to
thinking.budget_tokens(buildParamsL988-991). Mid-conversation effort switches rewrite request-body params — and the repo's owncache_efficiency_monitorlists "Reasoning effort changes" as a cache-storm cause. The cache-safe carrier was missing.Implementation (
internal/provider/anthropic.go, +346)beginEffortTracking: top-level carrier attaches only after effort level is stable across two consecutive requests/effortswitch: suppressed once, then rebuilt (one deliberate cache rewrite, not a churn storm)OutputConfig.Effortonly wheneffortCarrier && conversationEffort == lastCallEffort; comments record the SDK-marker upgrade pathisEffortError: mirrorsisThinkingError(4xx/404/422 status gate + anchored phrases); chat and streaming paths retry once on endpoint rejection with CAS carrier teardown, degrading to budget_tokens semantics for the sessionSetReasoningEffortextended withxhigh/max(carrier-only);CloneWithModelinherits the breaker state, resets the stability windowTests (zz_effort_carrier_test.go, 213 lines)
Hysteresis matrix, OutputConfig emission gating,
isEffortErrortrue/false cases, clone semantics, xhigh/max normalization.Verification
go build -tags goolm(worktree + main side) ✅-racetargeted tests ✅Co-Authored-By: ggcode noreply@ggcode.dev