Skip to content

anthropic: cache-aware top-level output_config.effort carrier - #2473

Merged
topcheer merged 1 commit into
mainfrom
anthropic-effort-carrier
Sep 17, 2026
Merged

topcheer merged 1 commit into
mainfrom
anthropic-effort-carrier

Conversation

@topcheer

Copy link
Copy Markdown
Owner

Concept

Anthropic 2026 GA effort parameter lives at top-level output_config.effort (not inside thinking blocks). Official guidance: top-level effort changes do preserve the prompt cache prefix and combining with extended thinking is the recommended pattern (effort steers overall token envelope, budget_tokens caps explicit thinking). SDK v1.68 exposes MessageNewParams.OutputConfig (GA); per-message markers are not expressible yet.

Gap

ggcode maps effort only to thinking.budget_tokens (buildParams L988-991). Mid-conversation effort switches rewrite request-body params — and the repo's own cache_efficiency_monitor lists "Reasoning effort changes" as a cache-storm cause. The cache-safe carrier was missing.

Implementation (internal/provider/anthropic.go, +346)

  1. Hysteresis state machine beginEffortTracking: top-level carrier attaches only after effort level is stable across two consecutive requests
    • adaptive effort apply→restore (constant level per request) correctly establishes the carrier; oscillating levels never attach
    • user /effort switch: suppressed once, then rebuilt (one deliberate cache rewrite, not a churn storm)
  2. buildParams: emits OutputConfig.Effort only when effortCarrier && conversationEffort == lastCallEffort; comments record the SDK-marker upgrade path
  3. Rejection circuit breaker isEffortError: mirrors isThinkingError (4xx/404/422 status gate + anchored phrases); chat and streaming paths retry once on endpoint rejection with CAS carrier teardown, degrading to budget_tokens semantics for the session
  4. SetReasoningEffort extended with xhigh/max (carrier-only); CloneWithModel inherits the breaker state, resets the stability window

Tests (zz_effort_carrier_test.go, 213 lines)

Hysteresis matrix, OutputConfig emission gating, isEffortError true/false cases, clone semantics, xhigh/max normalization.

Verification

  • go build -tags goolm (worktree + main side) ✅
  • -race targeted tests ✅
  • provider full package (1.9s) + agent full package (24.6s) regression ✅
  • pre-commit hooks (gofmt/vet/build) ✅

Co-Authored-By: ggcode noreply@ggcode.dev

Adopt Anthropic's GA effort parameter as the top-level reasoning effort
carrier, composed with per-turn thinking budget_tokens (documented best
practice for Opus 4.5+/5). Cache-aware hysteresis: a top-level effort
change does not preserve cached prefixes, so the carrier attaches only
after a level stabilizes across two consecutive requests:

- Per-turn adaptive-effort oscillation never touches the carrier
  (apply/restore flips per request; only stable resting levels qualify)
- User switches (/effort, config) re-establish after one suppressed call
  — one deliberate rewrite instead of churn
- Endpoint rejection latch (mirrors isThinkingError: 4xx status gate +
  anchored phrases): Chat and stream retries once without output_config,
  latch keeps effort semantics on budget_tokens for the session
- CloneWithModel inherits the capability latch, resets the stability
  window; SetReasoningEffort accepts xhigh/max (carrier-only levels)

SDK note: v1.68 typed params cannot express the per-message
output_config marker for cache-preserving mid-conversation switching;
re-establishments rewrite the prefix once (documented in buildParams).

Tests: hysteresis matrix (stabilization, oscillation/alternation,
switch re-establishment, rejection latch), OutputConfig emission
gating, isEffortError true/false cases, clone latch/window semantics.

Co-Authored-By: ggcode <noreply@ggcode.dev>

Co-Authored-By: ggcode <noreply@ggcode.dev>

@topcheer topcheer left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审通过 ✅(两重点核销;首轮 CI 失败=mcp SSE 时序 flake,rerun 绿——本地 3 次复跑过+包域零交集)

① KV-cache 前缀影响——hysteresis 设计正确:body 引用的官方语义"top-level effort preserve prefix"(注释同样写明)→ 解法是双连续稳定才挂载体:adaptive-effort 的每轮 set+restore 振荡永远不稳定→永不触载体(budget_tokens 逐轮调制 cache-neutral);用户显式切换第二次调用重建载体=一次有意的 cache rewrite 而非风暴。effort off 重置窗口(振荡防泄漏)。与 #2445 前缀稳定性约定一致:稳定优先,变更刻意

② 旧网关拒载回退:isEffortError 锚定短语+4xx 参数类状态码限定(镜像 isThinkingError 形态);CAS latch 单次降级重试(Chat+ChatStream 双路径,Stream 仅未发射时);latch 持久+clone 继承(endpoint 级能力记忆)但会话窗口重置(clone 各自 cache prefix 重新学习)——边界划分准确。

SDK 边界诚实:per-message marker 不可表达(v1.68)→重建走 top-level,注释留 flip 点。

5 测试。CI rerun 绿。可合并。

@topcheer
topcheer merged commit b975615 into main Sep 17, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant