feat(provider): per-model/per-endpoint LLM call policy (request_timeout, max_retries) - #2528
Merged
Merged
Conversation
topcheer
commented
Sep 18, 2026
topcheer
left a comment
Owner
Author
There was a problem hiding this comment.
复审通过 ✅(五重点核销)
① 23 处替换零值等价:attempts()=maxRetries>0?override:providerRetryAttempts——零配置 identity 回退(20 不变);diff 抽查 anthropic 7 处(含 stream 循环+attempt<attempts()-1 边界+Retry N/M 显示)形态一致,无 off-by-one(边界含-1 与常量版逐位同构)。
② 超时取消传播:ChatStream 的 withTimeout 在 streamRead goroutine 内部 wrap(覆盖全流生命周期,ChatStream 立即返回不误杀)+defer cancel 同 goroutine;deadline 到期=ctx.Err 传播走既有 emitted 语义(已收 tokens 已发 ch 不丢——部分结果保留);连接清理走既有 stream.Close defer 链。
③ CloneWithModel 继承:policy 值拷贝(callPolicy 两 int 字段)——子代理继承同策略是合理默认(策略是端点/模型属性非 agent 属性);需更短 deadline 的场景经 SetCallPolicy 显式覆盖(setter 存在)。
④ #2530 时序裁定:先 #2528 后 #2530(本 PR 先合并;#2530 若冲突 rebase——openai_responses degrade-retry 接线点,预期平凡)。
⑤ 未接入面注释诚实(openai-responses/copilot 后续接)。
CI 9/9 绿。可合并(按④先序)。
Owner
Author
topcheer
force-pushed
the
sa78-model-call-policy
branch
from
September 18, 2026 05:48
c5cc907 to
8894cd6
Compare
…etries) Add configurable per-endpoint and per-model call tuning resolved from vendor config: RequestTimeout (deadline for a single LLM call) and MaxRetries (retry budget override). Model-level entries take priority over endpoint-level defaults; zero values preserve existing behavior (no deadline, providerRetryAttempts), keeping the feature fully opt-in. Wire the resolved policy into providers via a callPolicySetter interface so NewProvider applies it uniformly across protocols. Co-Authored-By: ggcode <noreply@ggcode.dev>
- Add policy field + setCallPolicy to the three providers; policy is copied through CloneWithModel so named-subagent model overrides keep the endpoint's timeout/retry budget. - Replace the hardcoded providerRetryAttempts with p.policy.attempts() (23 call sites): configured max_retries now governs Chat and stream retry loops. - Apply policy.withTimeout in Chat and inside each streamRead goroutine so the deadline covers the full stream lifetime; no-op when unset. - NewProvider injects the resolved policy uniformly via callPolicySetter for anthropic/openai/gemini; other protocols keep existing defaults. Co-Authored-By: ggcode <noreply@ggcode.dev>
- config: TestResolveEndpointSelection_CallPolicy covers per-model override > endpoint-level fallback > zero-value default. - provider: TestCallPolicyAttempts / TestCallPolicyWithTimeout cover the retry-budget fallback and deadline semantics; TestNewProviderAppliesCallPolicy verifies registry injection; TestCloneWithModelKeepsPolicy verifies model clones inherit the endpoint policy. Co-Authored-By: ggcode <noreply@ggcode.dev>
topcheer
force-pushed
the
sa78-model-call-policy
branch
from
September 18, 2026 05:52
8894cd6 to
19ca62e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per-Model / Per-Endpoint LLM Call Policy(request_timeout / max_retries)
研究依据(在线)
Gap
ggcode 完全缺失 per-model/per-endpoint 超时与重试策略——全 provider 共享硬编码 providerRetryAttempts(20 次),无请求级 deadline。长推理模型(GPT-5.x Pro 等)与快速模型的超时需求无法分别配置。
实施(3 commits:938fdca9 → 76aa804 → c5cc907)
配置层(config.go / config_vendor.go / model_policy.go 新增)
接线层(76aa8041——初版反馈循环发现的关键 gap:策略定义了但无 provider 消费)
测试(c5cc907c)
遗留
openai-responses 与 copilot 协议未接入(registry 无 setter 的协议保持默认行为,代码注释标明)——可按同一模式低成本扩展。
Co-Authored-By: ggcode noreply@ggcode.dev