Skip to content

feat(provider): add service_tier processing-tier control for openai endpoints - #2530

Merged
topcheer merged 1 commit into
mainfrom
sa81-service-tier
Sep 18, 2026
Merged

topcheer merged 1 commit into
mainfrom
sa81-service-tier

Conversation

@topcheer

Copy link
Copy Markdown
Owner

Summary

Adds service_tier (sa-81) support on openai and openai-responses protocols — the OpenAI processing-tier parameter that trades cost against latency (flex = cheaper/slower, priority/fast = Fast mode, the 2026-07 rename of Priority processing). Follows the exact plumbing pattern of the existing text_verbosity / reasoning_effort controls.

Changes

  • config: new service_tier endpoint field (service_tier: yaml key), trimmed into ResolvedEndpoint alongside the other hints
  • provider: new ServiceTierProvider interface; registry wires it for openai, openai-responses, and the /responses URL-sniff path
  • Allowlisted setters: unknown values are ignored (existing setting preserved) so a typo cannot silently flip an expensive production tier; valid values auto | default | flex | priority | fast | scale
  • openai.go: tier rides Chat Completions (non-stream + stream) requests; transparent one-shot degrade-retry when a relay rejects the parameter (same pattern as retryWithoutReasoningEffort)
  • openai_responses.go: tier rides Responses requests with graceful degrade identical to text.verbosity
  • fallback.go: tier forwarded across fallback chains via forEach
  • CloneWithModel carries the tier across model swaps
  • docs: new "Service Tier" section in docs/guide/providers.md

Anthropic/Gemini/Copilot endpoints are unaffected (no interface implemented → field ignored, documented).

Tests

10 new tests, all passing:

  • config: yaml resolution + trim + empty default
  • openai: setter validation, applyServiceTier, clone-carries-tier, error classification
  • openai-responses: setter validation, wire injection (service_tier in request body), omission when unset, degrade-retry against a gateway returning 400 Unknown parameter: 'service_tier'

Verification: gofmt clean, go vet -tags goolm clean, go build -tags goolm ./... OK, full internal/provider + internal/config regression suites green, pre-commit/pre-push hooks passed.

Co-Authored-By: ggcode noreply@ggcode.dev

@topcheer topcheer left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审通过 ✅(四重点核销)

显式 6 值 case 表+未知忽略(typo 不翻转昂贵档,三 setter 家族形态一致)。#2528 无耦合——tier 重试 req 级内层/callPolicy 外层正交;与 effort 重试串联(双 drop 最多 3 次请求上界合理)。 Fallback 链式广播同 verbosity 形态。 CloneWithModel 值继承(端点属性语义)。retryWithoutServiceTier Param 精确→消息兜底两态。

CI 绿。可合并。

@topcheer

Copy link
Copy Markdown
Owner Author

复审已过(见 review:①显式 6 值 case 防翻转+未知忽略(typo 无声面=effort/verbosity 家族既有取舍一致)②与 #2528 正交——tier 重试 req 级内层/callPolicy 外层,rebase 时接口注册顺序无耦合(不同 setter 名)③fast 拒收走 retryWithoutServiceTier 统一降级(Param/消息两态判别覆盖老网关)④fallback 广播+Clone 值继承 ⑤scale 透传由 OpenAI 域判定(未知 tier 服务端拒→降级路径兜底))。与 main 冲突——#2529 等先合并动了 openai.go 相邻区。分支 sa81-service-tier 有归属者本地 worktree 在途——请归属者 rebase main 解冲突,解后 CI 绿即合并。不代解。

…ndpoints

sa-81: support the OpenAI service_tier parameter (auto/default/flex/
priority/fast/scale) on openai and openai-responses protocols.

- config: new `service_tier` endpoint field, trimmed into ResolvedEndpoint
- provider: ServiceTierProvider interface; wired in registry for
  openai/openai-responses (allowlisted setters reject unknown values so a
  typo cannot silently flip an expensive production tier)
- openai.go: tier rides Chat Completions + stream requests; transparent
  one-shot degrade-retry when a relay rejects the parameter
- openai_responses.go: tier rides Responses requests with the same
  graceful degrade as text.verbosity
- fallback: tier forwarded across fallback chains
- docs: new Service Tier section in docs/guide/providers.md
- tests: config resolution, wire injection, omission, degrade retry,
  clone-carries-tier, error classification (10 new tests)

Co-Authored-By: ggcode <noreply@ggcode.dev>
@topcheer
topcheer merged commit a6831b4 into main Sep 18, 2026
1 check passed
@topcheer
topcheer deleted the sa81-service-tier branch September 18, 2026 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant