Skip to content

feat(providers): multiple API keys per provider with primary-first failover - #392

Open
YUZHEthefool wants to merge 1 commit into
Stack-Cairn:mainfrom
YUZHEthefool:feat/provider-multi-api-key-failover
Open

feat(providers): multiple API keys per provider with primary-first failover#392
YUZHEthefool wants to merge 1 commit into
Stack-Cairn:mainfrom
YUZHEthefool:feat/provider-multi-api-key-failover

Conversation

@YUZHEthefool

@YUZHEthefool YUZHEthefool commented Aug 7, 2026

Copy link
Copy Markdown

Linked issue

Closes #365

Summary

Following Cherry Studio, a single provider can now configure multiple API keys. Requests use the primary key (the first one) first; on a retryable failure before streaming starts (quota / auth / transient network errors), the request automatically falls over to the next key, cycling back to the primary once exhausted and continuing within the existing retry budget. Once content is committed (text_delta / thinking_delta / toolcall_start) the key is no longer switched, so partial output is never retried. Single-key paths (usage query, Go model fetch) are unchanged (zero regression), and prompt-cache hit rate is unaffected since the primary key is preferred and only swapped on failure.

Change scope

  • Data model agent-gui/src/lib/settings/index.ts, normalize.ts: CustomProvider gains apiKeys?: string[] + apiKeyCount?: number (redacted snapshot); apiKey is always derived as apiKeys[0]. normalizeApiKeys trims/dedupes preserving order; legacy single-key snapshots migrate to a single-element array.
  • Failover agent-gui/src/lib/providers/runtime/:
    • streamRetry.ts: StreamRetryConfig gains apiKeyFailover; rotate(attemptIndex) is invoked before each retry.
    • streamByApi.ts: each API branch's factory re-reads apiKey/auth headers from the attemptAuth holder on every factory() call; resolveAttemptHeaders replaces only auth headers (authorization/x-api-key/x-goog-api-key), preserving proxy-routing/session/custom headers.
    • requestOptions.ts: createProviderApiKeyFailover() builds the mutable holder + rotate callback (rebuilds auth headers per key).
    • providerRuntimeConfig.ts / types.ts: switched from cross-request round-robin to primary-first; ProviderRuntimeConfig carries apiKeys.
    • textOnlyRuntime.ts / chat/runner/agentRunner.ts: call sites wire in attemptAuth + apiKeyFailover.
  • Gateway sync agent-gui/src/lib/settings/sync.ts: providerApiKeyUpdates upgrades from Record<string,string> to Record<string,string[]>; redaction strips apiKeys and emits apiKeyCount; mergeSyncedCustomProviders supports sidecar full-replace / apiKeyConfigured===false explicit clear / otherwise keep-local.
  • Desktop UI agent-gui/src/pages/settings/ProvidersSection.tsx: API Key becomes a multi-row editor (add/remove, unified show/hide) with a failover hint; WebUI (remote) keeps a single redacted input plus an "N keys configured, re-entering replaces all" hint (replace semantics, no regression).
  • Cherry Studio import agent-gui/src/pages/settings/CherryStudioImportModal.tsx, src-tauri/src/commands/config/settings/cherry_import.rs: imports all enabled keys (v1 comma-separated / v2 {isEnabled,key} array) instead of only the first; drops the "will use the first" warning.
  • Rust redaction src-tauri/src/commands/config/settings/providers.rs: redact_provider_credential strips apiKeys and writes apiKeyCount.
  • Gateway web mirror agent-gateway/web/src/ (i18n/config.ts, lib/settings/{index,normalize,sync}.ts, pages/settings/ProvidersSection.tsx): mirrored to keep gateway-build consistent (no runtime layer, no failover wiring).
  • Tests agent-gui/test/settings/normalization.test.mjs, test/chat/agent-runner.test.mjs: updated sidecar assertions (string → string[]), added createProviderApiKeyFailover to the llmMock.

Screenshots / preview

image

Failover behavior (smoke test, calling createProviderApiKeyFailover directly):

multi-key (apiKeys=[key-0,key-1,key-2]) rotate sequence: key-0 -> key-1 -> key-2 -> key-0 -> key-0
  (primary first; cycles back to primary when exhausted)
single key: failover=undefined, attemptAuth holds the primary
no/undefined apiKeys: attemptAuth=undefined, factory falls back to options.apiKey (zero regression)
attemptAuth.headers rebuilt per key (Authorization/x-api-key/x-goog-api-key) on rotate

Verification

  • cd crates/agent-gui && tsc --noEmit → exit 0 (no type errors).
  • Frontend tests: node --test test/providers/*.test.mjs test/settings/normalization.test.mjs test/chat/agent-runner.test.mjs ... → 859/863 pass; the 4 failures are pre-existing on main (composer/caret DOM tests, reproduced on a clean main via git stash, unrelated to this change).
  • Rust: cargo build --release (src-tauri) → passes; cargo test --release → 646 pass / 2 fail (automation_commands::hook PowerShell-in-bash environment issue, fails identically on clean main, unrelated).
  • Failover logic smoke-tested (above).
  • Affected tests updated (normalization sidecar string→string[], agent-runner llmMock gains createProviderApiKeyFailover).

Pre-submit checklist

  • A requirement issue is linked (Closes [Feature] 一个供应商添加多个api key #365).
  • Synced with the target branch; no merge conflicts (based on latest main 00a2c6fc).
  • The change is focused, with no unrelated modifications (reverted a Cargo.toml line-ending noise).
  • No secrets, tokens, or personal data included.
  • Docs: i18n strings (zh-CN + en) added for the new UI hint; new user-facing capability (multi-key config + failover) with no existing deploy/config docs to sync.

@StackCairn
StackCairn marked this pull request as draft August 7, 2026 16:36
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR governance checks passed. Awaiting human review.

@YUZHEthefool
YUZHEthefool marked this pull request as ready for review August 7, 2026 16:38
每个供应商可配置多个 API Key(参照 Cherry Studio)。请求优先使用主 Key
(首个),流式开始前遇到限额/鉴权/网络瞬时等可重试错误时,自动切换到
下一个 Key 重试,Key 用尽回退主 Key 继续重试预算;一旦开始产出内容则
不再换 Key,避免半截输出。单 Key 链路(用量查询、Go 模型拉取)零回归。

- 数据模型:CustomProvider 新增 apiKeys?: string[] + apiKeyCount?: number
  (脱敏快照),apiKey 恒派生为 apiKeys[0],旧单 Key 快照自动迁移。
- 归一化:normalizeApiKeys 逐项 trim/去空/去重保序。
- 故障转移:streamRetry 增 apiKeyFailover,重试前 rotate;streamByApi 各
  API 分支 factory 每次重读 attemptAuth(apiKey+鉴权头),仅替换鉴权头
  保留代理路由/会话/自定义头。createProviderApiKeyFailover 构造 mutable
  holder + rotate 回调。
- 网关同步:providerApiKeyUpdates 由 Record<string,string> 升级为
  Record<string,string[]>;脱敏摘除 apiKeys 并写 apiKeyCount。
- 桌面端 UI:API Key 改为多行编辑器(增删/统一显隐)+ 故障转移提示。
  WebUI 保持单行脱敏输入 + "已配置 N 个 Key" 提示(替换语义)。
- Cherry Studio 导入:导入全部启用的 Key(v1 逗号分隔 / v2 {isEnabled,
  key} 数组),不再只取第一个、不再报"将使用第一个"警告。
- Rust 脱敏:redact_provider_credential 摘除 apiKeys、写 apiKeyCount。
- gateway web 镜像同步相同改动保持 gateway-build 一致。

Closes Stack-Cairn#365
@YUZHEthefool
YUZHEthefool force-pushed the feat/provider-multi-api-key-failover branch from 3428b4d to 69e1495 Compare August 7, 2026 16:43
@su-fen

su-fen commented Aug 8, 2026

Copy link
Copy Markdown
Member

与之前做的故障转移存在冲突了

@su-fen

su-fen commented Aug 8, 2026

Copy link
Copy Markdown
Member

#385
在这个PR里已经提交了比较完善的故障转移部分功能实现,感觉按供应商来做故障转移更加合理一些。
取舍之下,多个key可以配置多个供应商,然后配置供应商级的故障转移,也能实现同样的效果。感谢佬的提交!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 一个供应商添加多个api key

2 participants