diff --git a/devlog/_plan/260914_godfile_round2/000_plan.md b/devlog/_plan/260914_godfile_round2/000_plan.md new file mode 100644 index 0000000000..ffe4ed88bf --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/000_plan.md @@ -0,0 +1,86 @@ +# 260914 godfile round2 — src 갓파일 분해와 파일 크기 래칫 + +`dev`에는 사람이 유지하는 2,000줄 이상 텍스트 파일이 52개 있고(래칫이 실제로 스캔하는 범위, 즉 `devlog/` 제외 기준으로는 51개) 그중 15개가 `src/`에 있다. 이 단위는 그중 7개를 facade 보존 순수 이동으로 분해하고, 같은 일이 다시 쌓이지 않도록 파일 크기 래칫 게이트를 CI에 넣는다. 기여자에게 바뀌는 것은 두 가지다. 새 파일은 처음부터 2,000줄 미만이어야 하고, 기존 초과 파일은 더 길어질 수 없다. 분해 대상 파일을 import하던 코드는 facade가 남으므로 바뀌지 않는다. + +숫자의 출처는 `git ls-files`에 대한 줄 수 측정이며, 생성물·벤더 스냅샷·로케일 미러 12경로를 제외한 값이다. 그 12경로는 `010_phase1_file_size_ratchet.md`의 `generated` 목록이 권위를 갖는다. + +## 로프스펙 + +| 항목 | 내용 | +|---|---| +| Loop archetype | satisfy-spec. 완료 조건이 파일별로 고정돼 있고 사이클마다 종료한다 | +| Trigger | 사용자 요청: 스택 PR로 쌓고 tip-only CI로 추적하며 5~6 PABCD 사이클로 dev 머지까지 완료 | +| Goal | 7개 `src/` 갓파일을 1,999줄 이하로 분해하고 래칫 게이트와 함께 `origin/dev`에 머지 | +| Non-goals | 기능 정책 변경, 버그 수정 동반, `tests/` 33개 분해, i18n·생성물·devlog 분해, `core.ts`·`server/index.ts`·`auth-api.ts`·`providers/registry.ts` 본체(별도 단위) | +| Verifier | hosted CI. 수동 브랜치 체인의 tip PR 실행을 레인 게이트로 쓴다(DEV-STACK-08, owner 승인) | +| Stop condition | 6개 사이클의 D가 모두 닫히고 레인이 `dev`에 머지될 때 | +| Memory artifact | 이 단위(`devlog/_plan/260914_godfile_round2/`)와 각 PR 본문 | +| Expected outcomes | 성공 = 7파일 facade화 + 래칫 녹색 / 차단 = tip CI 적색이 반복되고 원인이 분해 외부일 때 | +| Escalation | tip CI가 분해와 무관한 이유로 적색이거나, 래칫 기준선이 다른 작업과 충돌할 때 | + +## 제약 + +로컬에 `node_modules`가 없고 사용자 환경에서 install·build·typecheck·full suite를 돌리지 않는다. 따라서 이 단위의 모든 검증은 hosted CI이며, 문서와 PR 본문에서 로컬 실행 결과를 주장하지 않는다. 각 PR의 CI를 개별로 기다리지 않고 후행 추적한다. 최종 판정은 레인 tip의 exact-head 실행이다. + +열린 PR과의 충돌은 순서 제약에서 제외한다(사용자 지시). 순서는 기술 의존성만으로 정한다. 그 대가로 `src/config.ts`에 걸린 21건을 포함해 47건이 리베이스 대상이 되며, 이 단위는 그 비용을 감수한 것으로 기록한다. + +## 작업 단계 지도 + +| 사이클 | 문서 | 대상 | 브랜치 | PR base | +|---|---|---|---|---| +| 0 | `000_plan.md` + 010~050 | 로드맵(코드 변경 없음) | `codex/m2k-l1-roadmap` | `dev` | +| 1 | `010_phase1_file_size_ratchet.md` | 래칫 게이트 | `codex/m2k-l2-ratchet` | L1 | +| 2 | `020_phase2_state_and_shim.md` | `src/responses/state.ts`, `src/codex/shim.ts` | `codex/m2k-l3-state-shim` | L2 | +| 3 | `030_phase3_inject_and_catalog_sync.md` | `src/codex/inject.ts`, `src/codex/catalog/sync.ts` | `codex/m2k-l4-inject-sync` | L3 | +| 4 | `040_phase4_routing_and_quota.md` | `src/codex/routing.ts`, `src/providers/quota.ts` | `codex/m2k-l5-routing-quota` | L4 | +| 5 | `050_phase5_config.md` | `src/config.ts` | `codex/m2k-l6-config` | L5 | + +의존은 단순하다. 사이클 1의 래칫이 먼저 있어야 이후 사이클이 만드는 새 파일이 게이트를 통과했다는 증거를 남길 수 있고, 사이클 2~5는 서로 파일이 겹치지 않으므로 체인 순서는 리뷰 편의를 위한 것이다. 사이클 5를 마지막에 두는 이유는 `src/config.ts`가 가장 많은 문서(10곳)와 오라클(9건)을 끌고 있어 앞 단계에서 얻은 패턴을 그대로 쓰기 위해서다. + +사이클 내부의 PR 순서는 각 decade 문서가 소유한다. 특히 `050_phase5_config.md`는 초안의 묶음에 순환 의존이 있음을 실측으로 확인하고 순서를 재배치했다(salvage가 `configSchema`를, diagnostics가 salvage와 load-degrade를, live-reconcile이 `persistConfigUnlocked`를 쓴다). 이 문서의 표는 사이클 경계만 정의하며, 사이클 안의 순서는 decade 문서를 따른다. + +## 스택 형태 + +수동 브랜치 체인이다. GitHub 네이티브 스택은 사용하지 않는다(DEV-STACK-OPT-IN-01: 명시적 opt-in 없음). 각 링크의 PR base는 바로 아래 링크의 head 브랜치이고, 최하단 L1만 `dev`를 base로 한다. + +``` +codex/m2k-l6-config → PR (base: l5) ← tip +codex/m2k-l5-routing-quota → PR (base: l4) +codex/m2k-l4-inject-sync → PR (base: l3) +codex/m2k-l3-state-shim → PR (base: l2) +codex/m2k-l2-ratchet → PR (base: l1) +codex/m2k-l1-roadmap → PR (base: dev) ← bottom +─────────────────────────── dev +``` + +## CI 정책 (DEV-STACK-08, owner 승인) + +비-tip 링크의 head 커밋 제목에 `[skip ci]`를 붙여 tip만 비싼 스위트를 돌린다. 이 전략은 저장소 소유자가 이 배치에 한해 승인한 예외이며 기본값이 아니다. + +지켜야 할 것은 셋이다. 누락·스킵·취소된 체크는 통과가 아니다. `[skip ci]`가 trunk에 착지하는 커밋 제목에 도달하면 안 된다(머지 커밋 제목에는 붙이지 않는다). 레인이 착지한 뒤 `dev`를 관찰하고 적색이면 다음 레인을 멈춘다. + +## 머지 순서 + +체인 자식은 top-down으로 머지한다. 스택 자식을 머지하면 trunk가 아니라 부모 브랜치에 착지하기 때문이다. L6 → L5 → L4 → L3 → L2 순으로 각각 부모에 착지시키고, 마지막에 L1(base `dev`)을 머지하면 전체가 `dev`에 올라간다. L1을 머지하기 직전의 exact-head CI가 이 단위의 최종 게이트다. + +각 머지 전에 조상 불변식을 확인한다. + +```sh +git merge-base --is-ancestor origin/ +``` + +## 완료 조건 + +| 검사 | 조건 | +|---|---| +| 파일 크기 | 대상 7개가 전부 1,999줄 이하, 새 모듈 전부 1,999줄 이하 | +| 래칫 | 기준선 대비 증가 0. 각 사이클 D에서 `ratchet:update`로 기준선 회수 | +| 상태 소유권 | 각 decade 문서가 지정한 소유 모듈 배치대로, 인자로 새는 상태 0 | +| 금지 분할 | 각 decade 문서의 함정 항목 미발생 | +| 오라클·INV | 본문을 텍스트로 읽는 오라클의 읽기 경로 갱신 완료, INV 승계 모듈 지정 | +| 문서 | `structure/` 백틱 참조와 소유권 갱신, `bun run structure:check` 녹색 | +| 머지 | 6개 PR 전부 MERGED, `dev` 최종 CI 녹색 | + +## 사이클 D에서 기록할 것 + +각 사이클은 D에서 다음을 이 단위에 남긴다. 남은 초과 파일 수, PR 번호와 head SHA, 관찰한 CI 실행 ID와 결론, 개선되지 않은 것과 죽은 가설(LOOP-PESSIMIST-01). diff --git a/devlog/_plan/260914_godfile_round2/010_phase1_file_size_ratchet.md b/devlog/_plan/260914_godfile_round2/010_phase1_file_size_ratchet.md new file mode 100644 index 0000000000..c073f95f79 --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/010_phase1_file_size_ratchet.md @@ -0,0 +1,686 @@ +2,000줄 초과 파일이 더 길어지거나 새로 생기는 것을 CI가 막지 못한다. 이 단계는 파일을 쪼개지 않고, 커밋된 기준선 JSON과 `evaluate()` 판정기로 그 게이트만 넣는다. 다음 사이클이 갓파일을 줄이면 `ratchet:update`가 캡을 내리고, 기여자는 새 파일을 2,000줄 미만으로 유지해야 하며, 이미 기준선에 있는 파일은 한 줄도 늘릴 수 없다. + +## 이 사이클이 하는 일 / 하지 않는 일 + +하는 일: 스캐너, 순수 `evaluate()`, `--update`, 커밋된 기준선, 그 기준선을 읽는 bun 테스트 하나, layout 양쪽 등록, `package.json` 스크립트 한 줄. 게이트는 새 CI job이 아니라 기존 `bun test` 스위트다. `.github/workflows/ci.yml`은 만지지 않는다. + +하지 않는 일: `src/`·`tests/` 갓파일 분할, 생성물 재생성, `fetch-depth` 변경, layout 정규식에 `file-` 시드 추가, `prepush`에 래칫 연결, INV 신설, `structure/` 본문 수정. + +이동할 원본 행 범위: 없음. 이 사이클은 분할이 아니다. + +## 왜 git으로 base를 못 구하는가 (실측) + +`.github/workflows/ci.yml:7`은 `pull_request: {}`다. 테스트 job 체크아웃은 `.github/workflows/ci.yml:278-294`다. + +```yaml + - name: Checkout + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 + with: + persist-credentials: false + # ... + # Tags only, not full history: `fetch-depth: 0` would clone every commit to + # answer a question about refs. + fetch-tags: true +``` + +정정: 계약 초안은 "fetch-depth 기본 1(해당 줄 번호)"이라고 했지만, `fetch-depth: 1`을 적은 줄은 이 워크플로에 없다. `changes` job(`.github/workflows/ci.yml:161-166`)과 `test` job(`:278-294`) 모두 키를 생략한다. actions/checkout v7 `action.yml`의 `fetch-depth` default는 1이고, README는 "Only a single commit is fetched by default, for the ref/SHA that triggered the workflow"라고 한다. 같은 파일 `:289-293`은 `fetch-depth: 0`을 태그를 위해 켜지 말라고 적어 두었다. 히스토리를 일부러 안 가져온다. + +PR에서 그 단일 커밋의 ref는 `refs/pull//merge`다. checkout README의 "Checkout pull request HEAD commit instead of merge commit" 절이, 기본값이 merge commit임을 전제로 `ref: ${{ github.event.pull_request.head.sha }}` 예외를 보여 준다. 깊이 1짜리 merge commit에는 `origin/dev`도 부모 트리도 없다. `git diff origin/dev --stat`이나 `git show HEAD^:src/config.ts`는 CI에서 실패하거나 빈 비교가 된다. 그래서 캡은 반드시 커밋된 JSON이어야 한다. + +## 판정기 + +줄 수 공식(계약 그대로): + +```ts +text.split("\n").length - (text.endsWith("\n") ? 1 : 0) +``` + +빈 파일(`""`)은 이 공식에서 1이다. 2,000줄 판정에는 영향 없다. `wc -l`과 끝 개행 없는 파일에서 어긋날 수 있으므로 `wc -l`로 기준선을 만들지 않는다. + +`evaluate(files, baseline)`만 판정한다. 디스크를 읽지 않고, 전역 캐시도 없다. 스캔 목록은 호출자가 넣는다. + +| 조건 | verdict | CI | +|---|---|---| +| `path ∈ baseline.generated` | `GENERATED` | 통과. 줄 수 무시 | +| `path ∈ baseline.files` 이고 `lines > cap` | `GREW` | 실패 | +| `path ∈ baseline.files` 이고 `lines < cap` | `SHRANK` | 통과 | +| `path ∈ baseline.files` 이고 `lines === cap` | `UNCHANGED` | 통과 | +| baseline에 없고 `lines >= 2000` | `NEW_OVERSIZED` | 실패 | +| baseline에 없고 `lines < 2000` | `NEW_OK` | 통과 | + +실패는 `GREW`와 `NEW_OVERSIZED`뿐이다. + +스캔 대상은 `git ls-files`가 돌려 준 경로 중 아래를 통과한 것이다. 워킹트리 walk 금지. + +확장자 화이트리스트: `.ts` `.tsx` `.js` `.cjs` `.mjs` `.json` `.css` `.md` `.yml` `.yaml` `.sh`. `path.extname`으로 비교한다. `.mdx`·`.toml`·`.jsonc`는 화이트리스트 밖이다. 정정: `scripts/privacy-scan.ts:11`의 `TEXT_FILE_RE`는 html/jsonc/md/ps1/toml/txt까지 포함하지만, 이 게이트는 계약 화이트리스트만 쓴다. + +스캔제외(이 순서로): + +1. 경로가 `bun.lock` 또는 `gui/dist`와 정확히 같음 +2. 접두 `devlog/`, `assets/`, `docs-site/public/`, `docs-site/src/assets/`, `gui/dist/` +3. 확장자가 화이트리스트 밖 + +`node_modules/`는 tracked가 아니므로 `git ls-files`가 안 준다. 접두에 넣지 않는다. + +## generated 면제 — 정확 경로 12개, glob 금지 + +`baseline.generated`는 디렉터리가 아니라 아래 12개 문자열과 같아야 한다. + +``` +scripts/model-metadata.source.json +src/adapters/cursor/gen/agent_pb.ts +gui/src/i18n/de.ts +gui/src/i18n/en.ts +gui/src/i18n/fr.ts +gui/src/i18n/ja.ts +gui/src/i18n/ko.ts +gui/src/i18n/ru.ts +gui/src/i18n/tr.ts +gui/src/i18n/zh.ts +gui/src/i18n/zh-TW.ts +docs-site/src/data/frontier-benchmarks.json +``` + +정정: 이 12개가 모두 codegen은 아니다. 필드명은 계약대로 `generated`로 두되, 의미는 래칫 면제다. + +- `src/adapters/cursor/gen/agent_pb.ts:1` — `// @generated by protoc-gen-es v2.10.2`. 유일한 실제 생성물. +- `gui/src/i18n/en.ts:1` — "English — source of truth". 나머지 로케일은 키를 맞춰야 하는 손 유지 카탈로그. `000_plan.md` non-goals가 i18n 분해를 빼므로 면제한다. +- `scripts/model-metadata.source.json` — `scripts/generate-model-metadata.ts:29-32`가 읽는 벤더 스냅샷 소스다. 생성 출력은 `src/generated/model-metadata.ts`. 실측 86,334줄. +- `docs-site/src/data/frontier-benchmarks.json` — `docs-site/src/data/README-frontier.md:4-8`이 "hand-maintained snapshots"라고 적는다. 실측 2,665줄. + +`src/adapters/cursor/gen/**` glob으로 빼지 마라. 그 디렉터리에 손 파일이 생기면 래칫 대상이어야 한다. + +## 기준선 스키마와 --update + +`tests/fixtures/file-size-baseline.json`: + +```json +{ + "generated": [ "...12 paths..." ], + "files": { + "src/config.ts": 4707 + } +} +``` + +`files`는 grandfather 캡이다. 2,000줄 미만 파일은 넣지 않는다. 전부 넣으면 한 줄 추가마다 `GREW`가 나서 저장소가 동결된다. 새 파일은 1,999줄까지 `NEW_OK`로 자랄 수 있다. + +`--update` (시드가 아닐 때): + +1. 지금 tracked가 아닌 키는 삭제 +2. 남은 키는 `min(old, current)` — 캡을 올리지 않음 +3. 새 경로를 넣지 않음. 새 2,000+는 `NEW_OVERSIZED`로 남긴다 +4. 2,000 밑으로 줄어든 키도 삭제하지 않음. 갓파일 facade가 800줄이 되면 800이 새 캡이다 + +시드: `tests/fixtures/file-size-baseline.json`이 없을 때만. `generated`는 위 12개, `files`는 면제 목록을 뺀 현재 스캔 결과 중 `lines >= 2000`. `package.json`의 `ratchet:update`는 `--update`만 호출한다. 이후 사이클은 이 명령으로 캡을 회수한다. + +정정: 워킹트리 rglob 실측으로 2,000줄 이상 63개, 면제 12개를 빼면 사람 유지 51개다. `000_plan.md`의 53과 어긋난다. `src/` 15개는 일치한다(`core.ts` 8,911부터 `bridge.ts` 2,206, `agent_pb.ts` 제외). 커밋 숫자의 권위는 `git ls-files` 시드다. rglob 초안을 JSON에 붙이지 마라. + +## Write set (L2 구현 PR, 이 문서 제외) + +| 경로 | 동작 | 원본 행 범위 | 예상 줄 수 | +|---|---|---|---| +| `scripts/file-size-ratchet.ts` | NEW | 없음 | 184 | +| `tests/ci-workflows/file-size-ratchet.test.ts` | NEW | 없음 | 212 | +| `tests/fixtures/file-size-baseline.json` | NEW | 없음 | 시드 후 ~70 (generated 12 + files ~51) | +| `scripts/test-layout/layout.json` | MODIFY +1 | 703행과 704행 사이 삽입 | 1468 → 1469 | +| `tests/fixtures/test-layout-expected.json` | MODIFY +1 | 534행과 535행 사이 삽입 | 1275 → 1276 | +| `package.json` | MODIFY +1 | 55행 다음 삽입 | 120 → 121 | + +L1 문서(지금 이 파일)는 이미 로드맵 PR write set이다. L2가 이 문서를 다시 쓰지 않는다. + +### layout.json 삽입 (실측) + +`scripts/test-layout/layout.json:96-100` ci-workflows 시드: + +``` +"^(?:assert|build|bump|ci|cleanup|closed|docs|dsh|fixture|install|keyring|package|release|repo|skill|test|zz)-" +``` + +`file-`가 없다. `scripts/test-layout/schema.ts:47-59`는 explicit → child regex → domain regex 순이다. 시드만으로는 `file-size-ratchet.test.ts`가 `null`이 되어 `tests/test-layout-tooling.test.ts:255-262`의 `unresolvedNew`가 터진다. 정규식에 `file-`를 더하지 마라. 그건 +1이 아니고, 다른 `file-*.test.ts`를 ci-workflows로 끌어들인다. + +알파벳: `fetch-header-timeout.test.ts` < `file-size-ratchet.test.ts` < `fixture-dir-uniqueness.test.ts`. + +`scripts/test-layout/layout.json:703-704` 지금: + +``` + "fetch-header-timeout.test.ts": "server", + "fixture-dir-uniqueness.test.ts": "ci-workflows", +``` + +703과 704 사이에 한 줄: + +``` + "file-size-ratchet.test.ts": "ci-workflows", +``` + +`tests/fixtures/test-layout-expected.json:534-535` 지금: + +``` + "fetch-header-timeout.test.ts": "server", + "fixture-dir-uniqueness.test.ts": "ci-workflows", +``` + +534와 535 사이에 한 줄: + +``` + "file-size-ratchet.test.ts": "ci-workflows", +``` + +한 쪽만 고치면 `tests/test-layout-tooling.test.ts:248-250`이 `layout.explicit`과 `EXPECTED`의 완전일치를 요구하므로 적색이다. + +### package.json 삽입 (실측) + +`package.json:55-56` 지금: + +``` + "structure:check": "bun scripts/structure-ssot.ts", + "generate:model-metadata": "bun scripts/generate-model-metadata.ts", +``` + +55행 다음에: + +``` + "ratchet:update": "bun scripts/file-size-ratchet.ts --update", +``` + +`prepush`(`package.json:65`)에 넣지 마라. `tests/ci-workflows/ci-workflows.test.ts:5294`가 `"bun run privacy:scan && bun run doctor:gui:if-changed"` 문자열을 고정한다. `tests/ci-workflows/install-scripts.test.ts:72-76`은 특정 키만 보므로 키 추가는 안전하다. + +## 기존 패턴 (그대로 복제할 부분) + +`tests/ci-workflows/repo-hygiene.test.ts:4-6, 33-44, 61-66`: + +```ts +import { repoRoot as resolveRepoRoot } from "../helpers/repo-root"; +const repoRoot = resolveRepoRoot(); + +function trackedFiles(): string[] { + const result = Bun.spawnSync(["git", "ls-files"], { cwd: repoRoot }); + if (result.exitCode !== 0) { + throw new Error(`git ls-files failed: ${new TextDecoder().decode(result.stderr)}`); + } + return new TextDecoder() + .decode(result.stdout) + .split("\n") + .map((line) => line.trim()) + .filter(Boolean); +} + + expect(offenders).toEqual([]); +``` + +스캐너의 `gitLsFiles`는 이 spawn을 쓴다. 저장소 스캔 테스트는 `expect(rows.filter(isOffender)).toEqual([])`. + +정정: `scripts/privacy-scan.ts`는 `git ls-files`를 쓰지만 `import.meta.main` 없이 모듈 로드 시 스캔을 실행한다(`scripts/privacy-scan.ts:269` 최상위 `findings`). `tests/ci-workflows/privacy-scan-meta-key.test.ts:17`이 `scanText`를 import하는 순간 전체 스캔이 돈다. 이 스크립트는 `scripts/structure-ssot.ts:544`처럼 `if (import.meta.main)` 뒤에만 CLI를 둔다. 테스트를 위해 `evaluate`를 import할 때 디스크를 읽으면 안 된다. + +## 상태 소유권 + +캡의 SSOT는 `tests/fixtures/file-size-baseline.json`이다. `evaluate(files, baseline)`는 두 인자를 읽고 객체를 돌려줄 뿐 아무것도 쓰지 않는다. 줄 수 맵을 모듈 스코프에 캐시하지 않는다. CLI만이 `--update`일 때 JSON을 쓴다. 단위 테스트는 메모리의 `files`/`baseline`만 넘긴다. 저장소 스캔 테스트 한 건만 커밋된 JSON과 `git ls-files`를 읽는다. + +인자로 새면 안 되는 것: `process.cwd()`에 의존하는 스캔(테스트 cwd가 루트가 아닐 수 있다 — 반드시 `repoRoot()`/`import.meta.dir`에서 올린 루트), `HEAD` 또는 merge-base에서 읽은 "이전 줄 수", `baseline.generated`를 무시하고 스크립트 상수만 보는 판정(`evaluate`는 JSON이 권위). 상수 `GENERATED_PATHS`는 시드 폴백과 테스트의 12경로 고정 검사용이다. + +## 하지 말아야 할 분할 (함정) + +1. 이 PR에서 `src/`나 `tests/`를 쪼개지 마라. 게이트만 넣는다. +2. `ci.yml`에 `fetch-depth: 0`을 넣어 git 히스토리로 기준선을 대체하지 마라. write set 밖이고, `:289-293`이 거부한 비용이다. +3. generated를 glob이나 디렉터리로 빼지 마라. +4. 모든 스캔 파일을 `files`에 넣지 마라. 동결이다. +5. `--update`가 캡을 올리거나 새 2,000+를 편입하게 하지 마라. `NEW_OVERSIZED`가 죽는다. +6. 줄어든 키를 2,000 미만이라고 삭제하지 마라. facade가 다시 자란다. +7. layout 시드에 `file-`를 추가하지 마라. explicit 양쪽 +1만 한다. +8. 테스트를 `tests/file-size-ratchet.test.ts` 루트에 두지 마라. INV-TESTS-01 위반이다. +9. `prepush`에 래칫을 넣지 마라. +10. `privacy-scan.ts`의 import-시-실행을 베끼지 마라. +11. 기준선 숫자를 `wc -l`이나 워킹트리 rglob로 커밋하지 마라. 시드 CLI만 권위다. +12. `gui/src/i18n/**` 또는 `src/adapters/cursor/gen/**`로 면제하지 마라. 정확 12경로만. + +## 동반 수정 의무 + +structure/ 백틱 참조: 이 사이클은 경로를 옮기거나 지우지 않는다. `structure/INDEX.md:96` `scripts/` 소유 문서는 `overview.md`, `ops/docs-and-release.md`다. 둘 다 `scripts/file-size-ratchet.ts`를 백틱으로 지명하지 않고, 기존 백틱 경로가 사라지지도 않는다. 새 `src/` area가 없다. `structure/` MODIFY는 0이다. `bun run structure:check`는 파일 추가만으로 적색이 되면 안 된다. + +본문을 텍스트로 읽는 소스 오라클: + +- `tests/test-layout-tooling.test.ts:248-262` — layout explicit ↔ expected 일치, unresolvedNew. 양쪽 +1이 이 오라클의 동반 수정이다. +- `tests/test-layout.test.ts` — 도메인 폴더 배치. 테스트 파일이 `tests/ci-workflows/`에 있으면 통과. +- `tests/ci-workflows/ci-workflows.test.ts:5294` — `prepush` 문자열. 만지지 않으면 동반 수정 없음. +- `tests/ci-workflows/install-scripts.test.ts:72-76` — 특정 스크립트 키만. 동반 수정 없음. +- `tests/helpers/repo-root.ts:12-32` — 저장소 스캔 테스트는 `repoRoot()`/`repoPath()`만 쓴다. `import.meta.dir + "/../.."` 금지 (`structure/overview.md:105-107`). + +INV 승계: 분할이 없으므로 모듈 INV 승계는 없다. 이 테스트가 묶이는 기존 불변식은 `structure/overview.md:103-107` **INV-TESTS-01** (`scripts/test-layout/layout.json` + `tests/test-layout.test.ts`). 파일 크기 INV(`INV-SIZE-01` 등)는 이 사이클 write set 밖이다. 게이트는 테스트가 강제한다. + +layout 등록: 위 삽입 두 줄. `scripts/test-layout/layout.json` explicit와 `tests/fixtures/test-layout-expected.json` 둘 다. 시드 변경 없음. + +## 회귀 테스트 경로 + +구현 후 hosted CI가 돌리는 것(로컬 스위트 금지, 이 단위 제약): + +- `tests/ci-workflows/file-size-ratchet.test.ts` — 순수 5 + 저장소 스캔 1 +- `tests/test-layout.test.ts`, `tests/test-layout-tooling.test.ts` — layout 등록 +- 기존 스위트가 이 테스트를 shard에 포함 + +로컬에서 허용되는 것은 문서에 적힌 파일을 쓰는 것과, 기준선 시드를 위한 `bun scripts/file-size-ratchet.ts --update` 한 번뿐이다. `bun run test` / `typecheck` / `install`을 이 문서의 검증 주장에 쓰지 마라. + +## 구현 순서 + +1. 아래 초안을 `scripts/file-size-ratchet.ts`로 저장 +2. 기준선 파일이 없는 상태에서 `bun scripts/file-size-ratchet.ts --update` → JSON 생성 +3. 아래 초안을 `tests/ci-workflows/file-size-ratchet.test.ts`로 저장 +4. layout.json 703/704 사이, expected.json 534/535 사이, package.json 55행 다음 +5. 커밋. 게이트는 그 커밋을 head로 하는 hosted `bun test` + +## scripts/file-size-ratchet.ts 초안 (복붙, 184줄) + +```ts +import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { extname, join, resolve } from "node:path"; + +export const THRESHOLD = 2000; +export const BASELINE_REL = "tests/fixtures/file-size-baseline.json"; + +export const SCAN_EXTENSIONS = new Set([ + ".ts", + ".tsx", + ".js", + ".cjs", + ".mjs", + ".json", + ".css", + ".md", + ".yml", + ".yaml", + ".sh", +]); + +export const EXCLUDED_PREFIXES = [ + "devlog/", + "assets/", + "docs-site/public/", + "docs-site/src/assets/", + "gui/dist/", +] as const; + +export const EXCLUDED_EXACT = new Set(["bun.lock", "gui/dist"]); + +export const GENERATED_PATHS = [ + "scripts/model-metadata.source.json", + "src/adapters/cursor/gen/agent_pb.ts", + "gui/src/i18n/de.ts", + "gui/src/i18n/en.ts", + "gui/src/i18n/fr.ts", + "gui/src/i18n/ja.ts", + "gui/src/i18n/ko.ts", + "gui/src/i18n/ru.ts", + "gui/src/i18n/tr.ts", + "gui/src/i18n/zh.ts", + "gui/src/i18n/zh-TW.ts", + "docs-site/src/data/frontier-benchmarks.json", +] as const; + +export type Verdict = + | "NEW_OVERSIZED" + | "GREW" + | "SHRANK" + | "GENERATED" + | "UNCHANGED" + | "NEW_OK"; + +export type Baseline = { + generated: string[]; + files: Record; +}; + +export type FileSize = { + path: string; + lines: number; +}; + +export type Evaluation = FileSize & { + verdict: Verdict; +}; + +export function countLines(text: string): number { + return text.split("\n").length - (text.endsWith("\n") ? 1 : 0); +} + +export function isScannedPath(path: string): boolean { + if (EXCLUDED_EXACT.has(path)) return false; + if (EXCLUDED_PREFIXES.some((prefix) => path.startsWith(prefix))) return false; + return SCAN_EXTENSIONS.has(extname(path)); +} + +export function evaluate(files: FileSize[], baseline: Baseline): Evaluation[] { + const generated = new Set(baseline.generated); + return files.map((file) => { + if (generated.has(file.path)) return { ...file, verdict: "GENERATED" }; + const cap = baseline.files[file.path]; + if (cap === undefined) { + return { ...file, verdict: file.lines >= THRESHOLD ? "NEW_OVERSIZED" : "NEW_OK" }; + } + if (file.lines > cap) return { ...file, verdict: "GREW" }; + if (file.lines < cap) return { ...file, verdict: "SHRANK" }; + return { ...file, verdict: "UNCHANGED" }; + }); +} + +export function isOffender(row: Evaluation): boolean { + return row.verdict === "NEW_OVERSIZED" || row.verdict === "GREW"; +} + +export function gitLsFiles(repoRoot: string): string[] { + const result = Bun.spawnSync(["git", "ls-files"], { cwd: repoRoot }); + if (result.exitCode !== 0) { + throw new Error(`git ls-files failed: ${new TextDecoder().decode(result.stderr)}`); + } + return new TextDecoder() + .decode(result.stdout) + .split("\n") + .map((line) => line.trim()) + .filter(Boolean); +} + +export function scanRepo(repoRoot: string): FileSize[] { + const out: FileSize[] = []; + for (const path of gitLsFiles(repoRoot)) { + if (!isScannedPath(path)) continue; + out.push({ path, lines: countLines(readFileSync(join(repoRoot, path), "utf8")) }); + } + return out; +} + +export function loadBaseline(text: string): Baseline { + const parsed = JSON.parse(text) as Baseline; + if ( + !parsed + || typeof parsed !== "object" + || !Array.isArray(parsed.generated) + || typeof parsed.files !== "object" + || parsed.files === null + || Array.isArray(parsed.files) + ) { + throw new Error("invalid file-size baseline"); + } + return parsed; +} + +function sortRecord(input: Record): Record { + return Object.fromEntries( + Object.entries(input).sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0)), + ); +} + +export function updateBaseline(current: FileSize[], baseline: Baseline, seed: boolean): Baseline { + const now = new Map(current.map((file) => [file.path, file.lines] as const)); + const files: Record = {}; + for (const [path, cap] of Object.entries(baseline.files)) { + const lines = now.get(path); + if (lines === undefined) continue; + files[path] = Math.min(cap, lines); + } + if (seed) { + const generated = new Set(baseline.generated); + for (const [path, lines] of now) { + if (generated.has(path) || lines < THRESHOLD || files[path] !== undefined) continue; + files[path] = lines; + } + } + return { generated: [...baseline.generated], files: sortRecord(files) }; +} + +export function formatOffenders(rows: Evaluation[]): string { + return rows + .filter(isOffender) + .map((row) => `${row.verdict} ${row.path} ${row.lines}`) + .join("\n"); +} + +if (import.meta.main) { + const repoRoot = resolve(import.meta.dir, ".."); + const baselinePath = join(repoRoot, BASELINE_REL); + const existed = existsSync(baselinePath); + const baseline: Baseline = existed + ? loadBaseline(readFileSync(baselinePath, "utf8")) + : { generated: [...GENERATED_PATHS], files: {} }; + const current = scanRepo(repoRoot); + if (process.argv.includes("--update")) { + const next = updateBaseline(current, baseline, !existed); + writeFileSync(baselinePath, `${JSON.stringify(next, null, 2)}\n`); + console.log(`wrote ${BASELINE_REL} (${Object.keys(next.files).length} caps)`); + process.exit(0); + } + const offenders = evaluate(current, baseline).filter(isOffender); + if (offenders.length > 0) { + console.error("file-size ratchet failed:"); + console.error(formatOffenders(offenders)); + process.exit(1); + } + console.log("file-size ratchet passed"); +} +``` + +## tests/ci-workflows/file-size-ratchet.test.ts 초안 (복붙, 212줄) + +```ts +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; + +/** + * Cycle 1 of 260914_godfile_round2. No file is split here. The gate is a bun + * test in the existing suite, not a new ci.yml job, because PR checkouts are a + * single refs/pull/N/merge commit at fetch-depth 1 and cannot see origin/dev. + * + * The scanner exports evaluate() so this file can feed it synthetic FileSize + * rows. Importing the module must not scan the repository: privacy-scan.ts runs + * on import and that pattern is forbidden here. + * + * Source-oracle reads go through tests/helpers/repo-root.ts (INV-TESTS-01). + */ +import { + GENERATED_PATHS, + THRESHOLD, + countLines, + evaluate, + isOffender, + isScannedPath, + loadBaseline, + scanRepo, + updateBaseline, + type Baseline, + type FileSize, +} from "../../scripts/file-size-ratchet"; +import { repoPath, repoRoot } from "../helpers/repo-root"; + +/** + * The ratchet must fail for the reason it claims. A single "repo is currently + * green" test would stay green if evaluate() started returning NEW_OK for a + * 2,000-line new file, as long as this tree had no such file today. + * + * Five pure cases plus one repository scan. Do not add a seventh test(): + * SHRANK already covers updateBaseline (lower, drop missing, never raise, + * seed only when asked). + */ +const emptyBaseline = (): Baseline => ({ generated: [], files: {} }); + +const linesOf = (count: number): string => { + const rows = Array.from({ length: count }, (_, i) => `line ${i}`); + return `${rows.join("\n")}\n`; +}; + +describe("file-size ratchet: countLines", () => { + test("NEW_OVERSIZED: baseline에 없고 2000줄 이상이면 실패", () => { + // The formula is the contract: split on \n, then drop the phantom cell that a + // trailing newline creates. wc -l disagrees on files that do not end in a newline, + // so the helper is asserted here instead of trusted from the scanner comments. + expect(countLines(linesOf(THRESHOLD))).toBe(THRESHOLD); + expect(countLines(linesOf(THRESHOLD - 1))).toBe(THRESHOLD - 1); + expect(countLines("")).toBe(1); + expect(countLines("a\nb")).toBe(2); + expect(countLines("a\nb\n")).toBe(2); + + const oversized: FileSize[] = [{ path: "src/new-god.ts", lines: THRESHOLD }]; + const under: FileSize[] = [{ path: "src/new-small.ts", lines: THRESHOLD - 1 }]; + const baseline = emptyBaseline(); + + expect(evaluate(oversized, baseline)).toEqual([ + { path: "src/new-god.ts", lines: THRESHOLD, verdict: "NEW_OVERSIZED" }, + ]); + expect(evaluate(under, baseline)).toEqual([ + { path: "src/new-small.ts", lines: THRESHOLD - 1, verdict: "NEW_OK" }, + ]); + expect(evaluate(oversized, baseline).filter(isOffender)).toHaveLength(1); + expect(evaluate(under, baseline).filter(isOffender)).toEqual([]); + }); +}); + +describe("file-size ratchet: caps", () => { + test("GREW: baseline 캡보다 길어지면 실패", () => { + // Grandfathered files may stay oversized, but they may not grow. Equality is + // UNCHANGED, not SHRANK; a test that only checked isOffender() would not notice + // if equality started reporting GREW. + const baseline: Baseline = { generated: [], files: { "src/config.ts": 4707 } }; + const grew = evaluate([{ path: "src/config.ts", lines: 4708 }], baseline); + const same = evaluate([{ path: "src/config.ts", lines: 4707 }], baseline); + + expect(grew).toEqual([{ path: "src/config.ts", lines: 4708, verdict: "GREW" }]); + expect(same).toEqual([{ path: "src/config.ts", lines: 4707, verdict: "UNCHANGED" }]); + expect(grew.filter(isOffender)).toHaveLength(1); + expect(same.filter(isOffender)).toEqual([]); + }); + + test("SHRANK: 줄면 통과하고 --update는 캡을 내리기만 한다", () => { + // --update is operator tooling, not a seventh test(). The seed path is the only + // way a 2,000+ file enters `files`; after that, a later --update without seed + // must not re-grandfather a new godfile, must not raise a cap, and must keep a + // shrunken former godfile so the facade cannot grow back. + const baseline: Baseline = { + generated: [], + files: { "src/keep.ts": 2100, "src/gone.ts": 2500, "src/small.ts": 800 }, + }; + const current: FileSize[] = [ + { path: "src/keep.ts", lines: 2099 }, + { path: "src/small.ts", lines: 800 }, + { path: "src/new-ok.ts", lines: 1200 }, + ]; + + expect(evaluate(current, baseline)).toEqual([ + { path: "src/keep.ts", lines: 2099, verdict: "SHRANK" }, + { path: "src/small.ts", lines: 800, verdict: "UNCHANGED" }, + { path: "src/new-ok.ts", lines: 1200, verdict: "NEW_OK" }, + ]); + expect(evaluate(current, baseline).filter(isOffender)).toEqual([]); + + // seed=false: lower keep, drop gone, do not add new-ok (it is under 2000 and + // must remain free to grow until 1999). small.ts stays at 800 even though it + // is under the threshold — a former godfile must not grow back. + const lowered = updateBaseline(current, baseline, false); + expect(lowered.files).toEqual({ "src/keep.ts": 2099, "src/small.ts": 800 }); + expect(lowered.files["src/gone.ts"]).toBeUndefined(); + expect(lowered.files["src/new-ok.ts"]).toBeUndefined(); + + // A later --update must never raise. If it did, ratchet:update would launder GREW. + const notRaised = updateBaseline( + [{ path: "src/keep.ts", lines: 3000 }], + { generated: [], files: { "src/keep.ts": 2099 } }, + false, + ); + expect(notRaised.files["src/keep.ts"]).toBe(2099); + + // seed=true is the first-commit path only (baseline file missing). Exempt + // generated paths stay out of files even at 9000 lines. Under-threshold files + // stay out so the 2,000 cap remains the policy for new modules. + const seeded = updateBaseline( + [ + { path: "src/old.ts", lines: 2500 }, + { path: "src/fresh.ts", lines: 1800 }, + { path: "gui/src/i18n/en.ts", lines: 9000 }, + ], + { generated: ["gui/src/i18n/en.ts"], files: {} }, + true, + ); + expect(seeded.files).toEqual({ "src/old.ts": 2500 }); + }); + + test("GENERATED: baseline.generated 경로는 커져도 통과", () => { + // Exact paths only. A sibling under cursor/gen/ that is not in generated[] is a + // new oversized file, even though a glob would have exempted the whole directory. + const path = "src/adapters/cursor/gen/agent_pb.ts"; + const baseline: Baseline = { + generated: [path], + files: { [path]: 100 }, + }; + const rows = evaluate([{ path, lines: 99_999 }], baseline); + expect(rows).toEqual([{ path, lines: 99_999, verdict: "GENERATED" }]); + expect(rows.filter(isOffender)).toEqual([]); + + const globWouldHaveCaught = evaluate( + [{ path: "src/adapters/cursor/gen/hand-written.ts", lines: 2500 }], + { generated: [path], files: {} }, + ); + expect(globWouldHaveCaught[0]?.verdict).toBe("NEW_OVERSIZED"); + }); +}); + +describe("file-size ratchet: scan filter", () => { + test("스캔제외: 화이트리스트 밖·제외 접두·bun.lock은 evaluate에 안 들어온다", () => { + // evaluate() never sees excluded paths; the filter is isScannedPath(). devlog/, + // assets, docs-site public/assets, gui/dist, bun.lock, and non-whitelist + // extensions (.mdx, .png) stay out. src/generated/model-metadata.ts is scanned: + // it is not on the 12-path exemption list, and if it crosses 2,000 it must fail. + // Whitelist hits. .yml and .json are in the contract list; .mdx is not. + expect(isScannedPath("src/config.ts")).toBe(true); + expect(isScannedPath("gui/src/pages/Models.tsx")).toBe(true); + expect(isScannedPath(".github/workflows/ci.yml")).toBe(true); + expect(isScannedPath("scripts/foo.sh")).toBe(true); + expect(isScannedPath("package.json")).toBe(true); + expect(isScannedPath("README.md")).toBe(true); + expect(isScannedPath("gui/src/styles.css")).toBe(true); + expect(isScannedPath(".github/scripts/issue-quality.test.cjs")).toBe(true); + expect(isScannedPath("scripts/foo.mjs")).toBe(true); + + // Prefix and exact exclusions. gui/dist without a trailing slash is listed + // in the contract alongside gui/dist/ children. + expect(isScannedPath("devlog/_plan/260914_godfile_round2/010.md")).toBe(false); + expect(isScannedPath("assets/banner.png")).toBe(false); + expect(isScannedPath("docs-site/public/favicon.png")).toBe(false); + expect(isScannedPath("docs-site/src/assets/og.png")).toBe(false); + expect(isScannedPath("gui/dist/index.js")).toBe(false); + expect(isScannedPath("gui/dist")).toBe(false); + expect(isScannedPath("bun.lock")).toBe(false); + expect(isScannedPath("docs-site/src/content/docs/index.mdx")).toBe(false); + expect(isScannedPath("src/generated/model-metadata.ts")).toBe(true); + }); +}); + +describe("file-size ratchet: repository", () => { + test("저장소 스캔: 커밋된 기준선 대비 offender가 없다", () => { + // Mirrors tests/ci-workflows/repo-hygiene.test.ts: git ls-files + expect([]). + // An empty scan would also equal [], so scanned.length > 0 is the non-vacuous + // guard. generated[] is the committed JSON, not the script constant used alone. + const baseline = loadBaseline( + readFileSync(repoPath("tests/fixtures/file-size-baseline.json"), "utf8"), + ); + expect(baseline.generated).toEqual([...GENERATED_PATHS]); + + const scanned = scanRepo(repoRoot()); + expect(scanned.length).toBeGreaterThan(0); + expect(scanned.some((file) => file.path.startsWith("devlog/"))).toBe(false); + expect(scanned.some((file) => file.path === "bun.lock")).toBe(false); + + const rows = evaluate(scanned, baseline); + expect(rows.filter(isOffender)).toEqual([]); + expect( + rows.filter((row) => row.verdict === "GENERATED").map((row) => row.path).sort(), + ).toEqual([...GENERATED_PATHS].slice().sort()); + }); +}); +``` + +테스트는 `test()` 여섯 개다. 앞 다섯이 순수 단위(NEW_OVERSIZED, GREW, SHRANK, GENERATED, 스캔제외), 마지막이 저장소 스캔. SHRANK 케이스 안에 `updateBaseline`의 내리기·삭제·비시드·시드를 같이 둔다. 일곱 번째 `test()`를 만들지 마라. + +저장소 스캔의 GENERATED 경로 비교는 정렬 후 비교한다. JSON 시드가 상수 순서를 유지하면 정렬 없이도 통과하지만, 순서 drift를 스캔 실패로 위장하지 않기 위해서다. 경로 집합 자체는 `toEqual([...GENERATED_PATHS])`로 고정한다. + +## 완료 조건 (이 사이클 D) + +- write set 6개 파일이 L2 head에 있고, 이 문서 이외의 파일이 없다 +- 기준선 JSON의 `generated`가 위 12경로와 같고, `files`는 git ls-files 시드다 +- hosted CI에서 `file-size-ratchet.test.ts`와 layout 오라클이 그 head SHA로 녹색 +- 갓파일 줄 수가 이 PR에서 변하지 않는다 diff --git a/devlog/_plan/260914_godfile_round2/020_phase2_state_and_shim.md b/devlog/_plan/260914_godfile_round2/020_phase2_state_and_shim.md new file mode 100644 index 0000000000..687a084806 --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/020_phase2_state_and_shim.md @@ -0,0 +1,652 @@ +# 020 — 사이클 2: `src/responses/state.ts`와 `src/codex/shim.ts` 파사드 분해 + +`src/responses/state.ts` 2,432줄과 `src/codex/shim.ts` 2,466줄이 한 파일에 저장소·스필·스냅샷·리플레이와 심 설치·프로브·복원을 각각 들고 있어 래칫 이후에도 2,000줄을 넘긴다. 이 문서는 그 두 파일을 7개 PR로 줄이는 복붙 가능한 이동 계약이다. 구현자는 아래에 적힌 원본 행을 새 리프로 옮기고 파사드가 기존 export 이름을 그대로 다시보내며, 소비자 28+6곳은 import 경로를 건드리지 않는다. 기여자에게 바뀌는 것은 새 리프가 1,999줄 미만이어야 한다는 점과, 심 오라클 3건이 읽는 `src/codex/shim.ts` 본문에 지정 리터럴이 남아 있어야 한다는 점뿐이다. + +브랜치 `codex/m2k-l3-state-shim`, base는 사이클 1 래칫 브랜치 `codex/m2k-l2-ratchet`. 순수 이동, 동작 변경 없음. 로컬 스위트·typecheck·build는 이 단위 금지(hosted CI). 새 테스트 파일을 만들지 않는다. + +## 공통 이동 규칙 + +각 PR은 아래를 한 커밋으로 끝낸다. 원본 함수 본문을 고치지 않고 잘라 붙인다. 옮긴 함수는 파사드에서 삭제하고 `export { name } from "./…";` 한 줄로 다시보낸다. 내부 심볼은 파사드가 `import { name } from "./…";` 한다. 리프는 파사드를 import하지 않는다. `Date.now()`가 필요하면 파사드의 `now()`(`src/responses/state.ts:1236-1238`)를 import하지 말고 리프에서 `Date.now()`를 쓴다. + +모듈 수준 `let`/`const` 객체는 한 파일만 소유한다. `states`나 `spillCounters`를 인자로 넘겨 두 번째 참조를 만들지 않는다. 테스트 훅 setter는 소유 모듈에 두고 파사드가 기존 이름으로 다시보낸다. + +## 상태 소유권 + +### `src/responses/state.ts` + +| 바인딩 | 원본 행 | 소유 | 이유 | +|---|---|---|---| +| `states`, `storedResponseBytes`, `residentResponseBytes`, `oldestResidentId`, `oldestResidentAt`, `byteCapOverride` | 127-133 | 잔여 파사드 | ESM live binding. 이전 금지 | +| `stateRevision`, `lastSnapshotBytes`, `lastSnapshotDigest`, `lastSnapshotTarget` | 134-142 | 잔여 | 스냅샷 쓰기와 같은 파일 | +| `loaded`, `persistTimer`, `pendingPersistPath`, `persistGate`, `persistAttemptHookForTests` | 1229-1234 | 잔여 | `ensureLoaded`/`schedulePersist`와 같은 파일 | +| `replayOverlapSkips` | 1756 | 잔여 | `expandPreviousResponseInput:2154`가 증가, getter `:1835-1837`. 핑거프린트 리프로 옮기면 카운터가 갈라진다 | +| `replayScopeMismatchDrops` | 284 | 잔여 | `:2129`가 증가, `responseStateMetrics:2288`가 판독 | +| `pendingSpillUnlinks`, `PENDING_SPILL_UNLINKS_MAX` | 293-300 | 잔여 | `deleteEntry`/`replaceWithSpillFailure`/`drainPendingSpillUnlinks`가 잔여. 큐로 옮기면 순환 | +| `spillCounters`, `spillWriteHealth` | 172-214 | `spill-failure.ts` | 객체 변이. metrics는 import로 같은 객체를 판독 | +| `admissionCounters` | 283 | `spill-failure.ts` | 큐와 잔여가 필드만 증가. 객체를 인자로 넘기지 말 것 | +| `pendingResponseSpills`, `pendingResponseSpillById`, `pendingResponseSpillBytes` | 323-325 | `spill-queue.ts` | | +| `reservedResponseSpillBytes`, `unreclaimableSpillPaths`, `responseSpillPublicationTail` | 347, 361, 391 | `spill-queue.ts` | | +| 셧다운 예산 override 3개 | 392-394 | `spill-queue.ts` | 테스트 setter `:599-617`과 함께 | + +`responseStateMetrics`(`:2213-2289`)는 잔여에 남긴다. 이 함수를 별도 모듈로 빼면 `states`와 스필 카운터를 한곳에 다시 모아 순환이 생긴다. + +정정: 초안은 spill-failure 원본을 177-307로 적어 `spillCounters`(172-175)를 빠뜨리고 `pendingSpillUnlinks`(293-307)를 포함했다. 카운터는 172부터, unlink 큐는 잔여다. + +### `src/codex/shim.ts` + +| 바인딩 | 원본 행 | 소유 | 이유 | +|---|---|---|---| +| `lastShimDiscoveryError` | 선언 207, 기록 542·577·588·594, 판독 210·2181 | 잔여 | `findCodexOnPath`/`findWindowsCodexTargets`/`installCodexShimInternal`가 잔여. 탐색 분리 시 설치 메시지가 fallback으로 샌다 | +| `codexShimProbeHookForTests`, `codexShimProbeShellForTests`, `codexShimProbeObservationMs` | 806-807, 811 | `shim-probe.ts` | setter `:814-827`. 자식 오라클이 `setCodexShimProbeObservationMsForTests`를 `shim.ts`에서 import | +| guarded/fresh/rollback write 훅 3종 | 808-810, setter 829-850 | 잔여 | 설치/가드 리프레시 경로 | +| `guardedRefreshTransactionId` | 선언 1612, 사용 1850 | 잔여 | `applyGuardedRefreshTransaction`와 같은 파일 | + +정정: 프로브 훅 3종은 초안과 같다. 파일의 setter 6개 중 나머지 3개(write/rollback)는 잔여 소유다. + +## 하지 말아야 할 분할 + +1. `responseStateMetrics`를 별도 파일로 빼지 않는다. +2. store-core(`states`, `replaceMapEntry`, `deleteEntry`, `swapResidentForSpill`, `replaceWithSpillFailure`, `setResidentEntry`, `admitOversizedCandidate`)와 `spill-queue.ts`를 같은 PR에서 동시에 빼지 않는다. `runPendingResponseSpill:490,500`가 store-core를 호출하고 store-core 교체가 unlink 헬퍼를 호출한다. +3. `installCodexShimInternal`(`:2108-2293`)를 unix/windows로 나누지 않는다. 저널·롤백·probe가 한 함수다. +4. `findCodexOnPath`(`:541-585`)와 `findWindowsCodexTargets`(`:587-621`)를 리프로 옮기지 않는다. +5. `writeShim`(`:1427-1479`)를 리프로 옮기지 않는다. 오라클이 호출부 리터럴을 `shim.ts` 원문에서 찾는다. 빌더 정의만 옮긴다. +6. 소비자 import 경로를 리프로 바꾸지 않는다. + +## 동반 수정 의무 + +| 항목 | `state.ts` | `shim.ts` | +|---|---|---| +| structure 백틱 | 0. `structure/`가 `src/responses/state.ts`를 백틱하지 않음. 갱신 없음 | 2. `structure/runtime.md:41`, `structure/ops/docs-and-release.md:179`. 둘 다 `src/codex/shim.ts`. 파사드 파일명 유지 | +| 소스 오라클 | 0 | 3. 아래 오라클 절 | +| INV-* | 0. 이 파일을 묶는 INV 없음. 승계 모듈 없음 | 0 | +| `scripts/test-layout/layout.json` | 등록하지 않음 | 등록하지 않음 | +| `tests/fixtures/test-layout-expected.json` | 등록하지 않음 | 등록하지 않음 | +| `structure/manifest.json` / `INDEX.md` | 불필요. 리프는 `src/responses/` 중첩. 게이트는 `src/` 1-depth만 센다(`scripts/structure-ssot.ts:518-530`) | 불필요. 리프는 청구된 `src/codex/` 형제 | + +`structure/runtime.md:41`은 파사드 한 칸이다. PR 4부터 service.ts 행(`:42`)처럼 리프 백틱을 같은 칸에 나열한다. 없는 파일을 백틱하면 `structure:check`가 git index 기준으로 실패하므로 그 PR에서 만든 리프만 적는다. `ops/docs-and-release.md:179`는 파사드 파일명만 말하므로 본문 변경 없음. + +새 테스트 파일 금지: responses 도메인 regex는 `^(?:apply|chat|citation|continuation|eventstream|legacy|namespace|passthrough|responses|sse|thought|ws)-`이다. `spill-queue.test.ts`는 매칭되지 않아 explicit 등록이 강제된다. 기존 스위트가 순수 이동의 오라클이다. + +## 소스 오라클 (shim.ts 3건, 리터럴 재검증) + +`tests/codex-integration/codex-shim.test.ts:240-242`가 `readFileSync(repoPath("src", "codex", "shim.ts"), "utf8")` 후 아래 부분 문자열이 `shim.ts`에 있기를 요구한다. + + `\uFEFF${buildWindowsPowerShellCodexShim(realCodexPath, bun, cli, bunRuntimeSource)}` + +원본: `writeShim` 내부 `:1434` + + writeFileSync(wrapperPath, `\uFEFF${buildWindowsPowerShellCodexShim(realCodexPath, bun, cli, bunRuntimeSource)}`, "utf8"); + +`tests/codex-integration/codex-shim.test.ts:246-250`이 같은 파일에서 다음 세 리터럴을 찾는다. + +1. `const gitBashLauncher = join(dir, "codex");` → `findWindowsCodexTargets:608` +2. `for (const path of [cmd, ps1, gitBashLauncher])` → `findWindowsCodexTargets:610` +3. `buildUnixCodexShim(gitBashPath(realCodexPath), gitBashPath(bun), gitBashPath(cli), bunRuntimeSource, gitBashPath(serviceApiTokenFilePath()))` → `writeShim:1441` + +정정: 초안은 writeShim 호출부를 1434 한 줄로 적었다. 1434는 BOM 리터럴, Git-Bash 유닉스 빌더 호출은 1441이다. `writeShim` 함수는 1427-1479에 잔류한다. + +`tests/codex-integration/codex-shim.test.ts:1918-1922` — 자식이 `repoPath("src", "codex", "shim.ts")`를 `await import` 한다. 경로가 파사드여야 하고 파사드가 `autoRestoreCodexShim`와 `setCodexShimProbeObservationMsForTests`를 export해야 한다. 후자 정의는 프로브 리프로 옮겨도 `export { setCodexShimProbeObservationMsForTests } from "./shim-probe";`면 통과한다. + +## 소비자 (파사드 유지 시 write set 밖) + +정정: 초안 "src 임포터 38곳"은 과대. `responses/state`를 import하는 파일은 28곳이다. + +src 10: `src/cli/doctor.ts`, `src/lab/conformance/executor.ts`, `src/lib/app-owned-memory-stores.ts`, `src/lib/state-store-registrations.ts`, `src/server/lifecycle.ts`, `src/server/management/system-routes.ts`, `src/server/responses/collaboration.ts`, `src/server/responses/compact.ts`, `src/server/responses/core.ts`, `src/server/responses/encrypted-payload.ts`. scripts 1: `scripts/macos-rss-retention-sampler.ts`. tests/helpers 2 + tests 15. + +`src/adapters/kiro/stream.ts:895`는 주석만 있고 import가 아니다. + +shim 실임포트 src 6: `src/cli/codex-shim-autorestore.ts`, `src/cli/doctor.ts`, `src/cli/status.ts`, `src/client/machine-api.ts`, `src/remote-control/workspace-codex-sandbox.ts`, `src/server/startup-health-cache.ts`. + +--- + +## PR 1 — replay-fingerprint + temp-recovery + +브랜치 첫 커밋. `state.ts`만 줄인다. + +### NEW + +`src/responses/state/replay-fingerprint.ts` 예상 105줄 (import ~12 + 이동 79). + +원본에서 이동: + +- `:1752-1754` `REPLAY_FINGERPRINT_MAX_BYTES`, `REPLAY_FINGERPRINT_MAX_DEPTH` +- `:1758-1833` `replayItemFingerprint`, `providerIssuedIdentity`, `clientCarriedPrefixLength` (1758-1769 주석은 `replayItemFingerprint` 것) + +파사드에 남김: + +- `:1744-1750` `inputItems` — `expandPreviousResponseInput:2144,2167`와 `rememberResponseState:2350`가 사용 +- `:1756` `let replayOverlapSkips = 0;` +- `:1835-1837` `replayOverlapSkipsForTests` + +정정: 초안 ~115 (1744-1833)은 `inputItems`와 `replayOverlapSkips`를 포함했다. 실제 이동 본문은 79줄. + +`src/responses/state/temp-recovery.ts` 예상 270줄 (import ~20 + 이동 257). + +원본에서 이동: + +- `:70-84` `STALE_TEMP_GRACE_MS`, `STALE_TEMP_MAX_ENTRIES`, `STALE_TEMP_MAX_CLEANUPS`, `BOOT_FLOOR_SKEW_MS`, `PERIODIC_TEMP_MAX_ENTRIES`, `PERIODIC_TEMP_MAX_CLEANUPS`, `PERIODIC_TEMP_SCAN_DEADLINE_MS`, `RESPONSE_STATE_TEMP_NAME` +- `:1330-1527` `ResponseStateTempRecoveryResult`, `ResponseStateTempRecoveryIO`, `ResponseStateTempRecoveryOptions`, `processIsAlive`, `responseStateTempRecoveryIO`, `recoverStaleResponseStateTemps`, `responseStateSweepDirectories` +- `:1966-2009` `reclaimAbandonedResponseStateTemps`, `inspectAbandonedResponseStateTemps`, `sweepAbandonedResponseStateTemps` + +`responseStateSweepDirectories`의 `snapshotPath()` 호출은 `join(getConfigDir(), "responses-state.json")`로 인라인한다. 파사드의 `snapshotPath`를 import하지 않는다. `resolveWriteTarget`는 `../config`에서 가져온다. + +정정: 초안 ~200 (1330-1528)은 상수와 공개 래퍼를 빠뜨렸다. 래퍼를 파사드에 남기면 공개 API가 두 파일로 갈라진다. + +### MODIFY + +`src/responses/state.ts` — 위 행을 삭제하고 상단에 다음을 추가한다. + +```ts +import { clientCarriedPrefixLength } from "./state/replay-fingerprint"; +export type { ResponseStateTempRecoveryResult, ResponseStateTempRecoveryOptions } from "./state/temp-recovery"; +export { recoverStaleResponseStateTemps, reclaimAbandonedResponseStateTemps, inspectAbandonedResponseStateTemps, sweepAbandonedResponseStateTemps } from "./state/temp-recovery"; +``` + +`ensureLoaded:1546`의 `recoverStaleResponseStateTemps(dir)`는 재export된 이름을 그대로 쓴다. `expandPreviousResponseInput:2154`는 `clientCarriedPrefixLength(...)` 후 `replayOverlapSkips += 1`. 카운터는 이 파일에 남는다. + +예상 잔여 2,120줄 (2,432 − 79 − 15 − 198 − 44 + 글루 ~24). 아직 1,999 초과, 래칫은 감소이므로 통과. + +### DELETE + +없음. + +### write set + +- NEW `src/responses/state/replay-fingerprint.ts` +- NEW `src/responses/state/temp-recovery.ts` +- MODIFY `src/responses/state.ts` + +### 회귀 테스트 (hosted CI, 로컬 NOT RUN) + +- `tests/responses/continuation-dedup.test.ts` — `replayOverlapSkipsForTests` +- `tests/responses/responses-state.test.ts` — `recoverStaleResponseStateTemps` +- `tests/oauth/state-store-sweeper.test.ts` +- `tests/codex-integration/issue-702-expired-replay-state.test.ts` + +### 완료 조건 + +새 두 파일 각 ≤1,999. 파사드가 `replayOverlapSkips`를 소유. 리프가 `../state`를 import하지 않음. 소비자 diff 0. + +--- + +## PR 2 — spill-failure + snapshot-codec + +PR 1 위에 쌓는다. + +### NEW + +`src/responses/state/spill-failure.ts` 예상 145줄. + +원본에서 이동: + +- `:172-175` `spillCounters` +- `:177-214` `ResponseSpillWriteFailureCode`, `ResponseSpillWriteStatus`, `ResponseSpillWriteFailureOrigin`, `ResponseSpillWriteHealth`, `spillWriteHealth` +- `:216-275` `classifySpillWriteFailure`, `spillAclMemoRefusalOrigin`, `noteSpillWriteSuccess`, `noteSpillWriteFailure` +- `:283` `admissionCounters` +- `:287-288` `responseAdmissionCountersForTests` + +`noteSpillWriteSuccess:256` / `noteSpillWriteFailure:269`의 `now()`는 `Date.now()`로 바꾼다. 파사드를 import하지 않는다. + +파사드에 남김: + +- `:284` `let replayScopeMismatchDrops = 0;` +- `:293-307` `pendingSpillUnlinks`, `PENDING_SPILL_UNLINKS_MAX`, `MAX_PENDING_RESPONSE_SPILL_BYTES` + +파사드의 `responseStateMetrics:2273-2287`는 `import { spillCounters, spillWriteHealth } from "./state/spill-failure";` 후 기존 필드를 그대로 읽는다. 객체 identity가 하나라 변이가 metrics에 보인다. 잔여 `:1167,1188,1205,1562`의 `admissionCounters.* += 1`도 같은 import로 필드만 증가한다. 객체를 인자로 넘기지 않는다. + +파사드 재export: `ResponseSpillWriteFailureCode`, `ResponseSpillWriteStatus`, `ResponseSpillWriteFailureOrigin`, `responseAdmissionCountersForTests`. `ResponseStateMetrics` 인터페이스(`:2213-2233`)는 metrics 함수와 같이 잔여. + +`src/responses/state/snapshot-codec.ts` 예상 110줄. + +원본에서 이동: + +- `:1244-1251` `LegacySnapshotState` +- `:1253-1261` `isSpillRef` +- `:1263-1328` `loadSnapshotEntry` + +정정: 초안 ~110 (1244-1329) 대비 본문은 86줄. 파일 예상 110은 import+핸들 타입이다. + +`loadSnapshotEntry`는 지금 `replaceMapEntry`/`tombstone`/`measureResidentEntry`/`admitOversizedCandidate`/`byteCap`를 직접 호출한다. 리프가 파사드를 import할 수 없으므로 시그니처만 다음으로 바꾼다(본문 동작 동일). + +```ts +export interface SnapshotLoadStore { + replaceMapEntry(id: string, next: StoredResponseState, expected?: StoredResponseState): boolean; + tombstone(id: string, createdAt: number): SpillFailedResponseState; + measureResidentEntry(id: string, entry: ResidentInput): ResidentResponseState | null; + admitOversizedCandidate(id: string, expected: ResidentResponseState, previous: StoredResponseState | undefined): void; + byteCap(): number; +} +export function loadSnapshotEntry(id: string, value: unknown, store: SnapshotLoadStore): void { + // 원본 1263-1328 본문. states Map을 받지 않는다. +} +``` + +타입 `StoredResponseState` / `ResidentInput` / `ResidentResponseState` / `SpillFailedResponseState` / `SpilledResponseState`는 파사드 `:91-120`에 남긴다. 코덱은 `import type`만 한다. `import type`은 값 순환을 만들지 않는다. + +파사드 `ensureLoaded:1568` 호출을 `loadSnapshotEntry(entry[0], entry[1], { replaceMapEntry, tombstone, measureResidentEntry, admitOversizedCandidate, byteCap })`로 바꾼다. 핸들은 잔여 소유 함수의 참조이며 `states` 자체를 넘기지 않는다. + +### MODIFY + +`src/responses/state.ts` — 이동 행 삭제, import/재export 추가, `responseStateMetrics`와 admission 증가 지점이 `spill-failure`를 import, `clearResponseStateMemoryForTests:2402-2411`의 카운터 리셋이 import한 같은 객체의 필드를 0으로 만든다. + +예상 잔여 1,950줄 (PR1 잔여 2,120 − 4 − 99 − 1 − 2 − 86 + 글루 ~22). 이 PR 끝에서 `state.ts`가 1,999 이하가 된다. `structure/` 변경 없음. + +### DELETE + +없음. + +### write set + +- NEW `src/responses/state/spill-failure.ts` +- NEW `src/responses/state/snapshot-codec.ts` +- MODIFY `src/responses/state.ts` + +### 회귀 테스트 + +- `tests/responses/responses-state.test.ts` — spill write failure, tombstone, admission, snapshot round-trip (`:2001,2010,2240,2334,2383,2696`) +- `tests/responses/responses-state-write-amplification.test.ts` +- `tests/responses/continuation-dedup.test.ts` — metrics 키 집합 `:316-322` +- `tests/codex-integration/app-owned-memory.test.ts` — `MAX_STORED_RESPONSE_BYTES` (파사드 잔류) + +### 완료 조건 + +`responseStateMetrics`가 파사드에 남음. `spillCounters` identity 1개. `pendingSpillUnlinks`가 파사드에 남음. 코덱이 `states`를 인자로 받지 않음. + +--- + +## PR 3 — spill-queue + +PR 2 위에 쌓는다. store-core는 파사드에 남긴다. + +### NEW + +`src/responses/state/spill-queue.ts` 예상 620줄 (이동 559 + 핸들 타입 + import). + +원본에서 이동 `:309-867` 중 `deferSupersededSpill`를 제외한 전부: + +- `:309-321` `PendingResponseSpill` +- `:323-325` `pendingResponseSpills`, `pendingResponseSpillById`, `pendingResponseSpillBytes` +- `:347-394` `reservedResponseSpillBytes`, `unreclaimableSpillPaths`, `chargeUnreclaimableSpillPath`, `reconcileUnreclaimableSpillPaths`, `publicationFootprintBytes`, `responseSpillPublicationTail`, 셧다운 override 3개 +- `:404-867` `releasePendingResponseSpill`, `cancelPendingResponseSpill`, `isAclTimeout`, `spillPayloadForResident`, `runPendingResponseSpill`, `queuePendingResponseSpill`, `replaceWithPendingResponseSpill`, 테스트 export 5개(`:584-617`), 셧다운 fallback/drain + +`:396-402` `deferSupersededSpill`는 `pendingSpillUnlinks`(잔여)를 push한다. 이 함수는 파사드에 남기고 큐가 핸들로 호출한다. 초안 309-867을 통째로 옮기면 unlink 큐가 큐 모듈로 들어가 store-core와 순환한다. + +정정: 초안 ~560 (309-867) 범위는 맞지만 `deferSupersededSpill`는 잔여. 이동 본문은 약 552줄. + +큐 리프가 파사드를 import하지 않도록 모듈 로드 시점에 핸들만 주입한다. + +```ts +export interface SpillQueueStore { + swapResidentForSpill(id: string, expected: ResidentResponseState, ref: ResponseSpillRef): boolean; + replaceWithSpillFailure(id: string, candidate: ResidentResponseState, options?: { deferSpillUnlink?: boolean }): void; + deleteEntry(id: string, options?: { deleteSpill?: boolean }): void; + deferSupersededSpill(ref: ResponseSpillRef | undefined): void; +} +let store: SpillQueueStore | null = null; +export function bindSpillQueueStore(next: SpillQueueStore): void { + store = next; +} +function requireStore(): SpillQueueStore { + if (!store) throw new Error("spill-queue store is not bound"); + return store; +} +``` + +파사드는 store-core 함수가 정의된 다음 한 번 호출한다. + +```ts +import { bindSpillQueueStore } from "./state/spill-queue"; +bindSpillQueueStore({ swapResidentForSpill, replaceWithSpillFailure, deleteEntry, deferSupersededSpill }); +``` + +`runPendingResponseSpill:490,500` 등의 store-core 호출을 `requireStore().swapResidentForSpill(...)`로 치환한다. `noteSpillWriteSuccess`/`noteSpillWriteFailure`/`admissionCounters`는 `./spill-failure`에서 import한다. spill-store 심볼은 원본과 같이 `../spill-store`에서 import한다. + +파사드 재export: `flushPendingResponseSpillsForTests`, `awaitResponseSpillPublicationTailForTests`, `pendingResponseSpillMetricsForTests`, `setResponseSpillShutdownBudgetForTests`, `setResponseSpillAsyncAclAttemptBudgetForTests`, `setResponseSpillShutdownTerminalizationPassLimitForTests`. + +파사드 `clearResponseStateMemoryForTests:2393`의 `cancelPendingResponseSpill`와 `:2424-2425` `reservedResponseSpillBytes = 0` / `unreclaimableSpillPaths.clear()`는 큐 리프의 `resetSpillQueueForTests()` 한 함수로 모은다. 파사드 clear가 큐 모듈 바인딩을 필드 단위로 만지면 소유권이 샌다. + +파사드 `spilledResponseBytes:896`, `accountedResponseSpillBytes:917`는 `pendingSpillUnlinks`(잔여)와 `reservedResponseSpillBytes`(큐)를 함께 본다. 바이트 회계 함수는 잔여에 남기고, 큐는 getter를 제공한다. + +```ts +export function spillQueueAccounting(): { reservedBytes: number; jobOwnedBytes: number } { + // 원본 917-927과 동일 산식. states를 읽지 않는다. +} +``` + +잔여 `accountedResponseSpillBytes`가 이 getter를 더한다. + +### MODIFY + +`src/responses/state.ts` — `:309-867` 중 잔여 `deferSupersededSpill`만 남기고 삭제, bind 호출 추가, 테스트 export 재export, clear/accounting이 큐 getter를 사용. + +예상 잔여 1,370줄. + +### DELETE + +없음. + +### write set + +- NEW `src/responses/state/spill-queue.ts` +- MODIFY `src/responses/state.ts` + +### 회귀 테스트 + +- `tests/responses/responses-state.test.ts` — Windows ACL 큐, shutdown drain/fallback, pending unlink 128 cap (`:894,1240,1251,1285,1317,1357,1465,1514,1575,1643,1780,1791,2104,2413,2473`) +- `tests/helpers/responses-state-shutdown-budget-child.ts` +- `tests/helpers/responses-state-never-settling-acl-child.ts` + +자식 헬퍼는 계속 `from "../../src/responses/state"`를 import한다. 경로를 리프로 바꾸지 않는다. + +### 완료 조건 + +`state.ts` ≤1,999, `spill-queue.ts` ≤1,999. 큐가 `./state`를 import하지 않음. `bindSpillQueueStore` 1회. `states`를 인자로 넘기지 않음. store-core 함수가 파사드에 남음. + +이 PR로 `state.ts` 분해는 끝이다. 이후 PR은 `shim.ts`만 만진다. + +--- + +## PR 4 — shim-templates + +`shim.ts` 첫 분해. 오라클 리터럴이 있는 호출부는 옮기지 않는다. + +### NEW + +`src/codex/shim-templates.ts` 예상 270줄. + +원본에서 이동: + +- `:38-39` `SHIM_MARKER`, `UNIX_SHIM_REVISION_MARKER` +- `:46-47` `CODEX_SHIM_REENTRY_EXIT_CODE`, `CODEX_SHIM_REENTRY_DIAGNOSTIC` +- `:212-235` `CODEX_INTERNAL_COMMANDS` +- `:237-249` `CODEX_GLOBAL_OPTIONS_WITH_VALUE` +- `:674-685` `shQuote` +- `:687-767` `buildUnixCodexShim` (export) +- `:1050-1161` `windowsBatchValue`, `windowsBatchSet`, `buildWindowsCodexShim` (export), `psString`, `buildWindowsPowerShellCodexShim` (export) +- `:1414-1416` `gitBashPath` — inspect(`:1390`)와 writeShim(`:1441`)가 공유. 템플릿 리프에 둔다 + +정정: 초안 ~390은 `:48-204` `CODEX_SHIM_INSTALL_PROBE_SCRIPT`(157줄)를 템플릿에 넣은 합산이다. 그 스크립트는 `probeUnixShimInstall:888`만 쓰므로 PR 6 프로브 리프로 간다. 템플릿 본문은 ~251줄. + +`CODEX_SHIM_INSTALL_PROBE_SCRIPT`는 이 PR에서 옮기지 않는다. + +### MODIFY + +`src/codex/shim.ts` + +```ts +import { SHIM_MARKER, UNIX_SHIM_REVISION_MARKER, CODEX_SHIM_REENTRY_EXIT_CODE, CODEX_SHIM_REENTRY_DIAGNOSTIC, shQuote, windowsBatchSet, psString, gitBashPath } from "./shim-templates"; +export { buildUnixCodexShim, buildWindowsCodexShim, buildWindowsPowerShellCodexShim } from "./shim-templates"; +``` + +`writeShim:1434`와 `:1441` 호출부 텍스트를 한 글자도 바꾸지 않는다. 빌더 이름이 같은 스코프에 남아야 오라클이 통과한다(재export가 같은 바인딩을 제공한다). + +`isShim:331`, `isHealthyShim:339`는 잔여. `SHIM_MARKER`를 템플릿에서 import. + +`structure/runtime.md:41` MODIFY. 기존 칸의 파사드 백틱을 지우지 말고, 이 PR에서 만든 리프만 같은 칸에 추가한다. + + | `src/codex/shim.ts` | Codex autostart shim facade. Wrapper templates live in `src/codex/shim-templates.ts`. It skips startup for management subcommands even when value-taking global flags precede the subcommand, and transactionally restores complete, stable external launcher replacements without a watcher or PATH rediscovery. | + +`structure/ops/docs-and-release.md:179` 변경 없음. + +예상 잔여 2,235줄. + +### DELETE + +없음. + +### write set + +- NEW `src/codex/shim-templates.ts` +- MODIFY `src/codex/shim.ts` +- MODIFY `structure/runtime.md` (41행만) + +### 회귀 테스트 + +- `tests/codex-integration/codex-shim.test.ts` — 빌더 출력, 오라클 `:239-251`, management command skip, BOM, Git-Bash launcher +- `tests/codex-integration/codex-cli-install-provenance.test.ts` — `buildUnixCodexShim` +- `tests/adapters/openai/openai-provider-option-tooling.test.ts` — `buildUnixCodexShim` + +### 완료 조건 + +`:1434` BOM 리터럴과 `:1441` Git-Bash 호출 리터럴이 `shim.ts`에 존재. `findWindowsCodexTargets:608,610` 미이동. runtime.md가 `src/codex/shim.ts`를 백틱. + +--- + +## PR 5 — shim-fingerprint + shim-state-file + +### NEW + +`src/codex/shim-fingerprint.ts` 예상 210줄. + +원본에서 이동: + +- `:40` `CODEX_SHIM_PROBE_BYTES` +- `:289-303` `ShimPathFingerprint`, `StableShimPathProbe` +- `:351-515` `readShimProbePrefix`, `statFingerprint`, `sameFingerprint`, `sameFingerprintAfterRename`, `stableShimPathProbe`, `sameStableShimPathProbe`, `shimPathFingerprint`, `restoreWithoutReplacing`, `isHealthyShimProbe`, `isCurrentUnixShimProbe`, `hasUsableBackingPath` +- `:641-661` `isVersionManagerOwnedCodexPath` (export) — inspect(PR 7)가 파사드를 import할 수 없으므로 경로 판별을 여기 둔다 + +정정: 초안 ~165 (351-515)은 인터페이스와 version-manager 판별을 빠뜨렸다. `:351-515` 165줄은 맞고, 파일 총량은 ~210. + +`isShim`/`isHealthyShim`(`:331-349`)는 잔여. 전체 파일을 읽는 설치 경로용이다. 프로브 prefix 판별만 리프. + +`src/codex/shim-state-file.ts` 예상 140줄. + +원본에서 이동: + +- `:42` `CODEX_SHIM_STATE_MAX_BYTES` (export) +- `:251-265` `ShimState`, `ShimFileState` — 여러 리프가 쓰므로 상태 파일 모듈이 타입 소유 +- `:1163-1260` `ShimStateReadResult`, `fileErrorCode`, `readBoundedRegularFile`, `readStateResult`, `readState` +- `:1402-1412` `statePath`, `writeState` +- `:1522-1527` `stateFiles` — inspect와 설치가 공유. 파사드에 남기면 inspect가 파사드를 import한다 + +`fileErrorCode`는 잔여 롤백(`:1016,1816`)·restore-lock(`:1752`)·inspect(`:1300`)가 쓴다. 이 리프가 소유하고 나머지가 import한다. + +`primaryState:1528-1532`는 설치 경로, 잔여. + +정정: 초안 ~130은 read 블록+writeState와 비슷하다. 타입·`stateFiles`를 포함하면 ~140. + +### MODIFY + +`src/codex/shim.ts` — 이동 행 삭제. + +```ts +import { type ShimPathFingerprint, type StableShimPathProbe, statFingerprint, sameFingerprint, stableShimPathProbe, shimPathFingerprint, restoreWithoutReplacing, isHealthyShimProbe, isCurrentUnixShimProbe, hasUsableBackingPath } from "./shim-fingerprint"; +export { isVersionManagerOwnedCodexPath } from "./shim-fingerprint"; +import { type ShimState, type ShimFileState, fileErrorCode, readStateResult, readState, statePath, writeState, stateFiles } from "./shim-state-file"; +export { CODEX_SHIM_STATE_MAX_BYTES } from "./shim-state-file"; +``` + +`structure/runtime.md:41` 칸에 `src/codex/shim-fingerprint.ts`, `src/codex/shim-state-file.ts` 백틱을 추가한다. + +예상 잔여 1,940줄. 이 PR 끝에서 `shim.ts`가 1,999 이하. + +### DELETE + +없음. + +### write set + +- NEW `src/codex/shim-fingerprint.ts` +- NEW `src/codex/shim-state-file.ts` +- MODIFY `src/codex/shim.ts` +- MODIFY `structure/runtime.md` (41행) + +### 회귀 테스트 + +- `tests/codex-integration/codex-shim.test.ts` — fingerprint mismatch defer, version-manager 분류 `:2258`, stale lock 전 관측 구간 `:2040,2098,2119` +- `tests/codex-integration/codex-shim-autorestore.test.ts` — `CODEX_SHIM_STATE_MAX_BYTES` + +### 완료 조건 + +`findWindowsCodexTargets`/`writeShim` 잔류. 오라클 리터럴 잔류. 리프가 `./shim`을 import하지 않음. + +--- + +## PR 6 — shim-probe + shim-restore-lock + +두 모듈은 서로 import하지 않는다. 한 PR에 넣는 이유는 스택 길이다. + +### NEW + +`src/codex/shim-probe.ts` 예상 390줄. + +원본에서 이동: + +- `:44-45` `CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS`, `CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS` +- `:48-204` `CODEX_SHIM_INSTALL_PROBE_SCRIPT` +- `:769-805` `UnixShimProbeCleanupPhase`, `UnixShimProbeCleanup`, `UnixShimProbeResult`, `SHIM_PROBE_ERROR_CODES`, `SHIM_PROBE_SIGNALS`, `shimProbeCleanup` +- `:806-807` `codexShimProbeHookForTests`, `codexShimProbeShellForTests` +- `:811` `codexShimProbeObservationMs` +- `:814-827` `setCodexShimProbeHookForTests`, `setCodexShimProbeShellForTests`, `setCodexShimProbeObservationMsForTests` (export) +- `:852-981` `readProbeMetadata`, `probeUnixShimInstall`, `probeUnixShimFiles`, `unixProcessGroupAlive`, `terminateUnixProcessGroup` + +파사드에 남김 (설치 경로 훅): + +- `:808-810` guarded/fresh/rollback write 훅 바인딩 +- `:829-850` 그 setter 3개 + +프로브는 템플릿에서 `CODEX_SHIM_REENTRY_EXIT_CODE`, `CODEX_SHIM_REENTRY_DIAGNOSTIC`를 import한다. `MAX_DIAGNOSTIC_VALUE_BYTES`(`:206`)는 잔여 `lastShimDiscoveryError` truncate(`findCodexOnPath:577`)와 프로브 stderr cap이 공유한다. 상수 한 줄을 프로브 리프가 소유하고 파사드가 import한다. 복제하지 않는다. + +정정: 초안 ~215는 `:769-981`(213줄)만이다. 스크립트 157줄을 더하면 ~370 + import ≈ 390. + +`src/codex/shim-restore-lock.ts` 예상 175줄. + +원본에서 이동: + +- `:43` `CODEX_SHIM_RESTORE_LOCK_STALE_MS` +- `:1614-1758` `ShimRestoreLock`, `ShimRestoreLockRecord`, `ShimRestoreLockSnapshot`, `restoreLockPath`, `sameFileIdentity`, `readShimRestoreLockSnapshot`, `sameShimRestoreLock`, `reclaimStaleRestoreLock`, `tryAcquireShimRestoreLock` + +`:1612` `let guardedRefreshTransactionId = 0;`는 이동하지 않는다. `:1760`부터의 `planGuardedRefreshTransaction` / `applyGuardedRefreshTransaction`가 잔여에서 `++guardedRefreshTransactionId`(`:1850`)를 쓴다. + +restore-lock은 fingerprint에서 `stableShimPathProbe`, `sameFingerprint`, `ShimPathFingerprint`를, state-file에서 `fileErrorCode`를, `../lib/process-control`에서 `isProcessAlive`를 import한다. 파사드를 import하지 않는다. + +정정: 초안 ~165 (1614-1759) → 본문 145줄(1614-1758). 파일 예상 175. + +### MODIFY + +`src/codex/shim.ts` + +```ts +export { setCodexShimProbeHookForTests, setCodexShimProbeShellForTests, setCodexShimProbeObservationMsForTests } from "./shim-probe"; +import { probeUnixShimFiles } from "./shim-probe"; +import { tryAcquireShimRestoreLock, reclaimStaleRestoreLock } from "./shim-restore-lock"; +``` + +자식 오라클 `:1918`이 `setCodexShimProbeObservationMsForTests`를 `shim.ts`에서 가져오므로 재export가 필수다. + +`structure/runtime.md:41` 칸에 두 리프 백틱을 추가한다. + +예상 잔여 1,470줄. + +### DELETE + +없음. + +### write set + +- NEW `src/codex/shim-probe.ts` +- NEW `src/codex/shim-restore-lock.ts` +- MODIFY `src/codex/shim.ts` +- MODIFY `structure/runtime.md` (41행) + +### 회귀 테스트 + +- `tests/codex-integration/codex-shim.test.ts` — Unix install probe (`:335,417,483,517,617,895,927`), restore lock (`:1897,1985,2016`), 자식 import `:1918` +- `tests/codex-integration/codex-shim-autorestore.test.ts` + +### 완료 조건 + +write 훅 3종이 파사드에 남음. `guardedRefreshTransactionId` 파사드. `lastShimDiscoveryError` 파사드. 프로브 리프가 `./shim`을 import하지 않음. + +--- + +## PR 7 — shim-inspect + +마지막. 설치 본체는 여전히 파사드. + +### NEW + +`src/codex/shim-inspect.ts` 예상 185줄. + +원본에서 이동: + +- `:267-287` `CodexShimBackingForCommand` (export type) +- `:1262-1401` `isLocalAbsoluteInspectionPath` (export), `windowsShimInspectionIsDeferred`, `inspectCodexShimBackingForCommand` (export) + +inspect는 다음만 import한다. `./shim` 금지. + +- `./shim-state-file`: `readStateResult`, `fileErrorCode`, `stateFiles` +- `./shim-fingerprint`: `shimPathFingerprint`, `stableShimPathProbe`, `statFingerprint`, `isHealthyShimProbe`, `isVersionManagerOwnedCodexPath` +- `./shim-templates`: `shQuote`, `windowsBatchSet`, `psString`, `gitBashPath` + +정정: 초안 ~150 (1262-1401)은 타입 21줄을 빠뜨렸다. 본문 140 + 타입 21 + import ≈ 185. + +### MODIFY + +`src/codex/shim.ts` + +```ts +export type { CodexShimBackingForCommand } from "./shim-inspect"; +export { isLocalAbsoluteInspectionPath, inspectCodexShimBackingForCommand } from "./shim-inspect"; +``` + +`structure/runtime.md:41` 칸에 `src/codex/shim-inspect.ts`를 추가하고 칸을 마친다. 최종 칸이 백틱해야 할 경로: + +- `src/codex/shim.ts` (파사드, 기존) +- `src/codex/shim-templates.ts` +- `src/codex/shim-fingerprint.ts` +- `src/codex/shim-state-file.ts` +- `src/codex/shim-probe.ts` +- `src/codex/shim-restore-lock.ts` +- `src/codex/shim-inspect.ts` + +예상 잔여 1,320줄. `installCodexShimInternal:2108-2293`, `findCodexOnPath:541-585`, `findWindowsCodexTargets:587-621`, `writeShim:1427-1479`, `autoRestoreCodexShim`, `uninstallCodexShim`, `diagnoseCodexShim`는 전부 이 파일에 남는다. + +### DELETE + +없음. + +### write set + +- NEW `src/codex/shim-inspect.ts` +- MODIFY `src/codex/shim.ts` +- MODIFY `structure/runtime.md` (41행) + +### 회귀 테스트 + +- `tests/codex-integration/codex-shim.test.ts` — `:2287` local inspection paths, `:2304` Windows backing inspection fail-closed +- `src/remote-control/workspace-codex-sandbox.ts` 소비자는 계속 `from "../codex/shim"` (write set 밖, diff 0) + +### 완료 조건 + +`shim.ts` ≤1,999, 모든 리프 ≤1,999. 오라클 3건의 리터럴과 동적 import 경로가 `src/codex/shim.ts`를 가리킴. `findCodexOnPath`/`findWindowsCodexTargets`/`writeShim`/`installCodexShimInternal` 잔류. 소비자 diff 0. + +--- + +## 사이클 2 종료 시 파일 크기 + +| 파일 | 원본 줄 | 예상 최종 | 비고 | +|---|---|---|---| +| `src/responses/state.ts` | 2,432 | ~1,370 | 파사드+store-core+persist+metrics+replay 카운터 | +| `src/responses/state/replay-fingerprint.ts` | — | ~105 | | +| `src/responses/state/temp-recovery.ts` | — | ~270 | | +| `src/responses/state/spill-failure.ts` | — | ~145 | | +| `src/responses/state/snapshot-codec.ts` | — | ~110 | | +| `src/responses/state/spill-queue.ts` | — | ~620 | | +| `src/codex/shim.ts` | 2,466 | ~1,320 | 파사드+발견+writeShim+설치 본체 | +| `src/codex/shim-templates.ts` | — | ~270 | | +| `src/codex/shim-fingerprint.ts` | — | ~210 | | +| `src/codex/shim-state-file.ts` | — | ~140 | | +| `src/codex/shim-probe.ts` | — | ~390 | | +| `src/codex/shim-restore-lock.ts` | — | ~175 | | +| `src/codex/shim-inspect.ts` | — | ~185 | | + +합이 원본보다 ~400줄 많은 것은 파일 헤더·import·핸들 타입이다. 각 파일 1,999 미만이면 래칫 통과. 기준선 회수(`ratchet:update`)는 사이클 D에서 하며 이 7개 PR의 write set에 넣지 않는다. + +## 파사드가 다시보내야 하는 기존 export (누락 금지) + +`state.ts` 공개 이름. 리프로 정의가 옮겨도 파사드 이름이 그대로여야 한다: `MAX_STORED_RESPONSE_BYTES`, `MAX_SPILLED_RESPONSE_BYTES`, `PreviousResponseReplayFailure`, `ResponseSpillWriteFailureCode`, `ResponseSpillWriteStatus`, `ResponseSpillWriteFailureOrigin`, `responseAdmissionCountersForTests`, `flushPendingResponseSpillsForTests`, `awaitResponseSpillPublicationTailForTests`, `pendingResponseSpillMetricsForTests`, `setResponseSpillShutdownBudgetForTests`, `setResponseSpillAsyncAclAttemptBudgetForTests`, `setResponseSpillShutdownTerminalizationPassLimitForTests`, `setResponseStateByteCapForTests`, `getStoredResponseBytesForTests`, `setSpilledResponseByteCapForTests`, `getSpilledResponseBytesForTests`, `getAccountedResponseSpillBytesForTests`, `ResponseStateTempRecoveryResult`, `ResponseStateTempRecoveryOptions`, `recoverStaleResponseStateTemps`, `flushResponseState`, `replayOverlapSkipsForTests`, `sweepExpiredResponseStates`, `reclaimAbandonedResponseStateTemps`, `inspectAbandonedResponseStateTemps`, `sweepAbandonedResponseStateTemps`, `responseContinuationRetainedStoreSnapshot`, `evictOldestResponseContinuationForBudget`, `expandPreviousResponseInput`, `previousResponseReplayFailure`, `previousResponseReplayPrefixLength`, `copyPreviousResponseReplayProvenance`, `previousResponseScopeMismatch`, `previousResponseConversationId`, `previousResponseProviderState`, `ResponseStateMetrics`, `responseStateMetrics`, `markBodyNonPersistable`, `rememberResponseState`, `setResponseStatePersistAttemptHookForTests`, `runPendingResponseStatePersistForTests`, `responseStatePersistPendingForTests`, `clearResponseStateMemoryForTests`, `clearResponseStateForTests`. + +`shim.ts` 공개 이름: `CODEX_SHIM_REPLACEMENT_STABLE_MS`, `CODEX_SHIM_STATE_MAX_BYTES`, `lastCodexDiscoveryError`, `CodexPathScanDeps`, `findCodexOnPath`, `isWindowsInteropDir`, `isVersionManagerOwnedCodexPath`, `buildUnixCodexShim`, `setCodexShimProbeHookForTests`, `setCodexShimProbeShellForTests`, `setCodexShimProbeObservationMsForTests`, `setCodexShimGuardedWriteHookForTests`, `setCodexShimFreshWriteHookForTests`, `setCodexShimRollbackRestoreHookForTests`, `buildWindowsCodexShim`, `buildWindowsPowerShellCodexShim`, `isLocalAbsoluteInspectionPath`, `inspectCodexShimBackingForCommand`, `CodexShimBackingForCommand`, `CodexShimAutoRestoreResult`, `installCodexShim`, `autoRestoreCodexShim`, `uninstallCodexShim`, `isCodexShimInstalled`, `CodexShimDiagnostic`, `diagnoseCodexShim`, `codexShimStatus`. + +이 이름을 리네임하거나 소비자 import 경로를 바꾸면 이 사이클의 계약 위반이다. diff --git a/devlog/_plan/260914_godfile_round2/030_phase3_inject_and_catalog_sync.md b/devlog/_plan/260914_godfile_round2/030_phase3_inject_and_catalog_sync.md new file mode 100644 index 0000000000..dfeb9f334d --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/030_phase3_inject_and_catalog_sync.md @@ -0,0 +1,661 @@ +# 260914 godfile round2 — 사이클 3: inject.ts / catalog/sync.ts + +inject.ts 2,342줄과 catalog/sync.ts 2,698줄이 2,000줄 래칫을 넘고, TOML 루트 키 불변식·서브에이전트 5칸 창·히스토리 인라인 호출·카탈로그 mtime 계약이 한 파일에 섞여 있다. 이 문서는 기술 의존 순서로 나눈 7개 PR의 원본 행 범위, 예상 줄 수, write set, 재수출, 오라클 패치, structure 동반 수정을 복붙 실행 가능하게 고정한다. 실행자는 이 순서대로만 옮기고, 소비자 import 경로는 facade가 유지하므로 바뀌지 않으며, 본문을 텍스트로 읽는 테스트와 structure 백틱만 새 소유 모듈을 가리키게 바뀐다. + +기준 트리: 작업 디렉터리 `/Users/jun/.codex/worktrees/5880/opencodex`, 브랜치 `codex/m2k-l1-roadmap`, `origin/dev` `4f788f916e`. 열린 PR 충돌은 순서에서 제외한다. 로컬 install/typecheck/test는 하지 않는다. 검증은 hosted CI. + +`000_plan.md`는 사이클 3 봉투 브랜치를 `codex/m2k-l4-inject-sync`(base L3) 하나로 그린다. 아래 7개 PR은 그 봉투의 실행 분할이다. PR1 base는 L3, PR7 head가 사이클 3 tip이며 사이클 4(`codex/m2k-l5-routing-quota`)는 PR7 head를 base로 한다. 부모 레인이 단일 L4 PR을 고집하면 7 커밋을 그 브랜치에 쌓고 PR은 하나만 연다. write set은 달라지지 않는다. + +순수 이동. 동작 변경 금지. 원본 경로 facade 재수출 필수. + +## 정정 (초안 대비, 이 트리에서 재계측) + +정정: `inject/routing-target.ts` 초안 173–271 ~150줄은 실제 99줄이다. `standaloneCodexRoutingTarget`(216)가 `providerBaseHost`(272)를 호출하므로 173–271만 옮기면 컴파일되지 않는다. 범위는 173–288(116줄)로 확장한다. + +정정: `inject/config-toml.ts` 초안 96–135+272–513+620–894 ~620줄은 실제 557줄이다. `providerBaseHost`를 routing-target로 넘기면 96–135+289–513+620–894 = 540줄이 원본이다. 새 파일은 import를 더해 예상 610–650줄. + +정정: `inject/routing-classify.ts` 514–619는 106줄이다 (초안 ~110). + +정정: `inject/restore.ts` 초안 1832–2342 ~510줄(실제 511)은 `formatApplyHistoryFailure`(2324–2342, apply 문구)와 `getCodexConfigPath`(2314–2316, facade 잔여)를 포함한다. restore 본체는 1832–2312 = 481줄. apply가 restore를 import하면 방향이 뒤집힌다. + +정정: inject 잔여 초안 ~780은 실제 909줄이다 (1–95 import/재수출 + 136–172 `InjectCodexOptions` + 895–948 결과/훅 + 949–1671 impl). `injectCodexConfigImpl` 949–1671 = 723줄은 맞다. + +정정: 소스 오라클 `codex-retained-root-serialization.test.ts` 본문 슬라이스는 323행이 아니라 324행이다. 323은 `readFileSync`, 324가 `source.slice("const owningCodexHome" … "// Design B")`. 첫 `const owningCodexHome`는 2048, 그 이후 첫 `// Design B`는 2273. + +정정: inject structure 백틱은 7곳이 아니라 8참조/7파일이다. `config.md`가 75와 277 두 번 등장한다. + +정정: sync 잔여 초안 461–1393 ~700줄은 실제 933줄이다. + +정정: roster 91–259 = 169줄 (초안 ~175). derive-entry 260–460 = 201줄 (초안 ~200). auto-review 1596–2094 = 499줄 (초안 ~500). gated-native-warn 2095–2152 = 58줄 (초안 ~60). retained-sync 1394–1595+2153–2427+2476–2563 = 565줄 (초안 ~610). catalog restore 초안 2428–2475+2564–2698 ~190줄은 실제 183줄이며, 그 안에 `invalidateCodexModelsCache*`(2636–2698, 63줄)가 들어 있다. 이건 restore가 아니라 cache writer다. restore.ts는 2428–2475+2564–2635 = 120줄, invalidate는 retained-sync로 간다. + +정정: `effort.ts:42` `deriveEntry` import는 호출 사이트가 0이다 (123행 주석만). 경로만 `./derive-entry`로 바꾸면 `effort ↔ derive-entry` 순환이 남는다. 호출이 없으므로 해당 import 줄을 삭제한다. + +정정: retained-sync를 별 파일로 빼면서 build/merge(461–1393)를 `sync.ts`에 남기면 retained-sync가 `./sync`를 역참조해 순환한다. PR7에서 build/merge도 `catalog/build-entries.ts`로 같이 빼고 `sync.ts`는 facade만 남긴다. "build/merge는 마지막"은 앞 PR에서 빼지 말라는 뜻이지, 마지막 PR에서 순환을 만들라는 뜻이 아니다. + +정정: `INLINE_ALLOWED`의 `codex/inject.ts`(37행)는 `syncCodexHistoryProvider` 호출이 2287(`restoreNativeCodex`)에만 있다. 이동 후 facade에는 호출이 없다. `codex/inject/restore.ts`를 넣고 `codex/inject.ts`는 뺀다. + +## 현재 파일 해부 + +### src/codex/inject.ts 2,342줄 — NEW 디렉터리 src/codex/inject/ + +| 새 파일 | NEW/MODIFY | 원본 행 (inclusive) | 원본 줄 수 | 예상 줄 수 | 가져갈 심볼 | +|---|---|---|---:|---:|---| +| inject/routing-target.ts | NEW | 173-288 | 116 | 155 | CodexRoutingTarget, validateCodexRoutingTarget, usesProviderTable, standaloneCodexRoutingTarget, routingTargetOrigin, configuredManagedSubagentDefaults, providerBaseHost | +| inject/config-toml.ts | NEW | 96-135, 289-513, 620-894 | 540 | 630 | externalCodexModelProvider, currentExternalCodexModelProvider, dominantEol, applyEol, buildProviderTableBlock 오버로드, buildOpenaiBaseUrlLine 오버로드, buildRealtimeWsBaseUrlLine, setRootOpenaiBaseUrl 오버로드, setRootRealtimeWsBaseUrl, stripInjectedOpenaiBaseUrl, stripExistingModelProvider, stripRootContextWindowOverrides, stripRootRoutedModel, setRootModelProvider, readRootModelCatalogPath, setRootModelCatalogPath, removeProfileSection, normalizeServiceTier, ensureFastModeFeature, isOpencodexCatalogPath, stripOpencodexCatalogPath, buildProfileFile 오버로드, chooseCatalogPathForInjection | +| inject/routing-classify.ts | NEW | 514-619 | 106 | 145 | CodexRoutingKind, RoutingEndpointKind, ipv4Octets, classifyRoutingEndpoint, classifyCodexRouting, isCodexRoutingInjected, getCodexRoutingKind | +| inject/remove.ts | NEW | 1672-1831 | 160 | 210 | isOcxProviderHeaderLine, hasOcxProviderTable, removeOcxSection, StripOpencodexConfigResult, stripOpencodexConfigResult, stripOpencodexConfig, hasOpencodexRouting, removeCodexConfig | +| inject/restore.ts | NEW | 1832-2312 | 481 | 560 | restore 타입 4종, failedHistoryRestore*, externalProviderRestoreResult, foreignOwnershipRestoreRefusal, desiredEnabledRestoreSkip, skippedRestoreEnvelope, failedConfigRestoreEnvelope, restoreCodexConfigInline*, restoreCodexCatalogArtifact, restoreNativeCodexAsync*, restoreNativeCodex | +| inject.ts 잔여 | MODIFY | 1-95, 136-172, 895-1671, 2314-2342 | 909 + 재수출 ~40 | 960 | InjectCodexOptions, runClientWriteGuard, CodexInjectResult, historyArtifactStageForTests, beforeHistoryArtifactCommitForTests, injectCodexConfig, injectCodexConfigImpl (723줄, 쪼개지 않음), getCodexConfigPath, formatApplyHistoryFailure | + +inject/ 는 src/codex/ 아래라 structure/manifest.json 신규 area가 아니다. + +### src/codex/catalog/sync.ts 2,698줄 — 기존 디렉터리 src/codex/catalog/ + +| 새 파일 | NEW/MODIFY | 원본 행 (inclusive) | 원본 줄 수 | 예상 줄 수 | 가져갈 심볼 | +|---|---|---|---:|---:|---| +| catalog/subagent-roster.ts | NEW | 91-259 | 169 | 220 | MAX_SPAWN_AGENT_MODEL_OVERRIDES, PICKER_ORDER_PRIORITY_BASE, SPAWN_PRIORITY_FIELD, CATALOG_INACTIVE_REASON_FIELD, SpawnAgentSurface, SubagentRosterExclusion*, EffectiveSubagent*, isEligibleV2SubagentEntry, configuredCatalogEntry, configuredSubagentModelMatchesEntry, effectiveSubagentRoster | +| catalog/derive-entry.ts | NEW | 260-460 | 201 | 270 | finishUpstreamNativeEntry, isExactComboCatalogModel, isExactComboCatalogEntry, routedDisplayName, preservePinnedNativeCustomReasoning, deriveEntry | +| catalog/auto-review.ts | NEW | 1596-2094 | 499 | 560 | AUTO_REVIEW_ROOT_MARKER부터 finalizeAutoReviewModelOverride까지 스탬프/플랜/override 전부 | +| catalog/gated-native-warn.ts | NEW | 2095-2152 | 58 | 95 | gatedNativeReauthSuppressionReason, gatedNativeAccountLabel, warnedGatedNativeSuppression, resetGatedNativeSuppressionWarningsForTests, warnGatedNativeSuppressedOnce | +| catalog/retained-sync.ts | NEW | 1394-1595, 2153-2427, 2476-2563, 2636-2698 | 628 | 720 | retained read/revalidate/write, CodexCatalogSyncOptions, syncCatalogModels, invalidateCodexModelsCache* | +| catalog/restore.ts | NEW | 2428-2475, 2564-2635 | 120 | 170 | visibleAccountReplacementNatives, restoreAccountHiddenBareNatives, currentDisabledModelsForRestore, restoreCodexCatalogWithPermit, restoreCodexCatalog | +| catalog/build-entries.ts | NEW | 461-1393 | 933 | 1020 | ObservedCatalogEntryBuildInput, buildCatalogEntries*, resetCatalogRuntimeStateForTests, orderForSubagents, orderForModelPicker, merge/recovery 전부 | +| catalog/sync.ts facade | MODIFY | 1-90 정리 후 재수출만 | 90 -> ~80 | 80 | 모든 공개 심볼 재수출. 본문 함수 0 | +| catalog.ts 14줄 facade | 유지 | 변경 없음 | 14 | 14 | 계속 from "./catalog/sync" | +| catalog/effort.ts | MODIFY | 42행 1줄 삭제 | 560 -> 559 | 559 | unused deriveEntry import 삭제 | + +## 상태 소유권 (인자로 새면 안 되는 것) + +한 바인딩은 한 모듈. 테스트 훅 setter는 그 모듈에 두고 facade가 재수출한다. 자식 프로세스가 require("./src/codex/inject")로 setter를 잡는다 (codex-inject-integration.test.ts:174,217,269). facade 재수출이 빠지면 훅은 침묵한다. + +| 바인딩 | 현재 행 | 소유 모듈 | facade 재수출 | 비고 | +|---|---|---|---|---| +| historyArtifactStageForTests | 924 | inject.ts 잔여 | setHistoryArtifactStageForTests | applyNativeArtifacts(1349)가 호출. impl과 같이 잔여 | +| beforeHistoryArtifactCommitForTests | 932 | inject.ts 잔여 | setBeforeHistoryArtifactCommitForTests | 같은 클로저 | +| beforeRestoreConfigForTests | 928 | inject/restore.ts | setBeforeRestoreConfigForTests | restoreCodexConfigInlineImpl:2002가 호출. setter도 restore.ts로 이동한 뒤 facade가 export { setBeforeRestoreConfigForTests } from "./inject/restore" | +| warnedGatedNativeSuppression | 2130 | catalog/gated-native-warn.ts | resetGatedNativeSuppressionWarningsForTests | resetCatalogRuntimeStateForTests가 이 Set을 지우지 않는다. 두 리셋 경로를 합치지 말 것. 현재 테스트 호출 사이트는 0이어도 public seam이므로 재수출 유지 | +| aggregation/provider-fetch/bundled/model-cache 리셋 집합 | 726 | catalog/build-entries.ts | resetCatalogRuntimeStateForTests | 타 모듈 상태를 모아서 지운다. gated-native Set은 여기 넣지 않음 | + +인자로 새는 상태 금지: warnedGatedNativeSuppression을 함수 인자나 반환값으로 넘기지 않는다. permit(CatalogWritePermit)은 인자로 받는 것이 계약이다 (writeRetainedCatalogSync, restoreCodexCatalogWithPermit, invalidateCodexModelsCacheWithPermit). 추출 모듈 안에서 withCatalogWriteSerialization을 다시 호출해 permit을 재취득하지 않는다. invalidateCodexModelsCacheWithPermit:2639 주석이 말하는 재취득은 이미 있는 wrapper 동작이다. 새로 만들지 말 것. + +read.catalog는 in-place 변형이다. writeRetainedCatalogSync:2274가 catalog[RESERVE_SOURCE_CATALOG_FIELD]를 쓰고, :2363이 catalog.models = mergeCatalogEntriesFromObservedState(...)를 대입한다. 이 catalog는 revalidateRetainedCatalogSync:1556이 JSON clone한 객체다. read/revalidate/write를 모듈로 쪼개면 clone 타이밍이 어긋나 디스크에 부분 merge가 커밋된다. 세 함수는 retained-sync.ts에 고정. + +## 하지 말아야 할 분할 + +1. applyNativeArtifacts만 별 모듈 금지. 1349-1419 클로저가 자체 preImages(1352) + catch 보상(1389)을 갖고, 협조 경로 1491-1505가 그 바깥에서 다시 captureCodexPreImages/restoreCodexPreImages를 돈다. 함수만 빼면 이중 보상이 되거나, 바깥 보상이 안 잡힌 쓰기를 남긴다. 723줄 impl은 이 사이클에서 쪼개지 않는다. + +2. removeCodexConfig를 restore에 흡수 금지. restoreCodexConfigInlineImpl:2034가 removeCodexConfig({ preserveProfile })를 호출하는 것은 의존이지 합병이 아니다. removeCodexConfig는 CLI/저널 테스트의 public API다. 흡수하면 journal-fallback과 명시적 remove가 한 envelope를 공유해 preserveProfile와 artifact 보고가 섞인다. remove.ts와 restore.ts는 두 파일. restore가 remove를 import한다. + +3. writeRetainedCatalogSync를 빌드/커밋으로 분리 금지. :2400-2415가 바이트 동일 rewrite를 건너뛰어 mtime을 보존한다 (#857 app-server 신선도, #1407 이후 stale이면 모델 가이드가 침묵). 빌드와 커밋을 나누면 동일 바이트 판정이 빌드 쪽 복사본을 보거나, 커밋 쪽이 항상 write한다. + +4. build/merge를 PR1-6에서 빼지 말 것. PR7에서 build-entries.ts와 retained-sync.ts를 형제로 같이 뺀다. retained-sync가 ./sync를 import하면 순환이다. + +5. derive-entry.ts가 ./sync를 import하지 말 것. 절단점이다. roster 상수(SPAWN_PRIORITY_FIELD, CATALOG_INACTIVE_REASON_FIELD)는 ./subagent-roster에서 가져온다. effort 심볼은 ./effort에서 가져온다. + +## INV 승계 + +- INV-TOML-01 structure/overview.md:86-88 -> 테스트 바인딩 tests/codex-integration/codex-inject.test.ts:1 유지. 실질 소스 승계 모듈은 src/codex/inject/config-toml.ts. 파일 첫 줄에 테스트와 같은 id 주석을 넣는다. 테스트 주석은 삭제하지 않는다. +- INV-AGENT-01 structure/overview.md:92-94 -> 테스트 바인딩 tests/codex-integration/catalog-full-picker-order.test.ts:1 유지. 실질 소스 승계 모듈은 src/codex/catalog/subagent-roster.ts (MAX_SPAWN_AGENT_MODEL_OVERRIDES와 effectiveSubagentRoster). 헤더 id 주석 이관, 테스트 주석 유지. + +## 소스 오라클 (본문을 텍스트로 읽음 — 경로를 반드시 고침) + +1. tests/codex-integration/codex-retained-root-serialization.test.ts:323-326 + - 지금: readFileSync(.../src/codex/inject.ts) 후 const owningCodexHome ~ // Design B 슬라이스가 withCatalogWriteSerialization(owningCodexHome와 restoreCodexCatalogWithPermit를 포함하는지 본다. + - 이동 후 슬라이스 전체가 inject/restore.ts (restoreCodexCatalogArtifact:2048 ~ restoreNativeCodex:2273). + - 패치: readFileSync 대상을 src/codex/inject/restore.ts로 바꾼다. concat 불필요. + +2. tests/codex-integration/codex-inject-history-wording.test.ts:11,118-123 + - 지금: injectSource = readFileSync(src/codex/inject.ts). + - 리터럴 6종: 118 changed: rawHistory.rows > 0 || rawHistory.files > 0 -> restore.ts (restoreNativeCodex:2297). 119 restored original provider metadata for ${migratedRows} manifest-backed thread(s) -> 잔여 impl 1597 (apply). 120 original providers preserved -> restore.ts 2228과 2301. 121 No backed-up resume-history metadata was pending; untracked routed history was left unchanged. -> restore.ts 2229과 2302. 122-123 not.toContain 두 금지어는 두 파일 모두. + - 패치: const injectSource = readFileSync(repoPath("src/codex/inject.ts"), "utf8") + readFileSync(repoPath("src/codex/inject/restore.ts"), "utf8"); + - import 경로 failedHistoryRestoreFromOutcome, formatApplyHistoryFailure는 facade 유지. + +3. tests/codex-integration/codex-history-reachability.test.ts:35-39 + - 지금 INLINE_ALLOWED에 codex/inject.ts. + - 패치: codex/inject.ts를 빼고 codex/inject/restore.ts를 넣는다. 인라인 호출은 restoreNativeCodex:2287 한 곳. + +4. tests/providers/xai/grok-writer-boundary.test.ts:26 + - 주석만. 전 src/ walk라 자식 모듈이 grokHome+config.toml+write를 동시에 가지지 않는 한 통과. 코드 변경 없음. 주석의 codex/inject.ts는 facade 설명으로 남겨도 된다. + +추가 경로 주석 (기계 오라클은 아님, 행번호가 깨지므로 같은 PR에서 고친다): + +- tests/routing/routing-capability-catalog.test.ts:44 sync.ts:321-322 -> derive-entry.ts의 deriveEntry 본문. 행번호를 새 파일 기준으로 고치거나 행번호를 삭제한다. +- tests/providers/cursor/cursor-display-names.test.ts:10 routedDisplayName (codex/catalog/sync.ts) -> codex/catalog/derive-entry.ts. + +## structure 동반 수정 (같은 PR, 나중 정리 금지) + +structure/AGENTS.md: "Changing an area obliges the same change to update every doc listed for it." src/codex/ 소유 문서는 INDEX 표 그대로다. 신규 top-level area 없음. manifest.json 수정 없음. bun run structure:index 불필요. + +공개 API를 말하는 문장은 facade 경로를 유지한다. 소유 모듈이 바뀐 문장만 백틱을 갈아끼운다. + +inject.ts를 가리키는 8참조/7파일 — 히스토리 writer 문단은 facade+restore를 함께 적는다. 문장 골격은 유지하고 경로만 다음으로 교체한다. + +| 파일:줄 | 지금 백틱 | 변경 | +|---|---|---| +| structure/config.md:75 | src/codex/inject.ts writes one of two forms | 유지 (공개 inject 동작). 구현 소유를 쓰려면 inject.ts(impl) + inject/config-toml.ts(루트 키 배치)를 병기 | +| structure/config.md:277 | 히스토리 writer 문단 inject.ts | src/codex/inject.ts와 src/codex/inject/restore.ts 병기 | +| structure/runtime.md:371 | 동일 문단 | 동일 병기 | +| structure/catalog.md:325 | 동일 문단 | 동일 병기 | +| structure/subagents.md:346 | 동일 문단 | 동일 병기 | +| structure/gui-and-management-api.md:576 | 동일 문단 | 동일 병기 | +| structure/ops/docs-and-release.md:354 | 동일 문단 | 동일 병기 | +| structure/providers/openai-tiers.md:455 | 동일 문단 | 동일 병기 | + +sync.ts 2곳 — 소유가 옮겨졌으므로 경로를 교체한다. + +| 파일:줄 | 지금 | 변경 | +|---|---|---| +| structure/subagents.md:71 | MAX_SPAWN_AGENT_MODEL_OVERRIDES = 5 (mirrored in src/codex/catalog/sync.ts) | src/codex/catalog/subagent-roster.ts (sync.ts facade 재수출) | +| structure/catalog.md:343 | src/codex/catalog/sync.ts resolves exact case-preserving provider/model reviewer selectors | src/codex/catalog/auto-review.ts (retained sync와 convergence.ts가 facade를 통해 호출) | + +히스토리 writer 문단의 병기 문장 템플릿 (7파일에 동일 치환): + +src/codex/history-provider.ts refuses external writes to paginated or migration-capable history. src/codex/inject.ts (apply impl) and src/codex/inject/restore.ts check affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensate detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. + +## layout.json + +새 테스트 파일 없음. scripts/test-layout/layout.json explicit와 tests/fixtures/test-layout-expected.json에 등록하지 않는다. 기존 오라클 파일은 도메인 유지. + +## 공통 재수출 규칙 + +소비자는 계속 다음만 import한다. + +- src/codex/inject +- src/codex/catalog/sync +- src/codex/catalog (14줄, sync 재수출 유지) + +src/grok/inject.ts:5의 applyEol, dominantEol, providerBaseHost도 facade를 유지한다. 테스트 from "../../src/codex/inject" / require("./src/codex/inject") / require("./src/codex/inject.ts") 를 새 자식 경로로 바꾸지 않는다. 예외는 위에 적은 본문-슬라이스 오라클 세 파일뿐이다. + +순환 금지 그래프: + +inject/routing-target.ts -> loopback-target, config(subagentDefaultSyncEffective), types +inject/routing-classify.ts -> injected-marker, paths (inject.ts 금지) +inject/config-toml.ts -> routing-target, injected-marker, paths, context-compat (inject.ts 금지) +inject/remove.ts -> config-toml, injected-marker, journal, history-provider (restore 금지) +inject/restore.ts -> remove, config-toml, catalog/sync facade, journal, history-* +inject.ts 잔여 -> 위 전부 + apply impl + +catalog/subagent-roster.ts -> parsing/metadata/account-models/slug-codec (sync 금지) +catalog/derive-entry.ts -> roster, effort, parsing, metadata, identity (sync 금지) +catalog/effort.ts -> deriveEntry import 삭제. sync/derive-entry 금지 +catalog/auto-review.ts -> parsing, provider-validation (sync 금지) +catalog/gated-native-warn.ts -> entitlements, account-label (sync 금지) +catalog/build-entries.ts -> derive-entry, roster, effort, parsing, metadata, features (sync·retained-sync 금지) +catalog/retained-sync.ts -> build-entries, auto-review, gated-native-warn, derive-entry, catalog-writer (sync 금지) +catalog/restore.ts -> catalog-writer, parsing, metadata (sync·retained-sync 금지, permit은 인자) +catalog/sync.ts -> 위 모듈 re-export only +catalog.ts -> ./catalog/sync 유지 + +--- + +## PR 1 — inject routing-target + +브랜치: codex/m2k-l4-01-inject-routing-target. base: L3 (codex/m2k-l3-state-shim). 제목: refactor(codex): extract inject routing-target leaf + +Write set: + +- NEW src/codex/inject/routing-target.ts (원본 173-288, 예상 155줄) +- MODIFY src/codex/inject.ts (해당 블록 삭제, 아래 재수출 추가) +- tests/structure 수정 없음 (공개 경로 불변) + +원본에서 잘라 붙일 블록: export interface CodexRoutingTarget (173)부터 providerBaseHost 함수 닫는 중괄호 (288)까지. 바로 위 Design B 주석(128-134)은 InjectCodexOptions용이므로 잔여에 둔다. + +routing-target.ts 상단 import (이 집합만): + + import { subagentDefaultSyncEffective } from "../../config"; + import type { OcxConfig } from "../../types"; + import { type ManagedSubagentDefaults } from "../subagent-defaults"; + import { effectiveLoopbackListenerPort, isLoopbackHostname, shouldInjectApiAuthHeader } from "../loopback-target"; + +configuredManagedSubagentDefaults는 이 범위에 들어 있으나 impl만 쓴다. 같이 옮기고 잔여가 import한다. transformManagedSubagentDefaults 값 import가 이 함수에 없으면 type-only로 둔다. 원본 247-271을 그대로 옮겨 컴파일되면 그 형태를 유지한다. + +inject.ts에 추가할 public 재수출 (원본이 export하던 것만): + + export { + standaloneCodexRoutingTarget, + providerBaseHost, + type CodexRoutingTarget, + } from "./inject/routing-target"; + +잔여는 같은 모듈에서 validateCodexRoutingTarget, usesProviderTable, routingTargetOrigin, configuredManagedSubagentDefaults를 로컬 import한다. 이 넷은 원본 non-export 유지. + +회귀: tests/codex-integration/codex-inject.test.ts (standalone byte-compat 26행부터). tests/server/loopback-companion-client-targets.test.ts. hosted CI. 로컬 NOT RUN. + +완료 조건: wc -l src/codex/inject.ts < 2342, 새 파일 < 1999, from "./inject/routing-target" 외 새 공개 경로 0. + +--- + +## PR 2 — inject config-toml + routing-classify + +브랜치: codex/m2k-l4-02-inject-toml-classify. base: PR1. 제목: refactor(codex): extract inject TOML transforms and routing classify + +기술 의존: config-toml이 PR1의 CodexRoutingTarget / providerBaseHost / validateCodexRoutingTarget / usesProviderTable / routingTargetOrigin을 import한다. classify는 PR1과 독립이나 같은 PR에 묶어 래칫에 새 파일을 통과시킨다. + +Write set: + +- NEW src/codex/inject/config-toml.ts (원본 96-135 + 289-513 + 620-894, 원본 540줄, 예상 630) +- NEW src/codex/inject/routing-classify.ts (원본 514-619, 원본 106줄, 예상 145) +- MODIFY src/codex/inject.ts +- MODIFY src/codex/inject/config-toml.ts 헤더에 INV-TOML-01 주석 (NEW 파일의 첫 줄) +- MODIFY structure/config.md:75 — 루트 키 배치 소유를 inject/config-toml.ts로 병기 +- 테스트 파일 수정 없음 (INV 테스트 바인딩 유지) + +config-toml.ts 첫 줄: + + // Holds INV-TOML-01 from structure/overview.md; keep the id here if this file is split or renamed. + +세 원본 조각을 이 순서로 붙인다: 96-135 (provider/EOL) -> 289-513 (table/base_url) -> 620-894 (root keys/profile/catalog path). 조각 사이에 빈 줄 하나. 함수 본문 바이트 불변. + +classify는 514-619를 그대로. import는 injected-marker, paths, node:fs만. inject.ts 금지. + +inject.ts 재수출에 추가할 public 이름 (원본 export만): + + export { + externalCodexModelProvider, + currentExternalCodexModelProvider, + dominantEol, + applyEol, + buildProviderTableBlock, + buildOpenaiBaseUrlLine, + buildRealtimeWsBaseUrlLine, + setRootOpenaiBaseUrl, + setRootRealtimeWsBaseUrl, + stripInjectedOpenaiBaseUrl, + stripRootContextWindowOverrides, + buildProfileFile, + chooseCatalogPathForInjection, + } from "./inject/config-toml"; + export { + classifyCodexRouting, + isCodexRoutingInjected, + getCodexRoutingKind, + type CodexRoutingKind, + } from "./inject/routing-classify"; + +오버로드 시그니처(buildProviderTableBlock 289-315, buildOpenaiBaseUrlLine 342-355, setRootOpenaiBaseUrl 380-427, buildProfileFile 821-845)를 빠짐없이 옮긴다. 구현 함수(*ForTarget)는 non-export 유지. + +회귀: tests/codex-integration/codex-inject.test.ts 전체 (INV-TOML-01). tests/service/autostart-health.test.ts (classifyCodexRouting). tests/server/loopback-listener-admission.test.ts (buildProviderTableBlock). + +함정: setRootOpenaiBaseUrl는 루트 키를 첫 테이블 앞에 넣는다. 이 함수가 INV-TOML-01의 실체다. 프로파일 섹션 append로 바꾸지 말 것. + +--- + +## PR 3 — inject remove + +브랜치: codex/m2k-l4-03-inject-remove. base: PR2. 제목: refactor(codex): extract inject remove/strip primitives + +Write set: + +- NEW src/codex/inject/remove.ts (원본 1672-1831, 160줄, 예상 210) +- MODIFY src/codex/inject.ts + +remove.ts가 config-toml에서 import할 심볼: dominantEol, applyEol, stripInjectedOpenaiBaseUrl, removeProfileSection, stripRootRoutedModel, stripOpencodexCatalogPath. stripOpencodexConfigResult가 추가로 쓰는 것은 transformManagedSubagentDefaults + journal/marker. + +재수출: + + export { stripOpencodexConfig, removeCodexConfig } from "./inject/remove"; + +회귀: tests/codex-integration/codex-inject.test.ts (stripOpencodexConfig). tests/codex-integration/codex-journal.test.ts (removeCodexConfig require). tests/codex-integration/codex-inject-integration.test.ts remove 분기. + +함정: remove를 이 PR에서 restore와 합치지 않는다. restore는 다음 PR. + +--- + +## PR 4 — inject restore + +브랜치: codex/m2k-l4-04-inject-restore. base: PR3. 제목: refactor(codex): extract inject native restore + +Write set: + +- NEW src/codex/inject/restore.ts (원본 1832-2312, 481줄, 예상 560) +- MODIFY src/codex/inject.ts — restore 블록 삭제, 훅 beforeRestoreConfigForTests 이동, formatApplyHistoryFailure(2324-2342)와 getCodexConfigPath(2314-2316) 잔여 유지 +- MODIFY tests/codex-integration/codex-retained-root-serialization.test.ts:323-326 경로를 src/codex/inject/restore.ts +- MODIFY tests/codex-integration/codex-inject-history-wording.test.ts:11 concat +- MODIFY tests/codex-integration/codex-history-reachability.test.ts:35-39 INLINE_ALLOWED +- MODIFY structure 히스토리 문단 7파일 병기 (config.md:277, runtime.md:371, catalog.md:325, subagents.md:346, gui-and-management-api.md:576, ops/docs-and-release.md:354, providers/openai-tiers.md:455) + +beforeRestoreConfigForTests let + setter(928-930)를 restore.ts로 옮긴다. 잔여의 924-926, 932-934 훅 두 개는 impl과 함께 남는다. facade: + + export { + failedHistoryRestoreFromOutcome, + skippedRestoreEnvelope, + restoreNativeCodexAsync, + restoreNativeCodex, + setBeforeRestoreConfigForTests, + type CodexRestoreArtifactState, + type CodexRestoreConfigResult, + type CodexRestoreCatalogResult, + type CodexRestoreHistoryResult, + type CodexNativeRestoreResult, + } from "./inject/restore"; + +restore.ts는 removeCodexConfig를 ./remove에서, currentExternalCodexModelProvider를 ./config-toml에서, restoreCodexCatalogWithPermit를 ../catalog/sync에서 가져온다. 아직 catalog restore 추출 전이다. facade 경로는 이후 PR7에서도 유지. + +오라클 패치 원문. + +codex-retained-root-serialization.test.ts:323 부근을 다음으로 교체한다: + + const source = readFileSync(join(repoRoot, "src/codex/inject/restore.ts"), "utf8"); + const restoreRoot = source.slice(source.indexOf("const owningCodexHome"), source.indexOf("// Design B", source.indexOf("const owningCodexHome"))); + expect(restoreRoot).toContain("withCatalogWriteSerialization(owningCodexHome"); + expect(restoreRoot).toContain("restoreCodexCatalogWithPermit"); + +슬라이스 문자열이 파일에 그대로 있는지는 이동 후 확인한다. 2048-2273이 한 파일에 남아 있어야 한다. + +codex-inject-history-wording.test.ts:11: + + const injectSource = + readFileSync(repoPath("src/codex/inject.ts"), "utf8") + + readFileSync(repoPath("src/codex/inject/restore.ts"), "utf8"); + +codex-history-reachability.test.ts:35-39: + + const INLINE_ALLOWED = new Set([ + "codex/history-provider.ts", + "codex/inject/restore.ts", + "codex/internal/history-writer.ts", + ]); + +회귀: 위 오라클 3파일 + codex-inject-integration.test.ts (require setter) + codex-journal.test.ts restore + codex-restore-app-rewrite.test.ts. + +이 PR 후 wc -l src/codex/inject.ts 목표는 잔여 909 + 재수출 ≈ 960 < 1999. 자식 5파일 모두 < 1999. + +--- + +## PR 5 — catalog roster + derive-entry (순환 절단) + +브랜치: codex/m2k-l4-05-catalog-roster-derive. base: PR4. 제목: refactor(catalog): extract subagent roster and deriveEntry + +roster를 같은 PR에서 먼저 붙인다. derive-entry가 SPAWN_PRIORITY_FIELD, CATALOG_INACTIVE_REASON_FIELD를 roster에서 가져간다. + +Write set: + +- NEW src/codex/catalog/subagent-roster.ts (91-259, 169줄, 예상 220). 첫 줄 INV-AGENT-01 주석 +- NEW src/codex/catalog/derive-entry.ts (260-460, 201줄, 예상 270) +- MODIFY src/codex/catalog/sync.ts — 해당 블록 삭제, 재수출 추가 +- MODIFY src/codex/catalog/effort.ts:42 — import { deriveEntry } from "./sync"; 줄 삭제. 123행 주석은 문구 유지 +- MODIFY structure/subagents.md:71 경로를 src/codex/catalog/subagent-roster.ts +- MODIFY tests/routing/routing-capability-catalog.test.ts:44 행번호 주석 +- MODIFY tests/providers/cursor/cursor-display-names.test.ts:10 모듈 경로 + +catalog.ts:11-13은 그대로 from "./catalog/sync". sync facade가 roster/derive를 재수출하면 된다. + +subagent-roster.ts 첫 줄: + + // Holds INV-AGENT-01 from structure/overview.md; keep the id here if this file is split or renamed. + +sync.ts 재수출: + + export { + MAX_SPAWN_AGENT_MODEL_OVERRIDES, + PICKER_ORDER_PRIORITY_BASE, + SPAWN_PRIORITY_FIELD, + CATALOG_INACTIVE_REASON_FIELD, + isEligibleV2SubagentEntry, + configuredCatalogEntry, + effectiveSubagentRoster, + type SpawnAgentSurface, + type SubagentRosterExclusionReason, + type EffectiveSubagentModel, + type SubagentRosterExclusion, + type EffectiveSubagentRoster, + } from "./subagent-roster"; + export { + finishUpstreamNativeEntry, + isExactComboCatalogModel, + deriveEntry, + } from "./derive-entry"; + +잔여 sync(아직 build/merge가 여기 있음)는 deriveEntry와 roster 상수를 새 파일에서 import한다. 이 시점의 그래프는 sync -> derive-entry -> effort, sync -> roster, effort는 sync를 보지 않음. 순환 없음. + +derive-entry.ts가 가져야 할 import (원본 deriveEntry 본문이 실제로 쓰는 것만, 원본 1-90에서 복사 후 미사용은 삭제): + +- ./parsing (applyCatalogMetadata, applyRoutedCodexToolMode, ensureStrictCatalogFields, normalizeServiceTiers, normalizeRoutedCatalogEntry, types) +- ./metadata (applyNativeOpenAiContextOverride, hasNativeOpenAiCapabilityMetadata, upstreamNativeEntry, CODEX_CUSTOM_MODEL_CATALOG_KIND) +- ./effort (applyReasoningLevels, applyCatalogModelMetadata, isGpt56NativeSlug, ensureGpt56ReasoningLevels, ensureUltraReasoningLevel) +- ./subagent-roster (SPAWN_PRIORITY_FIELD, CATALOG_INACTIVE_REASON_FIELD) +- ../../adapters/identity (identifyRoutedModel) +- ../../providers/default-aliases (effectiveProviderAlias) — routedDisplayName용 +- ../../combos (COMBO_NAMESPACE) +- types CatalogModel, RawEntry, OcxConfig, NativeContextLimitsInput + +회귀: tests/codex-integration/catalog-full-picker-order.test.ts (INV-AGENT-01, deriveEntry import는 계속 catalog/sync). catalog-go-exact-efforts.test.ts. catalog-zero-credit-picker.test.ts. catalog-free-pricing-status.test.ts. codex-catalog.test.ts 중 derive/roster 구간. + +함정: catalog.ts가 derive-entry를 직접 가리키게 바꾸지 말 것. 이중 facade 계약은 catalog.ts + sync.ts 둘 다 재수출. + +--- + +## PR 6 — auto-review + gated-native-warn + +브랜치: codex/m2k-l4-06-catalog-review-warn. base: PR5. 제목: refactor(catalog): extract auto-review override and gated-native warn-once + +Write set: + +- NEW src/codex/catalog/auto-review.ts (1596-2094, 499줄, 예상 560) +- NEW src/codex/catalog/gated-native-warn.ts (2095-2152, 58줄, 예상 95) +- MODIFY src/codex/catalog/sync.ts +- MODIFY structure/catalog.md:343 경로를 src/codex/catalog/auto-review.ts + +상태: warnedGatedNativeSuppression Set은 gated-native-warn 소유. resetGatedNativeSuppressionWarningsForTests도 그 파일. sync 잔여의 resetCatalogRuntimeStateForTests에 .clear()를 추가하지 않는다. + +이 시점에 writeRetainedCatalogSync는 아직 sync.ts에 있다. 잔여가 finalizeAutoReviewModelOverride와 warnGatedNativeSuppressedOnce / gatedNativeReauthSuppressionReason / gatedNativeAccountLabel를 새 파일에서 import한다. + +재수출 (sync + 따라서 catalog.ts 경유 가능): + + export { + isValidAutoReviewModel, + applyAutoReviewModelOverride, + applyConfiguredAutoReviewModelOverride, + finalizeAutoReviewModelOverride, + type AutoReviewModelOverrideResult, + } from "./auto-review"; + export { + gatedNativeReauthSuppressionReason, + resetGatedNativeSuppressionWarningsForTests, + } from "./gated-native-warn"; + +회귀: tests/codex-integration/codex-catalog.test.ts auto-review require 구간 7260-7332. tests/codex-integration/catalog-gated-native-suppression-reason.test.ts (import는 catalog/sync 유지). convergence.ts는 facade 유지. + +함정: isValidAutoReviewModel는 sync.ts:1616이 provider-validation 심볼을 감싼 re-export다. 이 래퍼를 auto-review로 옮기고 sync가 다시 재수출한다. config/provider-validation.ts:258 원본을 삭제하지 말 것. + +--- + +## PR 7 — retained-sync + catalog restore + build-entries (마지막, 순환 절단) + +브랜치: codex/m2k-l4-07-catalog-retained-restore (000_plan.md의 codex/m2k-l4-inject-sync tip). base: PR6. 제목: refactor(catalog): extract retained sync, restore, and build-entries + +이 PR이 build/merge를 마지막으로 뺀다. 세 파일을 한 커밋에 twin으로 만들어 sync.ts를 facade로 남긴다. 나눠서 올리면 중간 커밋이 retained-sync -> sync -> retained-sync 순환을 갖는다. + +Write set: + +- NEW src/codex/catalog/build-entries.ts (461-1393, 933줄, 예상 1020) +- NEW src/codex/catalog/retained-sync.ts (1394-1595 + 2153-2427 + 2476-2563 + 2636-2698, 628줄, 예상 720) +- NEW src/codex/catalog/restore.ts (2428-2475 + 2564-2635, 120줄, 예상 170) +- MODIFY src/codex/catalog/sync.ts — 본문 함수 전부 제거, 1-90 import를 재수출 블록으로 교체. 목표 <= 80줄 +- catalog.ts 변경 없음 + +retained-sync.ts 조각 순서: 1394-1595 (read/revalidate/evidence) -> 2153-2427 (writeRetainedCatalogSync, mtime 가드 포함) -> 2476-2563 (syncCatalogModels) -> 2636-2698 (invalidate cache). 한 모듈. + +restore.ts 조각 순서: 2428-2475 (visibility helpers) -> 2564-2635 (restoreCodexCatalogWithPermit, restoreCodexCatalog). + +writeRetainedCatalogSync 통째 이동. 2400-2415 mtime 주석과 onDiskBytes.equals 분기를 분리하지 말 것. catalog.models = in-place 대입(2363)과 reserve 필드 변이(2274)도 같은 함수 안에 둔다. + +permit: writeRetainedCatalogSync와 invalidateCodexModelsCacheWithPermit와 restoreCodexCatalogWithPermit는 받은 permit만 replaceActiveCodexCatalog / replaceCodexModelsCache에 넘긴다. 모듈 내부에서 withCatalogWriteSerialization을 여는 것은 기존 wrapper (syncCatalogModels:2523, restoreCodexCatalog:2626, invalidateCodexModelsCache:2691)만. 그 wrapper는 각자 원래 있던 파일로 따라간다 — syncCatalogModels/invalidate는 retained-sync, restoreCodexCatalog는 restore.ts. + +sync.ts facade 최종 형태 (원본 public export와 1:1인지 추출 직후 rg '^export ' src/codex/catalog/sync.ts로 대조): + + export { + MAX_SPAWN_AGENT_MODEL_OVERRIDES, + PICKER_ORDER_PRIORITY_BASE, + SPAWN_PRIORITY_FIELD, + CATALOG_INACTIVE_REASON_FIELD, + isEligibleV2SubagentEntry, + configuredCatalogEntry, + effectiveSubagentRoster, + } from "./subagent-roster"; + export type { + SpawnAgentSurface, + SubagentRosterExclusionReason, + EffectiveSubagentModel, + SubagentRosterExclusion, + EffectiveSubagentRoster, + } from "./subagent-roster"; + export { finishUpstreamNativeEntry, isExactComboCatalogModel, deriveEntry } from "./derive-entry"; + export { + buildCatalogEntries, + buildCatalogEntriesFromObservedState, + resetCatalogRuntimeStateForTests, + orderForSubagents, + orderForModelPicker, + mergeCatalogModelsWithNativeRecovery, + applyFullModelPickerOrder, + mergeCatalogEntriesFromObservedState, + mergeCatalogEntriesForSync, + CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, + } from "./build-entries"; + export type { ObservedCatalogEntryBuildInput, ObservedCatalogMergeInput, ObservedCatalogMergePolicy } from "./build-entries"; + export { + isValidAutoReviewModel, + applyAutoReviewModelOverride, + applyConfiguredAutoReviewModelOverride, + finalizeAutoReviewModelOverride, + } from "./auto-review"; + export type { AutoReviewModelOverrideResult } from "./auto-review"; + export { + gatedNativeReauthSuppressionReason, + resetGatedNativeSuppressionWarningsForTests, + } from "./gated-native-warn"; + export { + syncCatalogModels, + invalidateCodexModelsCache, + invalidateCodexModelsCacheWithPermit, + } from "./retained-sync"; + export type { CodexCatalogSyncOptions } from "./retained-sync"; + export { restoreCodexCatalog, restoreCodexCatalogWithPermit } from "./restore"; + +빠지면 catalog.ts와 convergence/remote/inject가 깨진다. 특히 invalidateCodexModelsCacheWithPermit (catalog/remote.ts:10), mergeCatalogModelsWithNativeRecovery (convergence.ts), buildCatalogEntriesFromObservedState, CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, applyFullModelPickerOrder, SPAWN_PRIORITY_FIELD. + +회귀: + +- tests/codex-integration/codex-retained-root-serialization.test.ts (syncCatalogModels dynamic import 경로 ./src/codex/catalog/sync.ts 유지) +- tests/codex-integration/codex-models-cache-invalidate.test.ts +- tests/codex-integration/catalog-full-picker-order.test.ts +- tests/codex-integration/codex-catalog.test.ts +- tests/codex-integration/catalog-gated-native-suppression-reason.test.ts +- tests/codex-integration/reserve-catalog.test.ts +- tests/codex-integration/multi-agent-keep-native-v1.test.ts +- inject 쪽 restoreCodexCatalogWithPermit 경로 (facade) — PR4 오라클이 여전히 그린인지 + +완료 줄 수 목표: + +| 파일 | 상한 | +|---|---| +| src/codex/inject.ts | 1999 (목표 ~960) | +| src/codex/inject/*.ts 각 | 1999 | +| src/codex/catalog/sync.ts | 1999 (목표 ~80) | +| src/codex/catalog/{subagent-roster,derive-entry,auto-review,gated-native-warn,build-entries,retained-sync,restore}.ts 각 | 1999 | +| 새 파일 전부 | 1999 | +| 래칫 기준선 | 이 사이클 D에서 회수. 증가 0 | + +## 회귀 테스트 총표 (사이클 합본, hosted CI) + +inject: + +- tests/codex-integration/codex-inject.test.ts +- tests/codex-integration/codex-inject-integration.test.ts +- tests/codex-integration/codex-inject-history-wording.test.ts +- tests/codex-integration/codex-inject-write-lock.test.ts +- tests/codex-integration/codex-journal.test.ts +- tests/codex-integration/codex-restore-app-rewrite.test.ts +- tests/codex-integration/codex-retained-root-serialization.test.ts +- tests/codex-integration/codex-history-reachability.test.ts +- tests/codex-integration/codex-history-job.test.ts +- tests/codex-integration/client-injection-guard.test.ts +- tests/providers/xai/grok-writer-boundary.test.ts +- tests/service/autostart-health.test.ts +- tests/server/loopback-listener-admission.test.ts +- tests/server/loopback-companion-client-targets.test.ts + +catalog: + +- tests/codex-integration/catalog-full-picker-order.test.ts +- tests/codex-integration/catalog-go-exact-efforts.test.ts +- tests/codex-integration/catalog-zero-credit-picker.test.ts +- tests/codex-integration/catalog-free-pricing-status.test.ts +- tests/codex-integration/catalog-gated-native-suppression-reason.test.ts +- tests/codex-integration/codex-catalog.test.ts +- tests/codex-integration/codex-catalog-model-picker-order.test.ts +- tests/codex-integration/codex-models-cache-invalidate.test.ts +- tests/codex-integration/multi-agent-keep-native-v1.test.ts +- tests/codex-integration/reserve-catalog.test.ts +- tests/codex-integration/native-alias-maintainer-regressions.test.ts +- tests/codex-integration/codex-v2-gate.test.ts +- tests/providers/provider-model-aliases.test.ts + +로컬에서 이 목록을 실행하지 않는다. PR 본문에 NOT RUN을 적고 hosted exact-head만 증거로 쓴다. + +## 실행 순서 (기술 의존만) + +1. routing-target (leaf, providerBaseHost 포함) +2. config-toml + classify (toml이 1에 의존) +3. remove (toml EOL/strip에 의존) +4. inject restore (remove에 의존, catalog restore는 아직 sync facade) +5. roster + derive-entry + effort import 삭제 (순환 절단). roster가 derive보다 앞선다 +6. auto-review + gated-native-warn (상태 이전, write 경로보다 앞) +7. build-entries + retained-sync + catalog restore 동시 (mtime/in-place/permit 계약, sync facade화) + +PR 사이에 동작 커밋을 끼우지 않는다. 래칫이 사이클 1에 있으면 각 PR의 새 파일은 2,000줄 미만이어야 통과한다. 예상 최장 새 파일은 build-entries.ts ~1020, retained-sync.ts ~720, config-toml.ts ~630, auto-review.ts ~560, inject/restore.ts ~560. + +## 수용 기준 + +- inject.ts와 catalog/sync.ts 각각 1,999줄 이하, 새 모듈 전부 1,999줄 이하 +- public export 집합이 이동 전과 동일 (inject facade, sync facade, catalog.ts) +- 모듈 상태 싱글톤 분기 0. gated-native Set과 history 훅 3개가 표의 소유 모듈에만 있다 +- 함정 5항 미발생 +- INV 헤더 주석이 승계 모듈에 있고 테스트 바인딩 파일이 남아 있다 +- 오라클 3파일이 새 본문 경로를 읽는다 +- structure 8+2 백틱이 위 표대로다 +- layout.json 등록 없음 +- 로컬 스위트 NOT RUN, hosted CI exact-head 녹색 (레인 정책은 000_plan.md) + +## 실행자가 복사할 이동 명령 (각 PR C) + +행 범위는 이 문서 작성 시점의 inject.ts 2342 / sync.ts 2698 기준 inclusive다. 앞 PR이 줄을 지우면 이후 PR은 심볼 이름으로 잘라라. sed 행번호는 PR1에만 안전하다. + +PR1 (inject.ts 그대로일 때): + + mkdir -p src/codex/inject + sed -n '173,288p' src/codex/inject.ts + +PR2: + + sed -n '96,135p;289,513p;620,894p' src/codex/inject.ts + sed -n '514,619p' src/codex/inject.ts + +PR3: + + sed -n '1672,1831p' src/codex/inject.ts + +PR4: + + sed -n '1832,2312p' src/codex/inject.ts + sed -n '928,930p' src/codex/inject.ts + +PR5: + + sed -n '91,259p' src/codex/catalog/sync.ts + sed -n '260,460p' src/codex/catalog/sync.ts + +PR6: + + sed -n '1596,2094p' src/codex/catalog/sync.ts + sed -n '2095,2152p' src/codex/catalog/sync.ts + +PR7: + + sed -n '461,1393p' src/codex/catalog/sync.ts + sed -n '1394,1595p;2153,2427p;2476,2563p;2636,2698p' src/codex/catalog/sync.ts + sed -n '2428,2475p;2564,2635p' src/codex/catalog/sync.ts + +앞 PR이 이미 줄을 지웠으면 위 sed는 틀린 범위를 자른다. 그때는 이 문서의 심볼 표(함수/타입 이름)가 권위다. + diff --git a/devlog/_plan/260914_godfile_round2/040_phase4_routing_and_quota.md b/devlog/_plan/260914_godfile_round2/040_phase4_routing_and_quota.md new file mode 100644 index 0000000000..459f622cd6 --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/040_phase4_routing_and_quota.md @@ -0,0 +1,425 @@ +# 040 사이클 4 — routing.ts / quota.ts 갓파일 분해 + +사이클 4는 facade를 남긴 순수 이동으로 `src/codex/routing.ts`(3,507줄)와 `src/providers/quota.ts`(3,313줄)를 각각 1,999줄 이하로 나눈다. 소비자는 기존 경로를 그대로 import하고, 모듈 수준 싱글턴은 파일 하나에서만 살며, 재시도 예산은 `src/lib/request-execution-budget.ts`의 request 범위 밖을 만들지 않는다. 이 문서는 구현자가 행 범위를 복사해 옮길 수 있는 계약이다. 초안과 어긋난 실측은 본문에 `정정:`으로 적는다. + +로프 위치: 브랜치 `codex/m2k-l5-routing-quota`, base는 L4(`codex/m2k-l4-inject-sync`). 로컬 install/build/typecheck/suite는 NOT RUN. 검증은 hosted CI(레인 tip exact-head). 새 테스트 파일을 만들지 않으므로 `scripts/test-layout/layout.json`과 `tests/fixtures/test-layout-expected.json`은 등록하지 않는다. + +## 범위와 비범위 + +범위는 두 갓파일의 본문을 `src/codex/routing/`와 `src/providers/quota/` 아래로 옮기고, 원래 경로를 전량 re-export facade로 남기는 일이다. 기능 정책, 쿨다운 숫자, 프로브 엔드포인트, 관측 시임, 재시도 횟수는 바꾸지 않는다. + +비범위: `src/server/responses/core.ts` 본체, `src/lib/request-execution-budget.ts` 정책 값, `src/quota/` reset-observer 본체, `src/routing/`(별도 패키지), `src/codex/quota.ts`, 관리 라우트의 import 경로 변경, 인자로 상태를 넘기는 리팩터. + +디렉터리 충돌: 새 모듈은 반드시 `src/codex/routing/*.ts`와 `src/providers/quota/*.ts`다. 기존 `src/routing/`과 `src/quota/`(reset-observer)에 넣으면 소유권과 옵셔널 서브시스템 경계가 깨진다. `src/adapters/kiro.ts`+`src/adapters/kiro/`와 같은 facade+디렉터리 패턴을 따른다. macOS에서 `quota.ts`와 디렉터리 `quota/`는 확장자가 달라 공존한다. + +## 예산 범위 불변 + +재시도 예산은 `src/lib/request-execution-budget.ts`의 request 범위다. 생성 지점은 `src/server/responses/core.ts:3508`(`sendBudget: options.sendBudget ?? createRequestExecutionBudget()`)과 `:5039`(`const sendBudget = options.sendBudget ?? createRequestExecutionBudget()`). account-failover 퍼밋은 `:1506`에서 `executionBudget`을 읽고 `:1511-1515`에서 `sendClass: "account-failover"`로 `reserveDispatch`한다. 정정: 초안의 ":1506 account-failover 퍼밋"은 바인딩 행이고, 실제 퍼밋 소비는 1511-1515다. gated-model 동일 계정 재시도는 `:1654` `maxRetrySends = retrySameConfirmedAccount ? 7 : 1`이며 이 숫자는 core.ts 소유다. + +새 모듈 어디에도 시도 카운터, sendClass, maxRetrySends, reserveDispatch 복제를 신설하지 않는다. 링 전진은 계속 `src/codex/pool-rotation.ts`의 `pickRoundRobinAccount`가 수행한다. active 커서 승격 함수 `promoteActiveCodexAccount`는 `active-account.ts` 한 곳에만 둔다. + +정정: `recordCodexUpstreamOutcome`은 `promoteActiveCodexAccount`의 단독 호출자가 아니다. 현재 호출 사이트는 `reconcileCodexActiveAfterExclusion:2239`, `applyFailureFailover:2413`, `resolveCodexAccountForThreadDetailed:2917·2967·3029`, `recordCodexUpstreamOutcome:3375·3426`이다. 단독 호출자 제약은 429 경로의 `pickAlternateCodexAccount` 재호출 금지로 좁힌다. 같은 요청이 이미 고른 대체 계정은 `meta.promoteAccountId`로 재사용하며, 그 행은 초안 3369·3420이 아니라 **3371·3422**다. 이 재사용을 빼면 round-robin 링이 한 요청에 두 칸 전진한다. + +## 상태 소유권 — routing.ts + +모듈 수준 싱글턴은 아래 표의 소유 파일로만 이동한다. 인자로 Map/Set을 넘기거나, 테스트 훅으로 두 번째 인스턴스를 만들지 않는다. 관리 라우트 9곳이 `clearThreadAccountMap`을 직접 import하는 것은 facade 싱글턴을 가리키게 그대로 둔다. 인자로 넘기면 호출측 기본값과 facade가 갈라져 인스턴스가 분기한다. + +| 상태 | 현재 행 | 소유 모듈 | 비고 | +|---|---|---|---| +| `upstreamHealth` | 269-274 | `routing/health-store.ts` | `Map` | +| `quotaScopedHealth` | 275-284 | `routing/health-store.ts` | 계정→scope→health | +| `lastReconciledGeneration` | 292 | `routing/health-store.ts` | `reconcileCodexRoutingHealth`와 `recordCodexUpstreamOutcome`이 함께 읽음. 후자는 facade에 남고 health-store getter를 쓴다 | +| `liveHealthAccountIds` | 293-294 | `routing/health-store.ts` | 동일 | +| `threadAccountMap` | 331 | `routing/thread-affinity.ts` | 금지: 순수함수+상태인자 | +| `threadAffinityEntryTotal` | 332-333 | `routing/thread-affinity.ts` | map과 같이 증감 | +| `pendingReleaseReasons` | 455-481 | `routing/thread-affinity.ts` | `MAX_PENDING_RELEASE_REASONS = 4096` | +| `manualPreference` | 2118-2155 | `routing/active-account.ts` | 연산자 one-shot | +| `runtimeActiveCodexAccountId` | 142-143 | `routing/active-account.ts` | 프로세스 로컬 커서 | + +정리 진입점은 확인됨. `src/lib/state-store-registrations.ts:111` `{ name: "codex-routing-health", reconcileGeneration: reconcileCodexRoutingHealth }`. import는 같은 파일 `:15-16`에서 `../codex/routing` facade. 분해 후에도 facade에서 re-export한다. `listLiveCodexAccountIds`(`:419-427`)는 같은 파일 `:61` `buildGenerationContext`가 쓰므로 health-store가 구현을 갖고 facade가 re-export한다. + +관리 라우트 9곳(직접 import, 경로 유지): + +1. `src/server/management-api.ts:33` +2. `src/server/management/oauth-account-routes.ts:36` +3. `src/server/management/model-routes.ts:115` +4. `src/server/management/combo-routes.ts:37` +5. `src/server/management/agent-settings-routes.ts:40` +6. `src/server/management/config-routes.ts:44` +7. `src/server/management/provider-routes.ts:69` (`:962·1065·1340`는 기존 `deps.clearThreadAccountMap ?? clearThreadAccountMap` 테스트 시임. 새 주입을 늘리지 않는다) +8. `src/server/management/logs-usage-routes.ts:32` +9. `src/server/management/shared.ts:34` + +`src/server/index.ts:93-96`도 import하지만 관리 라우트가 아니다. 이것도 facade를 유지한다. + +## 상태 소유권 — quota.ts + +| 상태 | 현재 행 | 소유 모듈 | 비고 | +|---|---|---|---| +| `nativeMainReportGenerations` | 110 | `quota/report-cache.ts` | WeakMap, report 객체 키 | +| `accountReportCurrent` | 111 | `quota/report-cache.ts` | 동일 | +| `routingEvidence` | 112 | `quota/report-cache.ts` | 동일. 세 WeakMap을 다른 파일로 쪼개지 않는다 | +| `cache` / `inflight` / `invalidationEpoch` | 169-174 | `quota/report-cache.ts` | 프로세스 캐시 | +| `accountQuotaCache` | 1736 | `quota/account-cache.ts` | | +| `explicitAccountEpoch` | 1737 | `quota/account-cache.ts` | | +| `diskHydrated` | 1747 | `quota/account-cache.ts` | | +| `accountQuotaInflight` | 1772 | `quota/account-cache.ts` | | +| `lastReconciledGeneration` (quota) | 1773 | `quota/account-cache.ts` | routing의 동명 상태와 별개 | +| `liveAccountQuotaKeys` / `liveProviderQuotaKeys` | 1774-1775 | `quota/account-cache.ts` | | +| `anthropicUsageInflight` | 1480 | `quota/vendor-probes-oauth.ts` | anthropic 프로브 전용 | +| `antigravityOutboundDependencies` | 2915-2919 | `quota/antigravity.ts` | 테스트 시임 | +| `pendingProviderObservation` | 3144 | **facade에 잔류** | 이동 금지 | +| `providerQuotaBeforePublishForTests` | 113-120 | `quota/report-cache.ts` | publish 직전 훅 | + +`notifyProviderQuotaSnapshot`(`:3171-3201`)과 `pendingProviderObservation`은 `src/providers/quota.ts` facade에 남긴다. 동적 엣지 `import("../quota/reset-observer")`와 `import("../quota/window-mapping")`가 이 함수 안에 있다. 옮기면 `tests/usage/quota-reset-core-boundary.test.ts:37` `SEAMS`가 새 경로의 동적 엣지를 못 찾고, 또는 정적 import가 생기면 core 경로에 reset-observer가 올라간다. + +## 함정 (금지 분할) + +1. `recordCodexUpstreamOutcome`(3147-3497, 351줄)을 outcome class별 파일로 나누지 않는다. 계약 순서는 `dropSpentCredentialFailure`(3178) → success/caller/neutral/workspace/credential → scoped 429(reset-derived, 3324-3382) → account-wide 429(3384-3457) → transient(3459-3496)이다. 한 분기가 `preservedCooldownFields`와 lease generation을 공유하므로 분리하면 순서와 필드 보존이 깨진다. +2. affinity API를 `(map, threadId, ...)` 형태의 순수함수로 바꾸지 않는다. `threadAccountMap`과 `threadAffinityEntryTotal`은 `thread-affinity.ts`의 모듈 바인딩으로만 존재한다. +3. `notifyProviderQuotaSnapshot` / `pendingProviderObservation` 이동 금지. +4. `.json(` 오라클을 확장하지 않은 채 프로브 함수만 추출 금지. PR 5가 첫 프로브 이동이며 오라클 확장이 같은 커밋에 있어야 한다. +5. 새 시도 카운터 신설 금지 (예산 불변). +6. `promoteAccountId` 재사용 삭제 금지 (3371·3422). +7. 순환 import: `transientDetourAccount`(1760-1787)와 `isTransientOnlyAffinityBlock`(1714-1731)은 `pickAlternateCodexAccount`를 호출한다. 이를 `thread-affinity.ts`에 넣으면 selection과 순환한다. 잔여 resolve 경로에 남긴다. 정정: 초안 thread-affinity ~600은 이 블록을 포함했다. 실제 이동분은 ~480. + +## 오라클과 structure 동반 수정 + +### 오라클 1 — `tests/config/config-save-boundary.test.ts:22` + +`GUARDED_FILES`가 `"codex/routing.ts"`를 리터럴로 읽고 bare `saveConfig(`를 금지한다. 현재 writer는 bare가 아니라 `saveConfigPreservingClaudeCode`다. + +- `:2196` `setActiveCodexAccount` +- `:2305` `releaseDrainedCodexAccountPin` (reauth/pause 경로) +- `:2316` `releaseDrainedCodexAccountPin` (drained 경로) + +세 writer가 `active-account.ts`로 가면 **같은 PR에서** `GUARDED_FILES`에 `"codex/routing/active-account.ts"`를 추가한다. facade `codex/routing.ts` 항목은 남긴다(차후 writer가 facade에 다시 생기는 것을 막는다). 추가하지 않으면 오라클이 새 파일을 읽지 않아 bare `saveConfig(`가 통과한다. + +### 오라클 2 — `tests/providers/provider-quota.test.ts:122` + +`readFileSync(repoPath("src/providers/quota.ts"))` 본문에서 `/\.\s*json\s*\(/`를 금지한다. 프로브가 다른 파일로 나가면 그 파일도 같은 정규식으로 읽어야 한다. PR 5에서 배열로 확장하고, PR 6에서 oauth/antigravity 경로를 추가한다. + +### 오라클 3 — `tests/usage/quota-reset-core-boundary.test.ts:37` + +`SEAMS = ["src/codex/quota.ts", "src/providers/quota.ts"]`. facade에 `notifyProviderQuotaSnapshot`이 남는 한 SEAMS는 그대로다. 옮기면 SEAMS에 새 경로를 넣고 `OBSERVER_SPEC = "../quota/reset-observer"` 동적 엣지가 그 파일에서 발견돼야 한다. 이 사이클에서는 옮기지 않으므로 SEAMS 수정 없음. + +### 오라클 4 — `tests/usage/quota-reset-detector.test.ts:120` + +주석: `src/providers/quota.ts:279 and src/codex/quota.ts:192 disagree on whether 0 survives`. 정정: 현재 `quota.ts:279`는 `publicCapacityAggregation` 본문이며 0-survive와 무관하다. 실제 대립은 다음이다. + +- `src/providers/quota.ts:1686-1687` `validReset`: `resetAt > 0` → 0 폐기 +- `src/providers/quota-wire.ts:32` `epochMillis`: `value <= 0` → 0 폐기 +- `src/codex/quota.ts:184` `normalizeResetAt`: `numeric < 0` → 0 생존 (`:173-184`, 주석이 가리킨 `:192`도 어긋남) + +PR 7이 `normalizeAnthropicQuota`를 `account-cache.ts`로 옮기면 주석 경로를 새 파일의 `validReset` 행으로 고친다. 동작은 바꾸지 않는다. + +### structure 백틱 (본문 텍스트를 읽는 소스 오라클) + +| 문서 | 현재 | 이동 후 | PR | +|---|---|---|---| +| `structure/providers/openai-tiers.md:451` | `src/codex/routing.ts` applies optional `codexPool.excludedPlans` | 구현은 `src/codex/routing/selection.ts` (`isCodexAccountPlanExcluded` 1313-1325, `excludedCodexPoolPlanKeys` 1287-1312) | 4 | +| `structure/providers/openai-tiers.md:518` | `src/codex/routing.ts` supports `accountPoolStrategy: "reset-first"` | 구현은 `src/codex/routing/selection.ts` `pickResetFirstCodexAccount` 1788-1816 | 4 | +| `structure/runtime.md:342` | `src/providers/quota.ts` publishes routing evidence only when a producer explicitly supplies | WeakMap `routingEvidence` 소유가 `quota/report-cache.ts` | 8 | +| `structure/gui-and-management-api.md:502` | `src/providers/quota.ts` uses one exact normalized-base mapping for both Z.ai quota | `zaiQuotaMonitorHost` 356-373, `isCanonicalZaiBaseUrl` 374-377, `fetchZaiQuota` 881-928 → `quota/vendor-probes-key.ts` | 5 | +| `structure/transports/inventory.md:34` | 표 Discovery and quota에 `src/providers/quota.ts` | facade 유지. Spark DTO 억제는 `fetchProviderQuotaReports` 잔류 | 8에서 facade 잔류를 명시 | +| `structure/transports/inventory.md:134` | `src/providers/quota.ts` binds diagnoses to the probed credential/project | 진단 바인딩은 `fetchAccountQuota` 2170-2283(`account-cache.ts`)와 `probeAntigravityUsageQuota`(`antigravity.ts`) | 6+7 | + +`structure/manifest.json`은 이미 `runtime.md`가 `src/codex/`와 `src/providers/`를 문서화하므로 새 top-level src area가 아니다. `bun run structure:index`는 백틱 문구만 고치면 필요 없고, manifest documents 배열을 건드리지 않는다. INV 승계: `INV-OPENAI-01`(Pool/Direct)은 선택 로직이 selection.ts로 옮겨도 제품 불변식은 동일하고 테스트 승계 모듈은 `tests/codex-integration/codex-routing.test.ts`와 `tests/codex-integration/codex-pool-plan-exclusion.test.ts`. `INV-TESTS-01`은 새 테스트 파일이 없으므로 유지. 구조 게이트 승계는 `tests/ci-workflows/structure-ssot.test.ts`. + +## routing.ts 현재 지도 (3,507줄) + +원본 행은 이 HEAD 기준이다. + +| 구간 | 행 | 줄 수 | 목적지 | +|---|---|---|---| +| import | 1-39 | 39 | 각 모듈이 필요한 것만. facade는 자식 re-export만 | +| affinity 타입/헬퍼 | 41-140 | 100 | thread-affinity.ts | +| `runtimeActiveCodexAccountId` | 142-143 | 2 | active-account.ts | +| `CodexUpstreamHealth` | 144-210 | 67 | health-store.ts | +| cooldown 상수 | 211-244 | 34 | cooldown-math.ts | +| affinity 상수 | 245-267 | 23 | thread-affinity.ts | +| health 맵 + dropSpent + reconcile 커서 | 269-294 | 26 | health-store.ts | +| outcome/scope/probe 타입 | 295-326 | 32 | cooldown-math(295-304) / health-store(305-306) / probe-lease(307-326) | +| affinity 맵 | 327-336 | 10 | thread-affinity.ts | +| quota scope 매핑 | 338-355 | 18 | health-store.ts (`codexQuotaScopeForModel` export) | +| `CodexUpstreamOutcomeMeta` | 356-406 | 51 | cooldown-math.ts (순수 타입. promoteAccountId 필드 포함) | +| `hasConfiguredPoolAccount` | 407-418 | 12 | 잔여 (resolve가 사용) | +| `listLiveCodexAccountIds` | 419-427 | 9 | health-store.ts | +| `clearThreadAccountMap*` | 428-454 | 27 | thread-affinity.ts | +| pending release | 455-481 | 27 | thread-affinity.ts | +| health clear/reconcile/get/scoped mutators | 483-562 | 80 | health-store.ts | +| usage/classify/parse/computeQuotaCooldown | 563-736 | 174 | cooldown-math.ts | +| `codexQuotaAvoidUntil` / `isCodexQuotaAvoided` | 737-759 | 23 | health-store.ts (맵 읽기. 초안이 cooldown-math에 넣으면 순수 제약을 깨므로 정정) | +| `computeQuotaCooldownUntil` | 760-774 | 15 | cooldown-math.ts | +| probe-lease 전체 | 775-1094 | 320 | probe-lease.ts | +| `preservedCooldownFields` | 1095-1107 | 13 | health-store.ts (record/probe가 공유) | +| `resetCodexRoutingForManualSelection` | 1108-1151 | 44 | active-account.ts (커서+preference 쓰기, affinity/health를 호출) | +| cooldown 스냅샷/clear/soft-avoid | 1152-1286 | 135 | health-store.ts | +| plan exclusion + selectable + block reason | 1287-1366 | 80 | selection.ts | +| affinity bind/prune/handOff | 1367-1583 | 217 | thread-affinity.ts | +| eligible/headroom/cacheAffinity | 1584-1713 | 130 | selection.ts | +| transient hold/detour | 1714-1787 | 74 | **잔여** (순환 import 방지) | +| pick* / peek* / plan helpers | 1788-2117 | 330 | selection.ts | +| manualPreference + active cursor + saveConfig writers | 2118-2242, 2292-2318 | 152 | active-account.ts | +| `pickPriorityPreemption` | 2255-2291 | 37 | selection.ts | +| `applyQuotaAutoSwitch` ~ `applyFailureFailover` | 2319-2419 | 101 | selection.ts (`setActive`/`promote`를 active-account에서 import) | +| `resolveCodexAccountForThread` | 2420-2429 | 10 | 잔여 | +| refusal + preview/rebind | 2430-2777 | 348 | 잔여 | +| `resolveCodexAccountForThreadDetailed` | 2778-3146 | 369 | 잔여 | +| `recordCodexUpstreamOutcome` | 3147-3497 | 351 | 잔여 | +| `formatCodexProviderForLog` | 3498-3507 | 10 | 잔여 | + +정정: 잔여 ~900은 detailed 369 + record 351만으로 채워지지 않는다. 위 잔여 합은 hasConfigured(12)+transient(74)+resolve wrapper(10)+preview/rebind(348)+detailed(369)+record(351)+format(10) = 1,174에 facade re-export ~80을 더하면 ~1,250이다. 1,999 이하이므로 허용. 초안 ~900은 preview/rebind/transient를 빠진 채 센 숫자다. + +## quota.ts 현재 지도 (3,313줄) + +| 구간 | 행 | 줄 수 | 목적지 | +|---|---|---|---| +| import + type re-export | 1-82 | 82 | facade 유지 + 각 모듈이 필요한 import | +| key vendor URL 상수 | 83-105 | 23 | vendor-probes-key.ts | +| XAI URL | 106-107 | 2 | vendor-probes-oauth.ts | +| WeakMap 3개 + publish 훅 + 심볼 + Report 타입 | 109-174 | 66 | report-cache.ts | +| cache clear / cacheKey / capacity 공개 | 175-313 | 139 | report-cache.ts | +| `readProviderQuotaJsonForTests` | 314-318 | 5 | report-cache.ts (오라클이 이 심볼을 quota.ts에서 import. facade re-export) | +| canonical URL + key fetchers A6api~Neuralwatt | 319-1196 | 878 | vendor-probes-key.ts | +| `report` / `keyReport` / `tagNativeMainReport` / `publishKeyReportForTests` / `isProviderQuotaReportCurrent` | 1197-1273 | 77 | report-cache.ts | +| `fetchChatGptForwardQuota` | 1274-1347 | 74 | vendor-probes-oauth.ts | +| xai/claude/anthropic/kiro/muse/passive | 1348-1671 | 324 | vendor-probes-oauth.ts | +| account-cache 타입~explicit helpers | 1672-2163 | 492 | account-cache.ts | +| `antigravityQuotaDiagnosticIdentity` | 2164-2169 | 6 | antigravity.ts | +| `fetchAccountQuota` + `fetchProviderAccountQuotas` | 2170-2311 | 142 | account-cache.ts (antigravity probe를 import) | +| kimi/command parsers+fetchers | 2312-2597 | 286 | vendor-probes-key.ts (`keyQuotaReaderForProvider`가 닫힘) | +| `fetchCursorQuota` | 2598-2757 | 160 | vendor-probes-oauth.ts | +| antigravity parse/probe/fetch/test seam | 2758-3035 | 278 | antigravity.ts | +| `KeyQuotaReader` + `keyQuotaReaderForProvider` + `providerApiKeyQuotaMode` | 3036-3076 | 41 | vendor-probes-key.ts | +| `fetchProviderApiKeyQuotas` | 3077-3087 | 11 | 잔여 (`maybeFetchProviderQuota` 호출) | +| `maybeFetchProviderQuota` | 3088-3143 | 56 | 잔여 | +| 관측 시임 + `fetchProviderQuotaReports` | 3144-3313 | 170 | 잔여 | + +정정: vendor-probes-key 초안 ~900은 319-1196만 센 값(878). kimi/command(286)+selector(41)+URL 상수(23)를 같은 파일에 모아야 `keyQuotaReaderForProvider`가 컴파일되므로 예상 **~1,230**. antigravity 초안 ~230 → 실제 2758-3035(278)+identity(6) = **~284**. account-cache 초안 ~520 → 1672-2163(492)+fetchAccountQuota/fetchProviderAccountQuotas(142) = **~634**. report-cache 초안 ~230 → 109-174(66)+175-318(144)+1197-1273(77) = **~287**. 잔여 초안 ~700은 `fetchProviderQuotaReports` 3207-3313(107줄)이 아니다. 잔여 합은 fetchProviderApiKeyQuotas(11)+maybeFetch(56)+관측/reports(170)+import/re-export ~80 = **~320**. 1,999 이하. + +## PR 1 — cooldown-math + +목적: 상태 없는 쿨다운/사용량 산술만 분리해 이후 모듈이 숫자 규칙을 공유한다. + +Write set: + +- NEW `src/codex/routing/cooldown-math.ts` 예상 300줄. 정정: 초안 ~260은 `CodexUpstreamOutcomeMeta`(356-406, 51줄)를 뺀 값. Meta는 promoteAccountId를 담지만 값 객체 타입이라 여기에 둔다. +- MODIFY `src/codex/routing.ts` 해당 본문을 삭제하고 `export { ... } from "./routing/cooldown-math"` + +원본 이동 행: 211-244, 295-304, 356-406, 563-736, 760-774. + +내보낼 이름: `CODEX_QUOTA_PROBE_INTERVAL_MS`, `CODEX_FAILURE_WINDOW_MS`, `TERMINAL_SHORT_WINDOW_FRESHNESS_MS`, `CODEX_TRANSIENT_SOFT_AVOID_MS`, `CODEX_TRANSIENT_SOFT_AVOID_ESCALATION_MS`(const, 같은 파일), `CODEX_DEFAULT_QUOTA_COOLDOWN_MS`, `CODEX_MAX_QUOTA_COOLDOWN_MS`, `CODEX_MAX_RESET_DERIVED_COOLDOWN_MS`, `CODEX_MAX_QUOTA_AVOID_MS`, `CodexUpstreamOutcome`, `CodexUpstreamOutcomeClass`, `CodexCooldownSource`, `CodexUpstreamOutcomeMeta`, `computeCodexUsageScore`, `classifyCodexUpstreamOutcome`, `parseRetryAfterMs`, `parseResetCooldownMs`, `computeQuotaCooldown`, `computeQuotaCooldownUntil`. 같은 파일이 쓰는 비export `isTerminalShortWindow`, `clampCooldownMs`, `resetTimestampMs`, `quotaAvoidUntilFor`도 이 파일에 둔다. + +`quotaAvoidUntilFor`는 순수(meta+now+cooldownUntil)이므로 여기 둔다. `codexQuotaAvoidUntil`는 맵을 읽으므로 이동하지 않는다. + +회귀: `tests/codex-integration/codex-routing.test.ts`, `tests/codex-integration/codex-cooldown-recovery.test.ts`, `src/combos/failover.ts:1`이 `parseResetCooldownMs`를 routing facade에서 import하므로 facade re-export가 빠지면 combos가 깨진다. + +structure 수정 없음. layout 등록 없음. + +완료 조건: `routing.ts`가 위 함수 본문을 갖지 않고 re-export만 한다. 새 파일에 `let`/`Map` 없음. `computeCodexUsageScore`는 `CODEX_UNKNOWN_USAGE_SCORE`/`CODEX_EXHAUSTED_USAGE_PERCENT`(`../quota`)와 `isThirtyDayOnlyCodexPlan`(`../plan`)만 쓴다. + +## PR 2 — health-store + probe-lease + +목적: health 맵과 그 맵을 잠그는 probe lease를 한 PR에서 옮겨 싱글턴이 한 쌍으로만 존재하게 한다. 두 파일로 나누되 lease는 health-store의 mutator를 import한다. 맵을 인자로 받지 않는다. + +Write set: + +- NEW `src/codex/routing/health-store.ts` 예상 420줄 +- NEW `src/codex/routing/probe-lease.ts` 예상 330줄 (775-1094 = 320 + 타입 307-326 = 20 + import ≈ 350. 초안 ~330에 가깝다) +- MODIFY `src/codex/routing.ts` re-export + +health-store 원본 행: 144-210, 269-294, 305-306, 338-355, 419-427, 483-562, 737-759, 1095-1107, 1152-1286. + +포함 심볼: `CodexUpstreamHealth`, `CodexQuotaScope`, `dropSpentCredentialFailure`, `lastReconciledGeneration`, `liveHealthAccountIds`, `NATIVE_MODEL_QUOTA_SCOPES`, `codexQuotaScopeForModel`, `isIndependentCodexQuotaScope`, `codexPoolKeyForScope`, `listLiveCodexAccountIds`, `clearCodexUpstreamHealth`, `clearCodexUpstreamHealthForAccount`, `reconcileCodexRoutingHealth`, `getCodexUpstreamHealth`, `scopedHealthFor`, `setScopedHealth`, `deleteScopedHealth`, `codexQuotaAvoidUntil`, `isCodexQuotaAvoided`, `preservedCooldownFields`, `getCodexAccountCooldownUntil`, `getCodexAccountHealthSnapshot`, `getCodexQuotaHealthSnapshot`, `isCodexAccountInCooldown`, `clearCodexAccountCooldown`, `getCodexAccountSoftAvoidUntil`, `isCodexAccountSoftAvoided`. + +probe-lease가 맵을 직접 만지지 못하게 health-store는 `getAccountHealth`/`setAccountHealth`/`deleteAccountHealth`(이름은 구현자 선택, 의미는 account-wide Map mutator)를 같은 파일에서만 닫힌 채 export한다. 다른 패키지가 Map 값을 import하지 못하게 한다. facade는 기존 public 이름만 re-export. + +`recordCodexUpstreamOutcome`(잔여)는 `lastReconciledGeneration`과 `liveHealthAccountIds`를 읽는다. health-store가 `isHealthAccountAdmissible(accountId, writerGeneration)` getter를 제공하거나 두 바인딩의 읽기 함수를 export한다. 복제하지 않는다. + +probe-lease 원본 행: 307-326, 775-1094. + +포함 심볼: `CodexQuotaRecoveryProbeClaim`, `CodexQuotaRecoveryProbeProof`, `ManualResetCooldownClaim`, `ManualResetRefreshLineage`, `tryAcquireCodexQuotaProbeLease`, `canAcquireCodexQuotaProbeLease`, `claimDueCodexQuotaRecoveryProbes`, `claimManualResetCooldowns`, `settleManualResetCooldown`, `settleCodexQuotaRecoveryProbe`, `tryAcquireCodexQuotaScopeProbeLease`, `canAcquireCodexQuotaScopeProbeLease`, `releaseCodexQuotaProbeLease`, `releaseCodexQuotaScopeProbeLease`, `ownsProbeLease`, `probeMayClearCooldown`, `withProbeLeaseReleased`. `ownsProbeLease`는 record 경로가 쓰므로 export한다. + +회귀: `tests/codex-integration/codex-cooldown-recovery.test.ts`, `tests/codex-integration/reserve-quota-scope.test.ts`, `tests/oauth/oauth-health.test.ts`, `tests/oauth/state-store-sweeper.test.ts`(codex-routing-health 등록). + +완료 조건: 두 파일이 같은 프로세스에서 하나의 `upstreamHealth`를 본다. 테스트가 `clearCodexUpstreamHealth()` 후 probe lease가 빈 맵을 본다. + +## PR 3 — thread-affinity + +목적: 스레드 바인딩 맵과 LRU/TTL/generation hand-off를 한 모듈에 둔다. + +Write set: + +- NEW `src/codex/routing/thread-affinity.ts` 예상 480줄 (정정: 초안 ~600에서 transient detour 74줄을 잔여로 뺌) +- MODIFY `src/codex/routing.ts` + +원본 행: 41-140, 245-267, 327-336, 428-454, 455-481, 1367-1583. + +포함 심볼: affinity 타입 전부, `CODEX_THREAD_AFFINITY_*`, `CODEX_TRANSIENT_AFFINITY_HOLD_MS`, `clearThreadAccountMap`, `clearThreadAccountMapForAccount`, `debugCodexAffinityGenerations`, `handOffThreadAffinityGeneration`, 내부 `bindThreadAffinity`, `bindModelDetourAffinity`, `deleteThreadAffinitiesForAccount`, `getThreadAffinity`, `prune*`, pending reason 삼총사. 잔여 resolve/record가 bind/delete/get/pending을 쓰므로 이 내부 함수들은 `src/codex/routing/` 안에서 export한다. 외부 facade는 기존 public만. + +남기지 말 것: 1714-1787. + +관리 라우트 9곳의 import 경로는 그대로 `../../codex/routing` 또는 `../codex/routing`. + +회귀: `tests/server/session-affinity.test.ts`, `tests/codex-integration/codex-routing.test.ts`, `tests/codex-integration/codex-pool-rotation.test.ts`. + +완료 조건: `clearThreadAccountMap()`가 관리 라우트와 테스트에서 같은 맵을 비운다. 함수 시그니처에 Map 파라미터가 없다. + +## PR 4 — selection + active-account + +목적: 후보 선택과 active 커서/디스크 writer를 한 PR에서 옮겨, 선택이 승격 함수를 호출해도 커서가 한곳이다. + +Write set: + +- NEW `src/codex/routing/selection.ts` 예상 680줄 (정정: 초안 ~560) +- NEW `src/codex/routing/active-account.ts` 예상 240줄 +- MODIFY `src/codex/routing.ts` +- MODIFY `tests/config/config-save-boundary.test.ts` — `GUARDED_FILES`에 `"codex/routing/active-account.ts"` 추가. 기존 `"codex/routing.ts"`는 유지 +- MODIFY `structure/providers/openai-tiers.md:451`와 `:518` 백틱 구현 경로 + +selection 원본 행: 1287-1366, 1584-1713, 1788-2117, 2255-2291, 2319-2419. + +포함 심볼: `isCodexAccountPlanExcluded`, `getPoolAccountPlan`, `pickLowestUsageCodexAccount`, `pickAlternateCodexAccount`, 내부 pick/peek/eligible/headroom/`applyQuotaAutoSwitch`/`applyFailureFailover`/`shouldFailover`/`pickPriorityPreemption`. + +active-account 원본 행: 142-143, 1108-1151, 2118-2242, 2292-2318. + +포함 심볼: `resetCodexRoutingForManualSelection`, `getEffectiveActiveCodexAccountId`, `isEffectiveCodexAccountPinned`, `reconcileCodexActiveAfterExclusion`, 내부 `promoteActiveCodexAccount`, `setActiveCodexAccount`, `rememberActiveCodexAccount`, `releaseCodexAccountPinFor`, `releaseDrainedCodexAccountPin`, `consumeManualPreference`, `forgetManualPreference`, `manualPreferenceBlocks`. + +`setActiveCodexAccount:2196`, `releaseDrainedCodexAccountPin:2305·2316`의 `saveConfigPreservingClaudeCode`가 이 파일로 온다. 오라클 1 동반 수정이 이 PR의 완료 조건이다. + +`promoteActiveCodexAccount`는 이 파일의 패키지 내부 export다. 잔여 `recordCodexUpstreamOutcome`과 selection의 failover가 호출한다. 새 호출자를 만들지 않는다. 429 분기는 계속 `meta.promoteAccountId`를 재사용한다. 그 두 블록은 record 함수 안에 남는다. + +회귀: `tests/codex-integration/codex-pool-rotation.test.ts`, `tests/codex-integration/codex-pool-plan-exclusion.test.ts`, `tests/codex-integration/codex-main-rotation.test.ts`, `tests/config/config-save-boundary.test.ts`, `tests/codex-integration/codex-routing.test.ts`. + +이 PR 후 `src/codex/routing.ts` 잔여 본문 + re-export가 1,999줄 이하여야 한다. 예상 잔여 본문 ~1,174 + re-export ~80 ≈ 1,254. + +## PR 5 — vendor-probes-key + 오라클 확장 + +목적: API 키 프로브를 옮기고, 옮긴 파일에 `.json(` 오라클을 같이 건다. + +Write set: + +- NEW `src/providers/quota/vendor-probes-key.ts` 예상 1,230줄 +- MODIFY `src/providers/quota.ts` +- MODIFY `tests/providers/provider-quota.test.ts:122` 오라클 파일 목록 +- MODIFY `structure/gui-and-management-api.md:502` 백틱 + +원본 행: 83-105, 319-1196, 2312-2597, 3036-3076. + +`keyQuotaReaderForProvider`가 kimi/command/A6api/OpenCode Go/OpenRouter/DeepSeek/Cline/Ollama/Zai/Minimax/Moonshot/Venice/Synthetic/DeepInfra/Neuralwatt를 한 selector로 닫는다. 이 함수와 fetchers를 다른 PR로 쪼개지 않는다. + +오라클 확장 후 형태(동등): + +```ts +const QUOTA_PROBE_SOURCES = [ + "src/providers/quota.ts", + "src/providers/quota/vendor-probes-key.ts", +] as const; +for (const relative of QUOTA_PROBE_SOURCES) { + const source = readFileSync(repoPath(relative), "utf8"); + expect(source).not.toMatch(/\.\s*json\s*\(/); +} +``` + +프로브는 계속 `readQuotaJson`(`quota-wire.ts`)만 쓴다. 새 파일에 `response.json(` 또는 `.json(`가 생기면 이 테스트가 실패해야 한다. + +회귀: `tests/providers/provider-quota.test.ts`, `tests/providers/zhipu-bigmodel-responses-quota.test.ts`, `tests/providers/opencode-go-quota.test.ts`, `tests/providers/command-code-quota.test.ts`, `tests/providers/provider-api-keys.test.ts`. + +layout 등록 없음 (기존 테스트 수정). + +## PR 6 — vendor-probes-oauth + antigravity + +목적: OAuth 프로브와 Antigravity 프로브를 옮긴다. account-cache는 아직 남고 antigravity 함수를 현재 경로에서 import한다. + +Write set: + +- NEW `src/providers/quota/vendor-probes-oauth.ts` 예상 700줄 (1274-1671 398 + cursor 160 + XAI URL 2 + anthropicUsageInflight 포함 import ≈ 620~700) +- NEW `src/providers/quota/antigravity.ts` 예상 284줄 +- MODIFY `src/providers/quota.ts` +- MODIFY `tests/providers/provider-quota.test.ts` 오라클 배열에 두 파일 추가 +- MODIFY `structure/transports/inventory.md:134` — 진단 바인딩 구현 경로. 최종 문구는 PR 7에서 account-cache를 더한다 + +oauth 원본 행: 106-107, 1274-1671, 2598-2757. +antigravity 원본 행: 2164-2169, 2758-3035. + +export: `parseXaiCreditsResponse`, `isCanonicalAntigravityQuotaUrl`, `setAntigravityAccountQuotaTransportForTests`, `fetchAntigravityUsageQuota`. 내부 `fetchAnthropicQuota`/`fetchKiroQuota`/`fetchCursorQuota`/`fetchChatGptForwardQuota`는 `maybeFetchProviderQuota`가 쓰므로 패키지 내부 export. + +회귀: `tests/providers/provider-account-quota.test.ts`, `tests/adapters/anthropic/anthropic-ratelimit-headers.test.ts`, `tests/providers/muse-passive-quota-observation.test.ts`, `tests/providers/kiro/kiro-account-quota.test.ts`. + +## PR 7 — account-cache + +목적: per-account 캐시와 persist/reconcile을 한 모듈에 둔다. + +Write set: + +- NEW `src/providers/quota/account-cache.ts` 예상 634줄 +- MODIFY `src/providers/quota.ts` +- MODIFY `tests/usage/quota-reset-detector.test.ts:120` 주석 경로를 `account-cache.ts`의 `validReset` 행으로 +- MODIFY `structure/transports/inventory.md:134` 최종 구현 파일 `quota/account-cache.ts` + `quota/antigravity.ts` + +원본 행: 1672-2163, 2170-2311. + +포함 심볼: `ProviderAccountQuota`, `supportsPerAccountQuota`, `providerOAuthAccountQuotaMode`, `getCachedProviderAccountQuota`, `setCachedProviderAccountQuotaForTests`, `parseAnthropicRateLimitHeaders`, `recordAnthropicAccountQuotaFromHeaders`, `hasPassiveAccountQuota`, `recordPassiveAccountQuota`, `readPassiveProviderAccountQuotas`, `sweepExpiredProviderAccountQuotaRows`, `reconcileProviderAccountQuotaRows`, `resetProviderQuotaReconcileStateForTests`, `clearAccountQuotaCache`, `fetchProviderAccountQuotas`. + +state-store `provider-quota-history`는 `reconcileProviderAccountQuotaRows`를 facade에서 계속 import. + +회귀: `tests/providers/provider-account-quota.test.ts`, `tests/providers/provider-account-quota-persistence.test.ts`, `tests/oauth/state-store-sweeper.test.ts`, `tests/adapters/anthropic/anthropic-quota-dispatch.test.ts`, `tests/server/provider-account-quota-routes.test.ts`. + +## PR 8 — report-cache + +목적: report 객체 키 WeakMap 세 개와 프로세스 캐시를 한곳에 둔다. 관측 시임은 facade에 남긴다. + +Write set: + +- NEW `src/providers/quota/report-cache.ts` 예상 287줄 +- MODIFY `src/providers/quota.ts` — `fetchProviderQuotaReports` 3207-3313, `maybeFetchProviderQuota` 3088-3143, `notifyProviderQuotaSnapshot` 3171-3201, `pendingProviderObservation` 3144, `fetchProviderApiKeyQuotas` 3077-3087, 전량 re-export +- MODIFY `structure/runtime.md:342` WeakMap 소유를 `src/providers/quota/report-cache.ts`로 +- MODIFY `structure/transports/inventory.md:34`는 facade `src/providers/quota.ts`를 오케스트레이션으로 남긴다. 증거 바인딩 문장은 runtime.md와 중복되지 않게 report-cache를 가리킨다 + +원본 행: 109-174, 175-318, 1197-1273. + +세 WeakMap은 이 파일 밖으로 나가지 않는다. `tagNativeMainReport` / `keyReport` / `isProviderQuotaReportCurrent`만 export. + +`notifyProviderQuotaSnapshot`은 계속 facade에 있고 `import("../quota/reset-observer")` 동적 엣지를 유지한다. `SEAMS` 수정 없음. + +이 PR 후 `src/providers/quota.ts` 예상 ~320줄. + +회귀: `tests/providers/provider-quota.test.ts`, `tests/providers/provider-quota-observed-marker.test.ts`, `tests/usage/quota-reset-core-boundary.test.ts`, `tests/usage/quota-reset-account-key.test.ts`. + +## facade re-export 계약 + +두 facade는 분해 전 `export` 이름을 빠짐없이 다시보낸다. 철자가 바뀌면 소비자 전부가 빨간다. + +routing.ts public 목록 (현재 export): `CodexThreadResolution`, `CodexAffinityMove`, `CodexAffinityReason`, `CodexAffinityDecision`, `CODEX_QUOTA_PROBE_INTERVAL_MS`, `CODEX_FAILURE_WINDOW_MS`, `TERMINAL_SHORT_WINDOW_FRESHNESS_MS`, `CODEX_TRANSIENT_SOFT_AVOID_MS`, `CODEX_THREAD_AFFINITY_IDLE_TTL_MS`, `CODEX_THREAD_AFFINITY_MAX_ENTRIES`, `CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS`, `CODEX_TRANSIENT_AFFINITY_HOLD_MS`, `CodexUpstreamOutcome`, `CodexUpstreamOutcomeClass`, `CodexCooldownSource`, `CodexQuotaScope`, `CodexQuotaRecoveryProbeClaim`, `CodexQuotaRecoveryProbeProof`, `codexQuotaScopeForModel`, `CodexUpstreamOutcomeMeta`, `listLiveCodexAccountIds`, `clearThreadAccountMap`, `clearThreadAccountMapForAccount`, `clearCodexUpstreamHealth`, `clearCodexUpstreamHealthForAccount`, `reconcileCodexRoutingHealth`, `getCodexUpstreamHealth`, `computeCodexUsageScore`, `classifyCodexUpstreamOutcome`, `parseRetryAfterMs`, `parseResetCooldownMs`, `computeQuotaCooldown`, `computeQuotaCooldownUntil`, `tryAcquireCodexQuotaProbeLease`, `canAcquireCodexQuotaProbeLease`, `claimDueCodexQuotaRecoveryProbes`, `ManualResetCooldownClaim`, `claimManualResetCooldowns`, `ManualResetRefreshLineage`, `settleManualResetCooldown`, `settleCodexQuotaRecoveryProbe`, `tryAcquireCodexQuotaScopeProbeLease`, `canAcquireCodexQuotaScopeProbeLease`, `releaseCodexQuotaProbeLease`, `releaseCodexQuotaScopeProbeLease`, `resetCodexRoutingForManualSelection`, `getCodexAccountCooldownUntil`, `getCodexAccountHealthSnapshot`, `getCodexQuotaHealthSnapshot`, `isCodexAccountInCooldown`, `clearCodexAccountCooldown`, `getCodexAccountSoftAvoidUntil`, `isCodexAccountSoftAvoided`, `isCodexAccountPlanExcluded`, `debugCodexAffinityGenerations`, `handOffThreadAffinityGeneration`, `getPoolAccountPlan`, `pickLowestUsageCodexAccount`, `pickAlternateCodexAccount`, `getEffectiveActiveCodexAccountId`, `isEffectiveCodexAccountPinned`, `reconcileCodexActiveAfterExclusion`, `resolveCodexAccountForThread`, `previewCodexAccountForRequest`, `resolveCodexAccountForThreadDetailed`, `recordCodexUpstreamOutcome`, `formatCodexProviderForLog`. + +quota.ts public 목록: `ProviderQuota` 타입 re-export, `QUOTA_RESPONSE_MAX_BYTES`, `setProviderQuotaBeforePublishForTests`, `ProviderQuotaReport`, `ProviderQuotaResponse`, `clearProviderQuotaCache`, `readProviderQuotaJsonForTests`, `parseOllamaCloudQuota`, `parseZaiQuotaLimits`, `publishKeyReportForTests`, `parseXaiCreditsResponse`, `ProviderAccountQuota`, `supportsPerAccountQuota`, `providerOAuthAccountQuotaMode`, `getCachedProviderAccountQuota`, `setCachedProviderAccountQuotaForTests`, `parseAnthropicRateLimitHeaders`, `recordAnthropicAccountQuotaFromHeaders`, `hasPassiveAccountQuota`, `recordPassiveAccountQuota`, `readPassiveProviderAccountQuotas`, `sweepExpiredProviderAccountQuotaRows`, `reconcileProviderAccountQuotaRows`, `resetProviderQuotaReconcileStateForTests`, `clearAccountQuotaCache`, `fetchProviderAccountQuotas`, `isCanonicalAntigravityQuotaUrl`, `setAntigravityAccountQuotaTransportForTests`, `fetchAntigravityUsageQuota`, `providerApiKeyQuotaMode`, `fetchProviderApiKeyQuotas`, `providerObservationAccountKeyForTests`, `flushProviderQuotaObservationsForTests`, `fetchProviderQuotaReports`. + +## 사이클 완료 조건 + +- `src/codex/routing.ts` ≤ 1,999, `src/providers/quota.ts` ≤ 1,999, 새 모듈 전부 ≤ 1,999 +- 싱글턴이 표의 소유 파일에만 존재. 인자로 새는 상태 0 +- 함정 7항 미발생 +- 오라클 4건이 새 경로를 읽거나, 읽지 않아도 되는 이유(SEAMS facade 잔류)가 이 문서와 일치 +- structure 백틱 6곳이 구현 파일과 모순 없음 +- layout.json / test-layout-expected.json 변경 없음 +- 로컬 스위트 NOT RUN. 레인 tip hosted CI exact-head 녹색 후 D에서 ratchet:update + +## 정정 모아보기 + +1. `promoteAccountId` 재사용 행 3369·3420 → **3371·3422**. +2. `recordCodexUpstreamOutcome`는 `promoteActiveCodexAccount` 단독 호출자가 아님. 단독 제약은 429 링 이중 전진 방지로 좁힘. +3. vendor-probes-key ~900 → **~1,230** (kimi/command+selector 포함). +4. antigravity ~230 → **~284**. +5. account-cache ~520 → **~634**. +6. report-cache ~230 → **~287**. +7. routing 잔여 ~900 → preview/rebind/transient 포함 **~1,250**. +8. quota 잔여 ~700(`fetchProviderQuotaReports` 3207) → reports 본체는 107줄, 잔여 합 **~320**. +9. thread-affinity ~600 → 순환 import 제외 **~480**. +10. selection ~560 → apply*/priority 포함 **~680**. +11. cooldown-math ~260 → Meta 타입 포함 **~300**. +12. detector 주석 `quota.ts:279`는 현재 잘못된 행. 0-survive는 `quota.ts:1686-1687` vs `codex/quota.ts:184`. +13. core.ts account-failover 퍼밋은 1506이 아니라 **1511-1515**. +14. `codexQuotaAvoidUntil`는 cooldown-math가 아니라 health-store. +15. 새 디렉터리는 `src/codex/routing/`, `src/providers/quota/` (기존 `src/routing/`, `src/quota/` 금지). + diff --git a/devlog/_plan/260914_godfile_round2/050_phase5_config.md b/devlog/_plan/260914_godfile_round2/050_phase5_config.md new file mode 100644 index 0000000000..0c161f81d0 --- /dev/null +++ b/devlog/_plan/260914_godfile_round2/050_phase5_config.md @@ -0,0 +1,361 @@ +# 050 — 사이클 5: src/config.ts 파사드 분해 + +src/config.ts 4,707줄이 스키마·로드 열화·salvage·잠금·치환 쓰기·라이브 재결합을 한 파일에 들고 있어 래칫 이후에도 2,000줄을 넘긴다. 이 문서는 그 파일을 5개 PR로 줄이는 복붙 가능한 이동 계약이다. 구현자는 아래 원본 행을 새 리프로 옮기고 파사드가 기존 export 이름을 그대로 다시보내며, 소비자는 import 경로를 건드리지 않는다. create-only 경로 initializePersistedConfigIfMissing와 치환 경로 saveConfig는 잔여 파사드에 함께 남기되 공용 writeConfigBytes(mode)로 합치지 않고, 경고 메모 세 값은 인자로 넘기지 않으며, configSchema는 키 그룹으로 쪼개지 않는다. 초안의 PR 묶음은 salvage가 configSchema를, diagnostics가 salvage와 load-degrade를, live-reconcile이 persistConfigUnlocked를 쓰기 때문에 기술 의존성 순서로 재배치한다. + +브랜치 `codex/m2k-l6-config`, base는 사이클 4 `codex/m2k-l5-routing-quota`. 순수 이동, 동작 변경 없음. 로컬 스위트·typecheck·build는 이 단위 금지(hosted CI). 새 테스트 파일을 만들지 않으므로 layout.json과 test-layout-expected.json은 등록하지 않는다. 기준 트리 origin/dev 4f788f916e, 파일 4,707줄. 열린 PR 충돌은 순서에서 제외한다. + +## 정정 (초안 대비, 이 트리에서 재계측) + +정정: warnedConfigFallbacks 블록은 440-450(11줄)이지 leaf-validators 440-1247에 들어 있지 않다. leaf-validators 본체는 452-1247(796줄)이고 452-458은 retryOn429PolicySchema 주석이라 schema로 간다. + +정정: feature-flags 본체는 3836-3890(55줄)이다. 초안 3836-3905는 3892-3904 live-reconcile 배너 주석을 잘못 포함했다. 그 주석은 live-reconcile.ts로 이동한다. + +정정: mutation-lock 본체는 3400-3626(227줄)이다. 초안 3400-3665는 persistConfigUnlocked(3628-3664, 37줄)를 포함하며, 그 함수는 잠금 모듈로 이동 금지이므로 범위에서 뺀다. + +정정: load-degrade는 1823-2578(756줄)만이 아니다. loadConfig가 호출하는 sanitizeAliasesForLoad·sanitizeModelDisplayNamesForLoad·withRefreshedCostOverlays(2703-2775, 73줄)가 loadConfig(2579-2701) 뒤에 떨어져 있다. 세 함수는 load-degrade로 옮기고 loadConfig는 파사드 오케스트레이터로 잔류한다. + +정정: salvageConfigCandidate는 configSchema.safeParse를 4588과 4612에서 호출한다. schema 추출 전에 salvage를 빼면 salvage → 파사드 → salvage 순환이 생긴다. 초안 PR2 salvage+warn-memo / PR5 schema+load-degrade 순서는 불가능하다. + +정정: diagnostics(2777-3399)는 load-degrade 헬퍼, salvageConfigCandidate, configSchema, getDefaultConfig를 쓴다. reconcileLiveConfigFromDisk(4122)는 readConfigDiagnostics()를, saveConfigPreservingClaudeCode(4206)는 configDiagnosticsFromRaw·normalizePersistedClaudeCode·persistConfigUnlocked·withConfigMutationLockSync를 쓴다. 초안 PR3 live-reconcile / PR4 mutation-lock+diagnostics는 순환이다. + +정정: persistConfigUnlocked를 파사드에 남기고 saveConfigPreservingClaudeCode를 live-reconcile로 옮기면 live-reconcile → 파사드 순환이 된다. persistConfigUnlocked·failClosedClientPersistenceError·readRawConfigJson을 src/config/persist-unlocked.ts로 선분리한다. 이는 writeConfigBytes 병합이 아니다. initializePersistedConfigIfMissing는 이 모듈을 import하지 않고 publishInitialConfigNoReplace만 쓴다. + +정정: structure:check unowned는 src/ 1단만 본다(scripts/structure-ssot.ts:515-519). src/config/는 이미 config.md·runtime.md documents에 있어 src/config/schema/ 신설만으로 unowned 실패가 나지는 않는다. area 변경 의무로 config.md가 새 경로를 백틱 인용해야 하고, 백틱을 넣으면 git index에 파일이 있어야 한다. + +정정: tests/usage/user-cost-overlay-live-reconcile.test.ts:113,175,239는 mock.module이 아니라 자식 await import("./src/config.ts")다. mock.module("./src/config.ts")는 tests/service/init-eof.test.ts:190뿐이다(:182는 spread import, :252는 withConfigMutationLockSync 실import). + +정정: ADR-0016:8, ADR-0020:8/10, ADR-0003:8은 역사 기록이라 현재 트리에 맞춰 고치지 않는다. INDEX.md:107은 manifest 생성물이라 손대지 않는다. src/config/ documents가 이미 있어 structure:index도 불필요하다. + +정정: 711-732의 provider-name/provider-validation re-export는 leaf-validators 한가운데 있다. 리프로 가져가지 말고 파사드 상단 블록으로 올린다. + +정정: loadConfig의 수리 병합(2631-2644)과 diagnostics mergeConfigDefaults(2858-2874)는 같은 핀 세 키(subagentModelsVersion, multiAgentMode, multiAgentSurfaceAdvisoryVersion)를 복제한다. load-degrade로 옮길 때 인라인 병합을 mergeConfigDefaults 호출로 치환한다. 핀이 빠지면 v1 서브에이전트 표면이 침묵 수리된다(structure/subagents.md:45-48). + +## create-only 경계 (최우선 보존) + +structure/config.md:14-22 현행: + +> `initializePersistedConfigIfMissing` in `src/config.ts` is the create-only path consumed by +> `src/cli/init.ts`. It rechecks absence under the existing config-mutation lock and publishes through +> `src/config/initialize.ts`: a private descriptor is hardened before secret bytes are written, then +> linked without replacing an occupied destination. Existing invalid or unsafe entries are preserved. +> The initializer never truncates a staged inode or rolls back by unlinking the destination; cleanup +> only removes its own temporary name. ... Ordinary `saveConfig` replacement behavior remains unchanged. + +코드 재확인. initializePersistedConfigIfMissing(3669-3703)는 withConfigMutationLockSync 안에서 observeInitialConfigState()를 재확인한 뒤 publishInitialConfigNoReplace(getConfigPath(), JSON.stringify(...) + newline, io)만 호출한다. atomicWriteFile을 쓰지 않는다. saveConfig(3705-3721)는 withConfigMutationLockSync → persistConfigUnlocked(3628-3664) → 변경 시에만 atomicWriteFile(3659). persistConfigUnlocked 주석(3618-3626)은 잠금 비보유를 계약으로 못 박는다. + +금지: writeConfigBytes(mode). 두 공개 함수는 잔여 src/config.ts에 남긴다. 파사드 상단 import를 빈 줄로 나눠 create-only는 ./config/initialize만, replace는 ./config/persist-unlocked만 보게 한다. + +## 공통 이동 규칙 + +원본 함수 본문을 고치지 않고 잘라 붙인다. 옮긴 공개 심볼은 파사드에서 삭제하고 `export { name } from "./config/…";` 한 줄로 다시보낸다. 내부 심볼은 파사드가 `import { name } from "./config/…";` 한다. 리프는 파사드를 import하지 않는다. specifier는 extensionless. 새 테스트 파일 금지. 리프가 src/lab/를 import하면 tests/lab/core-lab-boundary.test.ts가 실패해야 하며 그 상태로 남기지 않는다. + +## 상태 소유권 + +모듈 수준 let/const/WeakMap/Set은 한 파일만 소유한다. Set 자체를 export하거나 인자로 넘겨 두 번째 참조를 만들지 않는다. + +| 바인딩 | 원본 행 | 소유 | 이유 | +|---|---|---|---| +| warnedConfigFallbacks | 440 | warn-memo.ts | salvage 4474/4674/4686이 기록. 인자로 넘기면 프로세스 1회성 경고가 갈라진다 | +| warnedInheritedFastWireConflicts | 441 | warn-memo.ts | load-degrade 2564가 기록. 동일 | +| lastWarningReconciledGeneration | 442 | warn-memo.ts | reconcileConfigWarningMemos(444-450)와 동거 | +| warnedProxyConfigDiscards | 4363 | proxy-env.ts | applyProxyEnvWith만 사용. warn-memo와 합치지 말 것 | +| claudeCodeBaseline WeakMap | 3906 | live-reconcile.ts | arm/read/save가 같은 파일. 지연 arm은 첫 save 전 hand-edit를 놓친다 | +| liveConfigBaseline WeakMap | 3912 | live-reconcile.ts | 동일 | +| persistedLiveServerBinding WeakMap | 3921 | live-reconcile.ts | 동일 | +| configMutationLockDepth | 3470 | mutation-lock.ts | persist-unlocked로 이동 금지 | +| configMutationDatabase | 3471 | mutation-lock.ts | persist는 DB 핸들을 받지 않는다. bump는 bumpGenerationForCooperatingConfigWrite | +| warnedConfigMutationDirectoryAcl | 3402 | mutation-lock.ts | 동일 | +| persistedConfigMutationBeforeCommitForTests | 3733 | 잔여 파사드 | mutatePersistedConfig(3752)와 동거 | + +warn-memo 공개 API. Set 자체는 export하지 않는다. + + export function reconcileConfigWarningMemos(generation: number): number + export function hasWarnedConfigFallback(configPath: string): boolean + export function markWarnedConfigFallback(configPath: string): void + export function hasWarnedInheritedFastWireConflict(configPath: string): boolean + export function markWarnedInheritedFastWireConflict(configPath: string): void + +has/mark는 현행 Set.has/add 래퍼다. warnConfigRepaired(4474), warnDroppedConfigSections(4674), warnAndBackupInvalidConfig(4686), warnInheritedFastWireConflicts(2564)만 이 API를 쓴다. + +## 하지 말아야 할 분할 + +1. configSchema(1248-1822)를 키 그룹 파일로 쪼개지 않는다. 1424의 passthrough().superRefine((config, ctx) => { 의 addIssue 순서가 schemaDiagnosticsError(2876)와 salvage 로그 문자열을 결정한다. +2. create-only와 saveConfig를 한 writer로 합치지 않는다. +3. persistConfigUnlocked를 mutation-lock.ts에 넣지 않는다. +4. WeakMap 3종을 live-reconcile 밖으로 빼거나 startServer가 아닌 모듈에 arm을 옮기지 않는다. +5. 리프가 ../config를 import하지 않는다. +6. 711-732 re-export를 leaf-validators로 가져가지 않는다. +7. UNSALVAGEABLE_ISSUE_MESSAGES의 CODEX_ACCOUNT_NAMESPACE_COMBO_ALIAS_COLLISION_ERROR를 일반 salvage로 지우지 않는다. 드롭하면 계정 셀렉터가 조용히 통과한다. + +## 모듈 지도 (inclusive 원본 행 → 대상, 본체 줄) + +| 대상 | 원본 | 본체 | 예상 wc | 공개(파사드 재수출 O/X) | +|---|---|---:|---:|---| +| NEW src/config/warn-memo.ts | 440-450 | 11 | 28 | O reconcileConfigWarningMemos. has/mark는 X | +| NEW src/config/openai-tier-backup.ts | 177-439 | 263 | 295 | O 에러 5종, classify/backup/preserve, IO 타입 | +| NEW src/config/feature-flags.ts | 3836-3890 | 55 | 78 | O websocketsEnabled, ultraFastTierEnabled, CATALOG_AUTO_REFRESH_*, isCatalogAutoRefreshEnabled, resolveCatalogAutoRefreshIntervalMs | +| NEW src/config/proxy-env.ts | 4293-4472 | 180 | 215 | O getDefaultConfig, resolveEnvValue, applyProxyEnv, applyProxyEnvWith, codexAutoStartEnabled, CODEX_SHIM_AUTO_RESTORE_ENV, codexShimAutoRestoreEnabled, multiAgentGuidanceEnabled, runtimeRole. 정정: 파일명은 proxy-env이나 초안 범위에 getDefaultConfig가 들어 있다 | +| NEW src/config/schema/leaf-validators.ts | 452-1247 중 711-732 제외 | 774 | 860 | O requestPacingConfigError, providerWebSearchBridgeConfigError, providerModelCostsConfigError, sanitizeModelCostsForDisplay, modelPreferHostedToolsConfigError. 내부 스키마는 형제 export, 파사드 재수출 금지 | +| NEW src/config/schema/config-schema.ts | 1248-1822 | 575 | 640 | X configSchema (현재 unexported. 형제만 export) | +| NEW src/config/load-degrade.ts | 1823-2578 + 2703-2775 | 829 | 900 | O hardenExistingSecret, retryOn429PolicyConfigError. sanitizer/warn/normalize/mergeConfigDefaults는 형제 export | +| NEW src/config/salvage.ts | 4473-4707 | 235 | 275 | O backupInvalidConfig. salvageConfigCandidate·warn*는 형제 export | +| NEW src/config/diagnostics.ts | 2777-3399 | 623 | 690 | O ConfigDiagnostics, subagentDefaultSyncEffective, loopbackCompanionBindError, validateConfigCandidate, readConfigDiagnostics, observeInitialConfigState, ConfigAdmissionSnapshot, readConfigAdmissionSnapshot. configDiagnosticsFromRaw·readConfigFileSnapshot는 형제 export | +| NEW src/config/mutation-lock.ts | 3400-3626 | 227 | 275 | O ConfigMutationLockError, NestedConfigMutationError, prepareConfigMutationDatabasePathForWrite, withConfigMutationLockSync, readConfigGeneration, observeConfigGeneration, readConfigGenerationInCurrentMutationTransaction, bumpConfigGeneration, withExpectedConfigGenerationSync. bumpGenerationForCooperatingConfigWrite는 형제 export | +| NEW src/config/persist-unlocked.ts | 3628-3664 + 3811-3834 + 4156-4172 | 78 | 130 | X persistConfigUnlocked, readRawConfigJson. 파사드 공개 재수출 금지 | +| NEW src/config/live-reconcile.ts | 3892-3904 + 3906-4154 + 4174-4291 | 380 | 450 | O armClaudeCodeBaseline, adoptPersistedProviderIntoLiveConfig, claudeCodeBaselineArmed, reconcileLiveConfigFromDisk, saveConfigPreservingClaudeCode | +| MODIFY src/config.ts 잔여 | 1-176 헤더 + 2579-2701 loadConfig + 3666-3810 init/save/mutate + 재수출 | 444 원본 | 560 | 현행 공개 심볼 전부 | + +잔여 444 = 헤더 176 + loadConfig 123 + init/save/mutate 145. persist를 잔여에 두면 saveConfigPreservingClaudeCode까지 남아 ~720이 된다. persist-unlocked가 ~560을 만든다. + +## 내부 export (파사드 공개 표면을 늘리지 말 것) + +- config-schema.ts: export const configSchema +- leaf-validators.ts: retryOn429PolicySchema, providerConfigSchema, clientConnectionSchema, hubConfigSchema, remoteGuiConfigSchema, runtimeRoleSchema, agentTaskRecoverySchema, quotaResetNotifySchema, catalogAutoRefreshSchema, codexPoolSchema, codexAccountPrioritiesSchema, codexQuotaAutoRefreshSchema, CODEX_ACCOUNT_PIN_PATTERN, configuredCodexPoolAccountIds +- load-degrade.ts: sanitize*ForLoad, warnDegraded*, normalizeApiKeyIds, normalizeClaudeSubagentEffort, normalizeNativeSubagentSync, normalizePersistedClaudeCode, mergeConfigDefaults, inheritedFastWireConflictProviderNames, inheritedFastWireConflictWarning, nativeSubagentSyncDisabledReason, rawClaudeSubagentEffort, isClaudeSubagentEffort, CLAUDE_SUBAGENT_EFFORTS, rawConfigRecord, malformed*, degraded*Warnings, withRefreshedCostOverlays +- salvage.ts: salvageConfigCandidate, warnConfigRepaired, warnDroppedConfigSections, warnAndBackupInvalidConfig +- diagnostics.ts: configDiagnosticsFromRaw, readConfigFileSnapshot +- mutation-lock.ts: bumpGenerationForCooperatingConfigWrite +- persist-unlocked.ts: persistConfigUnlocked, readRawConfigJson + +## 비순환 그래프 + + warn-memo + openai-tier-backup → paths, atomic-write, windows-secret-acl + feature-flags + proxy-env → types, subagent-models, multi-agent-surface, windows-system-proxy + schema/leaf-validators → provider-validation, types, providers/* + schema/config-schema → leaf-validators, combos/types, routing/profile, claude/desktop-profile, account-namespace-match + load-degrade → leaf-validators, warn-memo, provider-validation, fastwire, redact + salvage → config-schema, warn-memo, redact + diagnostics → load-degrade, salvage, config-schema, leaf-validators, proxy-env(getDefaultConfig) + mutation-lock → codex/generation, paths, bun:sqlite, windows-secret-acl, test-home-guard + persist-unlocked → leaf-validators(clientConnectionSchema), rebase-provenance, atomic-write, usage/user-cost-overlays. mutation-lock을 import하지 않음 + live-reconcile → mutation-lock, persist-unlocked, diagnostics, load-degrade(normalizePersistedClaudeCode), rebase-provenance, usage overlays + src/config.ts → 위 전부 재수출 + loadConfig + initializePersistedConfigIfMissing + saveConfig + mutatePersistedConfig + +persist-unlocked가 mutation-lock을 import하지 않는 것이 잠금 비보유 계약이다. 호출자(saveConfig, mutatePersistedConfig, saveConfigPreservingClaudeCode)가 이미 withConfigMutationLockSync 안에 있다. + +## 동반 수정 의무 + +| 항목 | 조치 | +|---|---| +| structure/runtime.md:31 | 파사드 설명을 유지하고 새 리프 파일명을 같은 칸에 백틱. 없는 파일을 백틱하지 말 것(그 PR에서 만든 리프만) | +| structure/config.md:14 | initializePersistedConfigIfMissing in src/config.ts 유지(함수 잔여) | +| structure/config.md:21 | saveConfig 치환이 src/config/persist-unlocked.ts → atomicWriteFile임을 PR4에서 명시. 두 경로 병합 금지 | +| structure/config.md:39 | src/config.ts re-exports 유지 | +| structure/config.md:49 | loader는 src/config.ts. PR2에서 src/config/schema/config-schema.ts, src/config/schema/leaf-validators.ts 백틱 추가 | +| structure/config.md:62 | Env 해석 구현 src/config/proxy-env.ts, 공개 경로는 파사드 | +| structure/config.md:67 | salvage 구현 src/config/salvage.ts | +| structure/config.md:201 | websocketsEnabled 구현 src/config/feature-flags.ts | +| structure/config.md:224 | Zod refinement 소비자 src/config/schema/config-schema.ts | +| structure/config.md:310 | cadence resolver src/config/feature-flags.ts | +| structure/overview.md:47 | OPENCODEX_HOME 공개 경로 src/config.ts 유지(getConfigDir 재수출) | +| structure/subagents.md:45 | getDefaultConfig 공개 src/config.ts, 구현 src/config/proxy-env.ts | +| structure/subagents.md:48 | pin 구현 src/config/load-degrade.ts mergeConfigDefaults | +| structure/providers/openai-tiers.md:326 | classifyOpenAiTierBackup src/config/openai-tier-backup.ts | +| ADR-0016:8, ADR-0020:8/10, ADR-0003:8 | 수정 금지 | +| INDEX.md:107 | 수동 수정 금지. src/config/는 1단에 이미 청구됨 | +| manifest.json | 변경 없음. structure:index 불필요 | +| layout.json / test-layout-expected.json | 등록하지 않음 | + +runtime.md:31은 파사드 한 칸이다. 각 PR에서 그 PR이 만든 리프만 백틱한다. 없는 경로를 백틱하면 structure:check가 git index 기준으로 실패한다. + +## 소스 오라클 (파사드 경로 유지) + +tests/config/config-mutation-lock.test.ts:84,151,395 — pathToFileURL(repoPath("src/config.ts")).href로 자식이 withConfigMutationLockSync를 import. 리프 URL로 바꾸지 마라. :5 정적 import도 ../../src/config. + +tests/codex-integration/codex-config-generation.test.ts:31 — 동일. :25가 bumpConfigGeneration, mutatePersistedConfig, observeConfigGeneration, readConfigGeneration, saveConfig, saveConfigPreservingClaudeCode, withExpectedConfigGenerationSync를 파사드에서 import. + +tests/usage/user-cost-overlay-live-reconcile.test.ts:113,175,239 — 자식 await import("./src/config.ts"). :5 정적 import도 파사드. + +tests/service/init-eof.test.ts:190 — mock.module("./src/config.ts")가 initializePersistedConfigIfMissing를 감싼다. 심볼이 파사드에 있어야 mock가 잡는다. :182는 configApi spread, :252는 withConfigMutationLockSync 실호출. + +tests/config/config-save-boundary.test.ts — src/config.ts 본문을 읽지 않는다. GUARDED_FILES와 server/index.ts의 armClaudeCodeBaseline 리터럴을 읽는다. arm 심볼이 파사드 재수출이면 index.ts 불변. + +## INV 승계 + +INV-WS-01 structure/overview.md:84. Enforced by tests/codex-integration/codex-catalog.test.ts (파일 1행 주석 유지). 구현 모듈 src/config/feature-flags.ts websocketsEnabled. 테스트 import 경로는 파사드. 테스트 파일 이동·개명 금지. 이 파일을 묶는 다른 INV는 없다. + +INV-TESTS-01 — 신규 테스트 없음. config 도메인 match는 layout.json:138-141. 새 테스트가 생기면 tests/config/config-*.test.ts로 두고 두 맵에 explicit 등록한다. 이 사이클은 등록하지 않는다. + +## 소비자 (파사드 유지, write set 밖) + +src/config.ts를 import하는 테스트는 176곳. 경로를 리프로 바꾸지 않는다. 새 리프를 router.ts·server/lifecycle.ts·server/responses/core.ts가 직접 import하지 않는다. + +--- + +## PR 1 — warn-memo + tier-backup + flags + proxy-env + +초안 PR1(tier-backup+flags+proxy-env)에 warn-memo를 당긴다. 세 모듈은 서로 독립이고, warn-memo는 salvage/load-degrade보다 먼저 소유권이 갈라져야 한다. base L5. 비-tip이면 커밋 제목 [skip ci] 가능. + +### NEW + +src/config/warn-memo.ts 예상 28줄. 원본 440-450. 위 has/mark API 추가만 허용. + +src/config/openai-tier-backup.ts 예상 295줄. 원본 177-439. sameBytes·isAlreadyExistsError 비공개 동반. import: node:fs chmodSync/copyFileSync/existsSync/linkSync/readFileSync/truncateSync/unlinkSync/writeFileSync, fsConstants, getConfigPath, nextAtomicTempSequence, isMissingPathError, hardenSecretPath, forgetEphemeralSecretPath. + +src/config/feature-flags.ts 예상 78줄. 원본 3836-3890. import type { OcxConfig } from "../types". + +src/config/proxy-env.ts 예상 215줄. 원본 4293-4472. import: DEFAULT_SUBAGENT_MODELS, SUBAGENT_MODELS_VERSION, MULTI_AGENT_SURFACE_ADVISORY_VERSION, OPENAI_PROVIDER_TIER_VERSION, DEFAULT_APP_OWNED_MEMORY_BUDGET_BYTES, describeProxyForLog, readWindowsSystemProxy, type OcxConfig, type OcxRuntimeRole. + +### MODIFY + +src/config.ts: 177-439 삭제 후 re-export. 440-450 삭제 후 warn-memo import+re-export. 3836-3890 삭제 후 re-export. 4293-4472 삭제 후 re-export. 잔여 load-degrade가 warnedInheritedFastWireConflicts를 쓰므로 warn-memo has/mark로 2564-2565를 치환. salvage 4474/4674/4686도 동일. 본문 로직은 바꾸지 않는다. + +structure/config.md:62,201,310 — 구현 경로 병기. 없는 리프를 미리 적지 말 것. + +structure/providers/openai-tiers.md:326 — classifyOpenAiTierBackup → src/config/openai-tier-backup.ts. + +structure/subagents.md:45 — getDefaultConfig 구현 src/config/proxy-env.ts, 공개는 src/config.ts. + +structure/runtime.md:31 — 이 PR의 네 리프 파일명 백틱. + +### DELETE + +없음. + +### 파사드 re-export (이 PR 후 상단) + + export { reconcileConfigWarningMemos } from "./config/warn-memo"; + export { OpenAiTierBackupCleanupError, OpenAiTierBackupRollbackError, OpenAiTierBackupCollisionError, OpenAiTierRollbackPreserveError, OpenAiTierBackupSecretResidualError, classifyOpenAiTierBackup, backupConfigBeforeOpenAiTierMigration, preserveOpenAiTierRollbackSnapshot, type OpenAiTierBackupIO, type OpenAiTierRollbackPreserveIO } from "./config/openai-tier-backup"; + export { websocketsEnabled, ultraFastTierEnabled, CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS, CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS, isCatalogAutoRefreshEnabled, resolveCatalogAutoRefreshIntervalMs } from "./config/feature-flags"; + export { codexAutoStartEnabled, CODEX_SHIM_AUTO_RESTORE_ENV, codexShimAutoRestoreEnabled, multiAgentGuidanceEnabled, runtimeRole, getDefaultConfig, resolveEnvValue, applyProxyEnv, applyProxyEnvWith } from "./config/proxy-env"; + +### 회귀 + +tests/server/proxy-env.test.ts, tests/config/config-catalog-auto-refresh.test.ts, tests/codex-integration/catalog-auto-refresh-scheduler.test.ts, tests/codex-integration/codex-catalog.test.ts (INV-WS-01), tests/codex-integration/codex-shim-autorestore.test.ts, tests/service/init-backup-cleanup.test.ts, tests/adapters/openai/openai-provider-option-startup.test.ts, tests/config/config-load-degrade.test.ts. + +예상: config.ts 4,707-263-11-55-180+재수출≈20 ≈ 4,218. + +--- + +## PR 2 — schema + +salvage·load-degrade·diagnostics가 configSchema를 쓰므로 그들보다 앞선다. 초안 PR5를 여기로 당긴다. + +### NEW + +src/config/schema/leaf-validators.ts 예상 860줄. 원본 452-1247에서 711-732를 뺀다. 711-732는 파사드 상단 기존 provider-name/provider-validation re-export와 합친다. + +src/config/schema/config-schema.ts 예상 640줄. 원본 1248-1822 그대로. 첫 import는 ./leaf-validators의 스키마들. export const configSchema. 파사드는 configSchema를 재수출하지 않는다. + +### MODIFY + +src/config.ts: 452-1822 삭제. import { configSchema } from "./config/schema/config-schema"; (loadConfig·salvage·diagnostics가 아직 파사드에 있으면 로컬 바인딩). 711-732를 상단으로 이동. + +structure/config.md:49 근처에 src/config/schema/leaf-validators.ts와 src/config/schema/config-schema.ts 백틱. :224에 schema 리프가 provider-validation을 소비한다고 적는다. + +### 회귀 + +tests/config/config-load-degrade.test.ts, tests/config/model-pinned-effort-config.test.ts, tests/server/config.test.ts, tests/routing/routing-profile.test.ts, tests/routing/routing-compatibility-boundaries.test.ts, tests/web-search/web-search-passthrough-bridge.test.ts, tests/providers/provider-cost-overlay-config.test.ts. + +함정: superRefine 본문이 원본 1248-1822와 export/import 외 일치. git diff로 확인. + +예상: config.ts ≈ 4,218-774-575+import ≈ 2,890. + +--- + +## PR 3 — salvage + load-degrade + +초안 PR2 salvage를 schema 뒤로, 초안 PR5 load-degrade를 같은 PR로 모은다. 둘 다 configSchema와 warn-memo가 필요하다. + +### NEW + +src/config/salvage.ts 예상 275줄. 원본 4473-4707. import: configSchema from ./schema/config-schema, has/markWarnedConfigFallback from ./warn-memo, redactSecretString, z from zod/v4, copyFileSync/chmodSync/existsSync, CODEX_ACCOUNT_NAMESPACE_COMBO_ALIAS_COLLISION_ERROR. + +src/config/load-degrade.ts 예상 900줄. 원본 1823-2578 + 2703-2775. import: leaf schemas, warn-memo inherited API, provider-validation, fastwire, redact, MODEL_ALIAS_PATTERN, MODEL_DISCOVERY_MAX_MODELS. + +### MODIFY + +src/config.ts: 1823-2578, 2703-2775, 4473-4707 삭제. loadConfig(2579-2701) 잔류. 2631-2644 인라인 병합을 mergeConfigDefaults(parsed) 호출로 치환. + +structure/config.md:67 — salvage 구현 src/config/salvage.ts. +structure/subagents.md:48 — pin 구현 src/config/load-degrade.ts mergeConfigDefaults. + +재수출: hardenExistingSecret, retryOn429PolicyConfigError from load-degrade. backupInvalidConfig from salvage. + +### 회귀 + +tests/config/config-load-degrade.test.ts, tests/config/config-user-edits.test.ts, tests/routing/fastwire-policy.test.ts, tests/server/config.test.ts, tests/config/settings-stream-mode.test.ts. + +예상: config.ts ≈ 2,890-829-235+import ≈ 1,850. + +--- + +## PR 4 — mutation-lock + persist-unlocked + diagnostics + +diagnostics는 salvage·load-degrade·schema·getDefaultConfig가 필요하다. persist-unlocked는 clientConnectionSchema가 필요하다. mutation-lock은 독립이나 persist를 잠금 모듈에 넣지 않기 위해 같은 PR에서 persist-unlocked를 만든다. + +### NEW + +src/config/mutation-lock.ts 예상 275줄. 원본 3400-3626. persistConfigUnlocked 주석 3618-3626은 persist-unlocked.ts로 옮긴다. + +src/config/persist-unlocked.ts 예상 130줄. 본문 순서: readRawConfigJson(4156-4172), failClosedClientPersistenceError(3811-3834), persistConfigUnlocked(3628-3664). mutation-lock을 import하지 않음. + +src/config/diagnostics.ts 예상 690줄. 원본 2777-3399. import: getDefaultConfig from ./proxy-env, salvageConfigCandidate from ./salvage, load-degrade 헬퍼, configSchema, leaf-validators 스키마. + +### MODIFY + +src/config.ts: 2777-3399, 3400-3626, 3628-3664, 3811-3834, 4156-4172 삭제. + +initializePersistedConfigIfMissing(3669-3703)와 saveConfig(3705-3721)는 잔류. 상단 import를 물리적으로 분리한다. + + // create-only path — never persist-unlocked / atomicWriteFile + import { publishInitialConfigNoReplace, type InitialConfigPublicationIO } from "./config/initialize"; + import { observeInitialConfigState } from "./config/diagnostics"; + + // replace path — never publishInitialConfigNoReplace + import { persistConfigUnlocked } from "./config/persist-unlocked"; + + import { withConfigMutationLockSync, bumpGenerationForCooperatingConfigWrite } from "./config/mutation-lock"; + +structure/config.md:14-22 — 치환 쓰기가 persist-unlocked.ts의 persistConfigUnlocked → atomicWriteFile임을 명시. 병합 금지. +structure/runtime.md:31 — mutation-lock.ts, persist-unlocked.ts, diagnostics.ts 백틱. + +재수출: mutation-lock 공개 심볼, diagnostics 공개 심볼. persistConfigUnlocked는 재수출하지 않는다. + +### 회귀 + +tests/config/config-mutation-lock.test.ts (오라클 :84 :151 :395), tests/codex-integration/codex-config-generation.test.ts:31, tests/codex-integration/codex-admission-primitives.test.ts, tests/config/config-load-degrade.test.ts, tests/server/loopback-listener-admission.test.ts, tests/service/init-eof.test.ts:190. + +예상: config.ts ≈ 1,850-623-227-37-24-17+import ≈ 950. + +--- + +## PR 5 — live-reconcile (레인 tip) + +diagnostics·persist-unlocked·mutation-lock·load-degrade가 필요하다. 이 PR이 tip이므로 커밋 제목에 [skip ci]를 붙이지 않는다. + +### NEW + +src/config/live-reconcile.ts 예상 450줄. 원본 3892-3904 주석 + 3906-4154 + 4174-4291. + +import: withConfigMutationLockSync, bumpGenerationForCooperatingConfigWrite from ./mutation-lock; persistConfigUnlocked, readRawConfigJson from ./persist-unlocked; configDiagnosticsFromRaw, readConfigDiagnostics from ./diagnostics; normalizePersistedClaudeCode from ./load-degrade. 파사드를 import하지 않는다. + +### MODIFY + +src/config.ts: 3892-4154, 4174-4291 삭제. armClaudeCodeBaseline, adoptPersistedProviderIntoLiveConfig, claudeCodeBaselineArmed, reconcileLiveConfigFromDisk, saveConfigPreservingClaudeCode를 live-reconcile에서 재수출. + +structure/config.md에 live-reconcile WeakMap 소유 한 문장. runtime.md:31에 live-reconcile.ts 백틱. + +### 잔여 파사드 골격 + +loadConfig(2579-2701), initializePersistedConfigIfMissing(3669-3703), saveConfig(3705-3721), mutatePersistedConfig(3752-3810), persistedConfigMutationBeforeCommitForTests(3733)와 setter(3736). atomicWriteFile은 initialize에 없다. + +### 회귀 + +tests/config/config-user-edits.test.ts, tests/config/config-save-boundary.test.ts, tests/usage/user-cost-overlay-live-reconcile.test.ts:113,175,239, tests/codex-integration/codex-config-generation.test.ts, tests/lab/core-lab-boundary.test.ts. + +예상: live-reconcile 450, config.ts ≈ 560. wc -l src/config.ts src/config/*.ts src/config/schema/*.ts 전부 1,999 이하. + +## 수락 기준 + +1. src/config.ts ≤ 1,999, 새 모듈 전부 ≤ 1,999. +2. initializePersistedConfigIfMissing가 persist-unlocked를 import하지 않고, persist-unlocked가 initialize를 import하지 않는다. atomicWriteFile은 save 경로에만 있다. +3. configSchema superRefine 본문이 원본과 동일(export/import 제외). 키 그룹 분할 없음. +4. warned* 세 값이 warn-memo.ts에만 있다. salvage와 load-degrade가 has/mark만 호출한다. +5. WeakMap 세 개가 live-reconcile.ts에만 있고 armClaudeCodeBaseline이 liveConfigBaseline과 claudeCodeBaseline을 함께 set한다. +6. 오라클 4개가 계속 repoPath("src/config.ts") 또는 import("./src/config.ts") 또는 mock.module("./src/config.ts")를 쓴다. +7. INV-WS-01 테스트 경로 불변. layout.json 불변. ADR 3개 불변. INDEX.md 수동 편집 없음. +8. 공개 export 집합이 PR 전후 동일. persistConfigUnlocked와 configSchema를 파사드 공개 표면에 추가하지 않는다. + diff --git a/package.json b/package.json index cc0c690747..6d595e1312 100644 --- a/package.json +++ b/package.json @@ -53,6 +53,7 @@ "skill:surface:check": "bun scripts/generate-ocx-skill-surface.ts --check", "structure:index": "bun scripts/structure-ssot.ts --fix", "structure:check": "bun scripts/structure-ssot.ts", + "ratchet:update": "bun scripts/file-size-ratchet.ts --update", "generate:model-metadata": "bun scripts/generate-model-metadata.ts", "build:gui": "cd gui && bun install --frozen-lockfile && bun run build && cd .. && bun run prepare:package", "build:remote-workspace-helper": "cargo build --release --locked --manifest-path native/remote-workspace-helper/Cargo.toml", diff --git a/scripts/file-size-ratchet.ts b/scripts/file-size-ratchet.ts new file mode 100644 index 0000000000..d524aff7c5 --- /dev/null +++ b/scripts/file-size-ratchet.ts @@ -0,0 +1,184 @@ +import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { extname, join, resolve } from "node:path"; + +export const THRESHOLD = 2000; +export const BASELINE_REL = "tests/fixtures/file-size-baseline.json"; + +export const SCAN_EXTENSIONS = new Set([ + ".ts", + ".tsx", + ".js", + ".cjs", + ".mjs", + ".json", + ".css", + ".md", + ".yml", + ".yaml", + ".sh", +]); + +export const EXCLUDED_PREFIXES = [ + "devlog/", + "assets/", + "docs-site/public/", + "docs-site/src/assets/", + "gui/dist/", +] as const; + +export const EXCLUDED_EXACT = new Set(["bun.lock", "gui/dist"]); + +export const GENERATED_PATHS = [ + "scripts/model-metadata.source.json", + "src/adapters/cursor/gen/agent_pb.ts", + "gui/src/i18n/de.ts", + "gui/src/i18n/en.ts", + "gui/src/i18n/fr.ts", + "gui/src/i18n/ja.ts", + "gui/src/i18n/ko.ts", + "gui/src/i18n/ru.ts", + "gui/src/i18n/tr.ts", + "gui/src/i18n/zh.ts", + "gui/src/i18n/zh-TW.ts", + "docs-site/src/data/frontier-benchmarks.json", +] as const; + +export type Verdict = + | "NEW_OVERSIZED" + | "GREW" + | "SHRANK" + | "GENERATED" + | "UNCHANGED" + | "NEW_OK"; + +export type Baseline = { + generated: string[]; + files: Record; +}; + +export type FileSize = { + path: string; + lines: number; +}; + +export type Evaluation = FileSize & { + verdict: Verdict; +}; + +export function countLines(text: string): number { + return text.split("\n").length - (text.endsWith("\n") ? 1 : 0); +} + +export function isScannedPath(path: string): boolean { + if (EXCLUDED_EXACT.has(path)) return false; + if (EXCLUDED_PREFIXES.some((prefix) => path.startsWith(prefix))) return false; + return SCAN_EXTENSIONS.has(extname(path)); +} + +export function evaluate(files: FileSize[], baseline: Baseline): Evaluation[] { + const generated = new Set(baseline.generated); + return files.map((file) => { + if (generated.has(file.path)) return { ...file, verdict: "GENERATED" }; + const cap = baseline.files[file.path]; + if (cap === undefined) { + return { ...file, verdict: file.lines >= THRESHOLD ? "NEW_OVERSIZED" : "NEW_OK" }; + } + if (file.lines > cap) return { ...file, verdict: "GREW" }; + if (file.lines < cap) return { ...file, verdict: "SHRANK" }; + return { ...file, verdict: "UNCHANGED" }; + }); +} + +export function isOffender(row: Evaluation): boolean { + return row.verdict === "NEW_OVERSIZED" || row.verdict === "GREW"; +} + +export function gitLsFiles(repoRoot: string): string[] { + const result = Bun.spawnSync(["git", "ls-files"], { cwd: repoRoot }); + if (result.exitCode !== 0) { + throw new Error(`git ls-files failed: ${new TextDecoder().decode(result.stderr)}`); + } + return new TextDecoder() + .decode(result.stdout) + .split("\n") + .map((line) => line.trim()) + .filter(Boolean); +} + +export function scanRepo(repoRoot: string): FileSize[] { + const out: FileSize[] = []; + for (const path of gitLsFiles(repoRoot)) { + if (!isScannedPath(path)) continue; + out.push({ path, lines: countLines(readFileSync(join(repoRoot, path), "utf8")) }); + } + return out; +} + +export function loadBaseline(text: string): Baseline { + const parsed = JSON.parse(text) as Baseline; + if ( + !parsed + || typeof parsed !== "object" + || !Array.isArray(parsed.generated) + || typeof parsed.files !== "object" + || parsed.files === null + || Array.isArray(parsed.files) + ) { + throw new Error("invalid file-size baseline"); + } + return parsed; +} + +function sortRecord(input: Record): Record { + return Object.fromEntries( + Object.entries(input).sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0)), + ); +} + +export function updateBaseline(current: FileSize[], baseline: Baseline, seed: boolean): Baseline { + const now = new Map(current.map((file) => [file.path, file.lines] as const)); + const files: Record = {}; + for (const [path, cap] of Object.entries(baseline.files)) { + const lines = now.get(path); + if (lines === undefined) continue; + files[path] = Math.min(cap, lines); + } + if (seed) { + const generated = new Set(baseline.generated); + for (const [path, lines] of now) { + if (generated.has(path) || lines < THRESHOLD || files[path] !== undefined) continue; + files[path] = lines; + } + } + return { generated: [...baseline.generated], files: sortRecord(files) }; +} + +export function formatOffenders(rows: Evaluation[]): string { + return rows + .filter(isOffender) + .map((row) => `${row.verdict} ${row.path} ${row.lines}`) + .join("\n"); +} + +if (import.meta.main) { + const repoRoot = resolve(import.meta.dir, ".."); + const baselinePath = join(repoRoot, BASELINE_REL); + const existed = existsSync(baselinePath); + const baseline: Baseline = existed + ? loadBaseline(readFileSync(baselinePath, "utf8")) + : { generated: [...GENERATED_PATHS], files: {} }; + const current = scanRepo(repoRoot); + if (process.argv.includes("--update")) { + const next = updateBaseline(current, baseline, !existed); + writeFileSync(baselinePath, `${JSON.stringify(next, null, 2)}\n`); + console.log(`wrote ${BASELINE_REL} (${Object.keys(next.files).length} caps)`); + process.exit(0); + } + const offenders = evaluate(current, baseline).filter(isOffender); + if (offenders.length > 0) { + console.error("file-size ratchet failed:"); + console.error(formatOffenders(offenders)); + process.exit(1); + } + console.log("file-size ratchet passed"); +} diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 205beca317..abe1e8db67 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -704,6 +704,7 @@ "fastwire-policy.test.ts": "routing", "featherless-provider.test.ts": "providers", "fetch-header-timeout.test.ts": "server", + "file-size-ratchet.test.ts": "ci-workflows", "fixture-dir-uniqueness.test.ts": "ci-workflows", "flash-route-image-modalities.test.ts": "providers", "format-result.test.ts": "web-search", diff --git a/src/codex/catalog/auto-review.ts b/src/codex/catalog/auto-review.ts new file mode 100644 index 0000000000..3487272b84 --- /dev/null +++ b/src/codex/catalog/auto-review.ts @@ -0,0 +1,507 @@ +import { redactSecretString } from "../../lib/redact"; +import type { OcxConfig } from "../../types"; +import { encodeRoutedModelId } from "../../providers/slug-codec"; +import { canonicalAutoReviewModelKey, isValidAutoReviewModel as isValidAutoReviewTarget } from "../../config/provider-validation"; +import { readConfiguredAutoReviewModel } from "./parsing"; +import type { RawEntry } from "./parsing"; +import { configuredCatalogEntry } from "./subagent-roster"; + +const AUTO_REVIEW_ROOT_MARKER = "opencodex_auto_review_root"; + +interface RootAutoReviewStamp { + slug: string; + original: string | null; + applied: string; +} + +function rootAutoReviewStamp(entry: RawEntry): RootAutoReviewStamp | undefined { + const value = entry[AUTO_REVIEW_ROOT_MARKER]; + if (!value || typeof value !== "object" || Array.isArray(value)) return undefined; + const stamp = value as Record; + if (stamp.slug !== entry.slug || typeof stamp.slug !== "string" + || typeof stamp.applied !== "string" + || (stamp.original !== null && typeof stamp.original !== "string")) return undefined; + return stamp as unknown as RootAutoReviewStamp; +} + + +/** True when the value is a valid Codex catalog auto-review selector. */ +export function isValidAutoReviewModel(value: unknown): value is string { + return isValidAutoReviewTarget(value); +} + +export type AutoReviewModelOverrideResult = "absent" | "applied" | "invalid" | "unresolved"; + +/** True when a catalog row was synthesized by opencodex instead of coming from upstream. */ +function isRoutedCatalogEntry(entry: RawEntry): boolean { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + return slug.includes("/") + || (typeof entry.description === "string" && entry.description.startsWith("Routed via opencodex → ")); +} + +/** Restore an owned native value, retaining provenance to avoid legacy reclassification. */ +function clearAutoReviewOverrideValue(entry: RawEntry): void { + const stamp = rootAutoReviewStamp(entry); + if (stamp) { + if (entry.auto_review_model_override === stamp.applied) entry.auto_review_model_override = stamp.original; + } else { + entry.auto_review_model_override = null; + delete entry[AUTO_REVIEW_ROOT_MARKER]; + } +} + +/** + * Legacy whole-catalog root stamp: releases before AUTO_REVIEW_ROOT_MARKER wrote root stamps that + * are textually identical to an upstream value, so the only way to recognize one is the uniform + * signature the no-provider path relies on — a single value that a routed row also carries. + * Returns the stamped values when the observed rows match that shape. + */ +function legacyRootStampValues(observedModels: readonly RawEntry[]): ReadonlySet | undefined { + if (observedModels.some(entry => entry?.[AUTO_REVIEW_ROOT_MARKER] !== undefined)) return undefined; + const configuredValues = new Set(observedModels.flatMap(entry => { + const value = entry?.auto_review_model_override; + return typeof value === "string" && value.trim() ? [value] : []; + })); + const globalStamp = configuredValues.size === 1 + && observedModels.some(entry => { + const value = entry.auto_review_model_override; + return isRoutedCatalogEntry(entry) + && typeof value === "string" + && value.trim().length > 0 + && configuredValues.has(value); + }) + && observedModels.every(entry => { + const value = entry?.auto_review_model_override; + return value === null + || value === undefined + || (typeof value === "string" && configuredValues.has(value)); + }); + return globalStamp ? configuredValues : undefined; +} + +/** + * Sweep legacy root stamps off the rows a root removal owns, before provider plans land. + * + * Root removal reaches marker-tagged native rows on its own, but a catalog written before the + * marker only carries the legacy signature — and provider stamping rewrites that signature before + * the root pass could read it, so the sweep has to run first. + */ +function clearLegacyRootStamps(models: readonly RawEntry[], sourceModels: readonly RawEntry[] = []): void { + const legacyStamp = legacyRootStampValues([...models, ...sourceModels]); + if (legacyStamp === undefined) return; + for (const entry of models) { + if (!entry || typeof entry !== "object") continue; + const current = entry.auto_review_model_override; + if (entry[AUTO_REVIEW_ROOT_MARKER] === undefined + && typeof current === "string" && legacyStamp.has(current)) clearAutoReviewOverrideValue(entry); + } +} + +/** + * Clear the root selector from every row this path owns: routed rows, rows stamped by a release + * that writes the provenance marker, and the legacy whole-catalog stamp that predates it. + */ +function clearAutoReviewModelOverride( + models: readonly RawEntry[], + sourceModels: readonly RawEntry[] = [], +): void { + const legacyStamp = legacyRootStampValues([...models, ...sourceModels]); + for (const entry of models) { + if (!entry || typeof entry !== "object") continue; + const current = entry.auto_review_model_override; + if (isRoutedCatalogEntry(entry) + || (entry[AUTO_REVIEW_ROOT_MARKER] === true || rootAutoReviewStamp(entry) !== undefined) + || (legacyStamp !== undefined && typeof current === "string" && legacyStamp.has(current))) { + clearAutoReviewOverrideValue(entry); + } + } +} + +/** Warn once about a malformed or unresolvable root auto-review selector. */ +function warnAutoReviewModelDiagnostic( + reason: "invalid" | "unresolved", + configured: string, +): void { + const safeConfigured = JSON.stringify(redactSecretString(configured)); + const detail = reason === "unresolved" + ? "the selector was not found in the final catalog" + : "the selector format is invalid"; + console.warn( + `[opencodex] auto_review_model ${detail} (${safeConfigured}); preserving normal upstream auto-review behavior.`, + ); +} + +/** Warn once about a malformed or unresolvable provider-scoped auto-review selector. */ +function warnProviderAutoReviewModelDiagnostic( + reason: "invalid" | "unresolved", + provider: string, + configured: string, +): void { + const safeProvider = JSON.stringify(redactSecretString(provider)); + const safeConfigured = JSON.stringify(redactSecretString(configured)); + const detail = reason === "unresolved" + ? "the selector was not found in the final catalog" + : "the selector format is invalid"; + console.warn( + `[opencodex] auto_review_model for provider ${safeProvider} ${detail} (${safeConfigured}); using the next valid provider/root selector or upstream behavior.`, + ); +} + +/** + * Note once when a bare selector resolves to a row outside the provider it was configured on. + * + * That is how a native model is named as a reviewer, so it stays usable, but a mistyped target must + * not be silent: the operator sees which catalog row actually supplies the reviewer. + */ +function warnProviderAutoReviewForeignTarget(provider: string, configured: string, target: string): void { + const safeProvider = JSON.stringify(redactSecretString(provider)); + const safeConfigured = JSON.stringify(redactSecretString(configured)); + const safeTarget = JSON.stringify(redactSecretString(target)); + console.warn( + `[opencodex] auto_review_model for provider ${safeProvider} (${safeConfigured}) resolved to ${safeTarget}, which is not a row of that provider; that catalog row supplies the reviewer.`, + ); +} + +/** Preserve native upstream overrides and the root-derived provenance marker from source rows. */ +function preserveNativeAutoReviewModelOverrides( + models: readonly RawEntry[], + sourceModels: readonly RawEntry[], +): void { + const existing = new Map(); + for (const entry of sourceModels) { + const slug = typeof entry.slug === "string" ? entry.slug : undefined; + const value = entry.auto_review_model_override; + if (!slug || isRoutedCatalogEntry(entry)) continue; + if (typeof value === "string" || value === null) { + existing.set(slug, { value, root: rootAutoReviewStamp(entry) ?? (entry[AUTO_REVIEW_ROOT_MARKER] === true ? true : undefined) }); + } + } + for (const entry of models) { + const slug = typeof entry.slug === "string" ? entry.slug : undefined; + if (!slug || isRoutedCatalogEntry(entry) || !existing.has(slug)) continue; + const saved = existing.get(slug)!; + entry.auto_review_model_override = saved.value; + if (saved.root) entry[AUTO_REVIEW_ROOT_MARKER] = structuredClone(saved.root); + else delete entry[AUTO_REVIEW_ROOT_MARKER]; + } +} + +/** Stamp a root-derived override and mark native rows so later root removal is durable. */ +function stampRootAutoReviewOverride(entry: RawEntry, target: string): void { + if (!isRoutedCatalogEntry(entry)) { + const previous = rootAutoReviewStamp(entry); + const current = entry.auto_review_model_override; + entry[AUTO_REVIEW_ROOT_MARKER] = { + slug: typeof entry.slug === "string" ? entry.slug : "", + original: previous && current === previous.applied + ? previous.original : typeof current === "string" ? current : null, + applied: target, + } satisfies RootAutoReviewStamp; + } else { + delete entry[AUTO_REVIEW_ROOT_MARKER]; + } + entry.auto_review_model_override = target; +} + +/** Stamp a provider-derived override; provider stamps never fall under root removal. */ +function stampProviderAutoReviewOverride(entry: RawEntry, target: string): void { + entry.auto_review_model_override = target; + delete entry[AUTO_REVIEW_ROOT_MARKER]; +} + +/** + * Apply the root Codex auto-review selector to every catalog row, or clear it when the value is + * absent, blank, malformed, or does not resolve against the assembled catalog. + */ +export function applyAutoReviewModelOverride( + models: RawEntry[] | undefined, + autoReviewModel: string | null | undefined, + sourceModels: readonly RawEntry[] = [], +): AutoReviewModelOverrideResult { + if (!models || !Array.isArray(models)) return "absent"; + if (autoReviewModel === null || autoReviewModel === undefined) { + clearAutoReviewModelOverride(models, sourceModels); + return "absent"; + } + const trimmed = autoReviewModel.trim(); + if (!trimmed) { + clearAutoReviewModelOverride(models, sourceModels); + return "absent"; + } + if (!isValidAutoReviewModel(trimmed)) { + clearAutoReviewModelOverride(models, sourceModels); + warnAutoReviewModelDiagnostic("invalid", trimmed); + return "invalid"; + } + if (!configuredCatalogEntry(models, trimmed)) { + clearAutoReviewModelOverride(models, sourceModels); + warnAutoReviewModelDiagnostic("unresolved", trimmed); + return "unresolved"; + } + for (const entry of models) { + if (entry && typeof entry === "object") { + stampRootAutoReviewOverride(entry, trimmed); + } + } + return "applied"; +} + +/** Validated provider-scoped target with both the configured spelling and catalog slug. */ +interface ValidProviderReviewTarget { + configured: string; + target: string; +} + +/** One provider's resolved provider-wide and per-model auto-review targets. */ +interface ProviderReviewPlan { + wide?: ValidProviderReviewTarget; + perModel: Map; +} + +/** Public provider namespace of a routed catalog row, when it has one. */ +function catalogEntryProviderName(entry: RawEntry): string | undefined { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + const slash = slug.indexOf("/"); + return slash > 0 && isRoutedCatalogEntry(entry) ? slug.slice(0, slash) : undefined; +} + +/** Encoded model-id segment of a routed catalog row, when it has one. */ +function catalogEntryModelSegment(entry: RawEntry): string | undefined { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + const slash = slug.indexOf("/"); + return slash > 0 ? slug.slice(slash + 1) : undefined; +} + +/** Case-preserving encoded key used to match per-model override maps. */ +function providerModelKey(modelId: string): string { + return canonicalAutoReviewModelKey(modelId); +} + +/** + * True when another routed row of this provider already carries `alias` as its own model id. + * + * The alias API validates against whatever ids discovery has reported so far, so on a cold start an + * alias can be persisted that later turns out to name a different row. A key using it is then not + * an alternate spelling of the aliased model — it is that row's id — and must not be propagated. + */ +function aliasNamesAnotherRoutedRow(models: readonly RawEntry[], provider: string, alias: string): boolean { + const encoded = encodeRoutedModelId(alias); + return models.some(entry => isRoutedCatalogEntry(entry) + && catalogEntryProviderName(entry) === provider + && catalogEntryModelSegment(entry) === encoded); +} + +/** Resolve one configured target against the assembled catalog; bare values name a model of the same provider. */ +function resolveProviderReviewTarget( + models: readonly RawEntry[], + provider: string, + configuredRaw: unknown, +): { kind: "valid"; value: ValidProviderReviewTarget; foreign?: boolean } | { kind: "invalid"; configured: string } | { kind: "unresolved"; configured: string } | { kind: "absent" } { + if (typeof configuredRaw !== "string") return { kind: "absent" }; + const configured = configuredRaw.trim(); + if (!configured) return { kind: "absent" }; + if (!isValidAutoReviewModel(configured)) return { kind: "invalid", configured }; + const prefix = `${provider}/`; + let match: RawEntry | undefined; + const sameProviderCandidate = (rawModelId: string): RawEntry | undefined => models.find(entry => { + if (!isRoutedCatalogEntry(entry) || typeof entry.slug !== "string" || !entry.slug.startsWith(prefix)) return false; + const segment = catalogEntryModelSegment(entry); + return segment !== undefined && segment === encodeRoutedModelId(rawModelId); + }); + // A bare selector names a model of this provider. A full selector that resolves in the + // assembled catalog already names the exact row, including a same-provider encoded slug. + if (!configured.includes("/")) { + match = sameProviderCandidate(configured); + } + match ??= configuredCatalogEntry(models, configured); + if (!match && configured.startsWith(prefix)) { + match = sameProviderCandidate(configured.slice(prefix.length)); + } + if (!match) { + // A raw model id may itself contain "/" (for example zenmux moonshotai/kimi-k3). + // After the full-selector lookup misses, try that spelling as a same-provider id. + match = sameProviderCandidate(configured); + } + if (!match) return { kind: "unresolved", configured }; + const target = typeof match.slug === "string" ? match.slug : configured; + // A qualified selector may name another provider's row on purpose; only a bare value that lands + // outside this provider is worth reporting. + const foreign = !configured.includes("/") && catalogEntryProviderName(match) !== provider; + return { kind: "valid", value: { configured, target }, ...(foreign ? { foreign: true } : {}) }; +} + +/** Build resolved per-provider plans and emit one diagnostic per bad selector. */ +function buildProviderReviewPlans( + models: readonly RawEntry[], + config: Pick, +): { plans: Map; failure?: "invalid" | "unresolved" } { + const plans = new Map(); + let failure: "invalid" | "unresolved" | undefined; + const warned = new Set(); + const recordFailure = (kind: "invalid" | "unresolved", provider: string, configured: string): void => { + const signature = `${provider}\u0000${configured}`; + if (warned.has(signature)) return; + warned.add(signature); + warnProviderAutoReviewModelDiagnostic(kind, provider, configured); + failure ??= kind; + }; + const recordForeignTarget = (provider: string, configured: string, target: string): void => { + const signature = `${provider}\u0000foreign\u0000${configured}`; + if (warned.has(signature)) return; + warned.add(signature); + warnProviderAutoReviewForeignTarget(provider, configured, target); + }; + for (const [name, provider] of Object.entries(config.providers ?? {})) { + if (provider.autoReviewModel === undefined && provider.autoReviewModelOverrides === undefined) continue; + const plan: ProviderReviewPlan = { perModel: new Map() }; + if (provider.autoReviewModel !== undefined) { + const resolved = resolveProviderReviewTarget(models, name, provider.autoReviewModel); + if (resolved.kind === "valid") { + plan.wide = resolved.value; + if (resolved.foreign) recordForeignTarget(name, resolved.value.configured, resolved.value.target); + } + else if (resolved.kind !== "absent") recordFailure(resolved.kind, name, resolved.configured); + } + if (provider.autoReviewModelOverrides !== undefined) { + for (const [modelId, rawTarget] of Object.entries(provider.autoReviewModelOverrides)) { + const resolved = resolveProviderReviewTarget(models, name, rawTarget); + if (resolved.kind === "valid") { + plan.perModel.set(providerModelKey(modelId), resolved.value); + if (resolved.foreign) recordForeignTarget(name, resolved.value.configured, resolved.value.target); + } else if (resolved.kind !== "absent") { + recordFailure(resolved.kind, name, resolved.configured); + } + } + } + // `modelAliases` publishes a second public name for a model id, and a routed row's slug always + // carries the upstream id — so accept an override key written in either spelling. + for (const [modelId, alias] of Object.entries(provider.modelAliases ?? {})) { + if (typeof alias !== "string" || !alias.trim()) continue; + if (aliasNamesAnotherRoutedRow(models, name, alias)) continue; + const idKey = providerModelKey(modelId); + const aliasKey = providerModelKey(alias); + if (idKey === aliasKey) continue; + const fromId = plan.perModel.get(idKey); + const fromAlias = plan.perModel.get(aliasKey); + if (fromId !== undefined && fromAlias === undefined) plan.perModel.set(aliasKey, fromId); + else if (fromAlias !== undefined && fromId === undefined) plan.perModel.set(idKey, fromAlias); + } + if (plan.wide !== undefined || plan.perModel.size > 0) plans.set(name, plan); + } + return { plans, failure }; +} + +/** Apply or clear the root selector only on rows without a provider stamp. */ +function applyRootSelectorToRemaining( + models: readonly RawEntry[], + rootValue: string | null | undefined, + providerStamped: ReadonlySet, +): AutoReviewModelOverrideResult { + const clearRemaining = (): void => { + for (const entry of models) { + if (!entry || providerStamped.has(entry)) continue; + // Native rows written by releases before the root marker cannot be told apart from upstream + // values once provider stamps diverge. clearLegacyRootStamps sweeps the ones the legacy + // uniform signature still recognizes before provider plans land, because provider stamping + // destroys that signature; a catalog that no longer matches it needs a one-off manual sync. + if (isRoutedCatalogEntry(entry) || entry[AUTO_REVIEW_ROOT_MARKER] === true || rootAutoReviewStamp(entry)) clearAutoReviewOverrideValue(entry); + } + }; + if (rootValue === null || rootValue === undefined) { + clearRemaining(); + return "absent"; + } + const trimmed = rootValue.trim(); + if (!trimmed) { + clearRemaining(); + return "absent"; + } + if (!isValidAutoReviewModel(trimmed)) { + clearRemaining(); + warnAutoReviewModelDiagnostic("invalid", trimmed); + return "invalid"; + } + if (!configuredCatalogEntry(models, trimmed)) { + clearRemaining(); + warnAutoReviewModelDiagnostic("unresolved", trimmed); + return "unresolved"; + } + for (const entry of models) { + if (!entry || providerStamped.has(entry)) continue; + stampRootAutoReviewOverride(entry, trimmed); + } + return "applied"; +} + +/** Provider-aware variant: provider rows win and the root selector is the fallback. */ +export function applyConfiguredAutoReviewModelOverride( + models: RawEntry[] | undefined, + rootAutoReviewModel: string | null | undefined, + config: Pick, + sourceModels: readonly RawEntry[] = [], +): AutoReviewModelOverrideResult { + if (!models || !Array.isArray(models)) return "absent"; + // Runs unconditionally because the sweep only fires on the uniform legacy signature. A resolved + // root selector restamps every row it touches below, so the call is behavior-preserving there; + // with the root absent, invalid, or unresolved those clears are final — which is the point, and + // also the limit: the legacy heuristic cannot tell a root stamp from an identical upstream value. + clearLegacyRootStamps(models, sourceModels); + const { plans, failure } = buildProviderReviewPlans(models, config); + const providerStamped = new Set(); + for (const entry of models) { + if (!entry || typeof entry !== "object") continue; + const provider = catalogEntryProviderName(entry); + if (!provider) continue; + const plan = plans.get(provider); + if (!plan) continue; + const modelSegment = catalogEntryModelSegment(entry); + const perModel = modelSegment === undefined ? undefined : plan.perModel.get(providerModelKey(modelSegment)); + const selected = perModel ?? plan.wide; + if (!selected) continue; + stampProviderAutoReviewOverride(entry, selected.target); + providerStamped.add(entry); + } + const rootResult = applyRootSelectorToRemaining(models, rootAutoReviewModel, providerStamped); + const providerApplied = [...providerStamped].some(entry => typeof entry.auto_review_model_override === "string"); + if (providerApplied) { + if (rootResult === "invalid" || rootResult === "unresolved") return rootResult; + return failure ?? "applied"; + } + return failure ?? rootResult; +} + +/** True when any provider row configures a provider-scoped auto-review selector. */ +function configHasProviderAutoReview(config: Pick): boolean { + return Object.values(config.providers ?? {}).some(provider => + provider.autoReviewModel !== undefined || provider.autoReviewModelOverrides !== undefined); +} + +/** Apply the root Codex auto-review selector after the final catalog merge. */ +export function finalizeAutoReviewModelOverride( + models: RawEntry[] | undefined, + sourceModels: readonly RawEntry[] = [], + config?: Pick, +): AutoReviewModelOverrideResult { + if (models && sourceModels.length > 0) preserveNativeAutoReviewModelOverrides(models, sourceModels); + if (config && configHasProviderAutoReview(config)) { + return applyConfiguredAutoReviewModelOverride(models, readConfiguredAutoReviewModel(), config, sourceModels); + } + return applyAutoReviewModelOverride(models, readConfiguredAutoReviewModel(), sourceModels); +} +/** + * Why an account-gated native model stopped being offered, but only when the answer is one the + * operator can act on. + * + * Suppression is an omission: the row is never built, so there is no catalog entry for a reason + * to ride on and no downstream consumer that could explain it later. #4212's reporter watched + * their models disappear and reasonably concluded the proxy was broken, because every surface + * that changed said nothing about the account that caused it. + * + * Returns `undefined` for the ordinary case — an account that is simply not entitled to a gated + * model. That is the default state for most installations, it is not news, and warning about it + * on every sync would bury the one case that matters. A credential the operator must repair is + * the case that matters, so that is the only one this speaks up about. + * + * Accounts are named with the durable `p`-prefixed log label, the same identifier the dashboard + * shows, never the raw pool id or the email. + */ diff --git a/src/codex/catalog/build-entries.ts b/src/codex/catalog/build-entries.ts new file mode 100644 index 0000000000..2372c599e3 --- /dev/null +++ b/src/codex/catalog/build-entries.ts @@ -0,0 +1,981 @@ +import { CODEX_REASONING_LEVELS } from "../../reasoning-effort"; +import { clearModelCache } from "../model-cache"; +import { routedSlug, slugEquivalenceKey } from "../../providers/slug-codec"; +import { COMBO_NAMESPACE } from "../../combos"; +import { + CODEX_CUSTOM_MODEL_CATALOG_KIND, + CODEX_PROVIDER_MODEL_CATALOG_KIND, + applyMultiAgentMode, + applyNativeOpenAiContextOverride, + catalogModelSlug, + ensureStrictCatalogFields, + isRoutedModelCompatibilityExcluded, + normalizeServiceTiers, +} from "./parsing"; +import type { CatalogModel, MultiAgentMode, RawEntry } from "./parsing"; +import { + CODEX_NATIVE_ALIAS_CATALOG_KIND, + NATIVE_OPENAI_MODELS, + SUPPORTED_NATIVE_OPENAI_SLUGS, + applyNativeVisibility, + isNativeAliasCatalogEntry, + isUnsupportedOpenAiNativeSlug, + shouldUpgradeToUpstreamEntry, + upstreamNativeEntry, + type NativeContextLimitsInput, +} from "./metadata"; +import { resetBundledCatalogCacheForTests } from "./bundled"; +import { isMultiAgentV2Enabled } from "../features"; +import { ensureUltraReasoningLevel, isGpt56NativeSlug } from "./effort"; +import { clearGatherRoutedModelsInflight, lastDropWarnSignature } from "./provider-fetch"; +import { + accountSelectorShadowCollisionWarnings, + clearLastComboCatalogOmissions, + comboCatalogWarningSignatures, + comboMasqueradeCollisionWarnings, + comboUnrestorableShadowWarnings, + openAiApiCollisionWarnings, + resolveSlugAliasCollisions, + slugAliasCollisionWarnings, + warnAccountSelectorShadowedProviderOnce, + warnComboMasqueradeCollisionOnce, + warnComboUnrestorableShadowOnce, +} from "./aggregation"; +import { accountBoundNativeDisplayName, CODEX_ACCOUNT_BOUND_CATALOG_KIND, trustedAccountBoundNativeCatalogSlug } from "./account-models"; +import { NATIVE_RESERVE_MODEL } from "./native-models"; +import { isReserveCatalogProjection, type ReserveCatalogProjection } from "./reserve"; +import { deriveEntry, finishUpstreamNativeEntry, isExactComboCatalogEntry } from "./derive-entry"; +import { PICKER_ORDER_PRIORITY_BASE, SPAWN_PRIORITY_FIELD } from "./subagent-roster"; + +export interface ObservedCatalogEntryBuildInput { + readonly template: RawEntry | null; + readonly gptSlugs: readonly string[]; + readonly goModels: readonly CatalogModel[]; + readonly featured?: readonly string[]; + /** Optional full picker ordering (config.modelPickerOrder); orders non-featured rows. */ + readonly modelPickerOrder?: readonly string[]; + readonly wsEnabled: boolean; + readonly multiAgentMode: MultiAgentMode; + readonly exactComboSlugs: ReadonlySet; + readonly accountSelectors: readonly string[]; + readonly suppressedBareNativeSlugs: ReadonlySet; + readonly disabledNativeAccountSlugs: ReadonlySet; + readonly multiAgentV2Enabled: boolean; + readonly keepNativeChatGptOnV1?: boolean; + readonly openaiContextCap?: NativeContextLimitsInput; + /** Additional native ids to clone under account selectors, without creating bare rows. */ + readonly accountNativeSlugs?: readonly string[]; + /** Per-selector account ids; unknown observations must not be copied to unrelated accounts. */ + readonly accountNativeSlugsBySelector?: ReadonlyMap; + /** Codex-only manual selector metadata; deliberately independent of live permission. */ + readonly reserve?: ReserveCatalogProjection; +} + +/** Build entries with the process-observed Codex feature state. */ +export function buildCatalogEntries( + template: RawEntry | null, + gptSlugs: string[], + goModels: CatalogModel[], + featured?: string[], + wsEnabled = false, + multiAgentMode: MultiAgentMode = "default", + exactComboSlugs: ReadonlySet = new Set(), + accountSelectors: readonly string[] = [], + suppressedBareNativeSlugs: ReadonlySet = new Set(), + disabledNativeAccountSlugs: ReadonlySet = new Set(), + contextCap?: NativeContextLimitsInput, + accountNativeSlugs?: readonly string[], + accountNativeSlugsBySelector?: ReadonlyMap, + keepNativeChatGptOnV1 = false, + modelPickerOrder: readonly string[] = [], +): RawEntry[] { + const entries = buildCatalogEntriesFromObservedState({ + template, + gptSlugs, + goModels, + featured, + modelPickerOrder, + wsEnabled, + multiAgentMode, + exactComboSlugs, + accountSelectors, + suppressedBareNativeSlugs, + disabledNativeAccountSlugs, + multiAgentV2Enabled: isMultiAgentV2Enabled(), + keepNativeChatGptOnV1, + openaiContextCap: contextCap, + accountNativeSlugs, + accountNativeSlugsBySelector, + }); + applyFullModelPickerOrder(entries, modelPickerOrder); + return entries; +} + +/** Build entries solely from caller-observed inputs, with no feature-state filesystem read. */ +export function buildCatalogEntriesFromObservedState({ + template, + gptSlugs, + goModels, + featured, + modelPickerOrder, + wsEnabled, + multiAgentMode, + exactComboSlugs, + accountSelectors, + suppressedBareNativeSlugs, + disabledNativeAccountSlugs, + multiAgentV2Enabled, + keepNativeChatGptOnV1, + openaiContextCap, + accountNativeSlugs, + accountNativeSlugsBySelector, + reserve, +}: ObservedCatalogEntryBuildInput): RawEntry[] { + // Codex's models-manager sorts by `priority` ASC and advertises the first 5 picker-visible + // models to spawn_agent (sort_by_key(priority) + MAX_MODEL_OVERRIDES_IN_SPAWN_AGENT=5). Catalog + // ARRAY order is discarded — so "featuring" a model = giving it the LOWEST priority (0..N-1) so + // it sorts to the front. This works for native gpt slugs AND routed slugs alike. + const rank = new Map((featured ?? []).map((slug, i) => [slug, i] as const)); + const priorityStride = Math.max(accountSelectors.length, 1); + // Optional full picker order (#1649). Independent of the 5-slot spawn_agent cap: it only + // rewrites the Codex-visible display `priority` of listed non-featured routed rows so a >5 + // catalog stays put across rebuilds. Featured rows keep their existing 0..N-1 band; when + // modelPickerOrder is unset the helper is a no-op and every priority below is byte-identical to + // before. The spawn_agent candidate window is derived separately from SPAWN_PRIORITY_FIELD, so + // this display reorder does not change OpenCodex's guidance candidate calculation. + const pickerOrder = normalizeModelPickerOrder(modelPickerOrder); + const pickerOrderRank = new Map(pickerOrder.map((slug, i) => [slug, i] as const)); + const pickerOrderActive = pickerOrder.length > 0; + // The display band reuses the existing high priority tier (>= PICKER_ORDER_PRIORITY_BASE, the + // same 1_000+ neighborhood account rows occupy), keeping listed rows visually after the featured + // band. OpenCodex guidance membership does not depend on this — see SPAWN_PRIORITY_FIELD. + /** + * Priority for a non-featured routed row that is explicitly LISTED in modelPickerOrder. Listed + * slugs sort in declared order within the high picker-order display tier + * (>= PICKER_ORDER_PRIORITY_BASE). This sets the Codex-visible `priority` only; the caller records + * the row's natural priority in SPAWN_PRIORITY_FIELD for OpenCodex's unchanged guidance window. + * Returns undefined when the feature is off or the row is not listed, so those rows + * keep their original assignment (default 5 / account 1_000+) untouched. + * + * Scope: only the generic routed `/` rows call this (see the goModels loop + * below). Native passthrough rows and account-qualified native rows keep their own priority + * logic and are intentionally not reordered in this legacy builder pass. The final merge can + * apply complete ordering when the configured list includes a bare id. + */ + const pickerOrderPriority = (slug: string, altSlug?: string): number | undefined => { + if (!pickerOrderActive) return undefined; + const hit = pickerOrderRank.get(slug) ?? (altSlug !== undefined ? pickerOrderRank.get(altSlug) : undefined); + if (hit === undefined) return undefined; + return PICKER_ORDER_PRIORITY_BASE + hit * priorityStride; + }; + const out: RawEntry[] = []; + const nativeEntries: RawEntry[] = []; + const collisionSkipped = resolveSlugAliasCollisions([...goModels]); + const emittedNativeAliases = new Set(); + const emittedNativeAliasSlugs = new Set(); + const nativeAliasesBySlug = new Map(); + for (const model of goModels) { + if (model.provider !== COMBO_NAMESPACE + || model.nativeAlias !== true + || typeof model.alias !== "string" + || model.alias.includes("/")) continue; + if (nativeAliasesBySlug.has(model.alias)) { + collisionSkipped.add(model); + if (!slugAliasCollisionWarnings.has(model.alias)) { + slugAliasCollisionWarnings.add(model.alias); + console.warn( + `[opencodex] native combo alias collision on "${model.alias}": keeping the first configured combo and omitting later duplicates from the catalog.`, + ); + } + continue; + } + nativeAliasesBySlug.set(model.alias, model); + } + const comboPublicSlugs = new Set(goModels + .filter(model => model.provider === COMBO_NAMESPACE) + .map(catalogModelSlug)); + for (const slug of gptSlugs) { + const native = deriveEntry(template, slug, "OpenAI native model (Codex OAuth passthrough).", 9, undefined, new Set(), openaiContextCap); + if (rank.has(slug)) native.priority = rank.get(slug)!; + nativeEntries.push(native); + const nativeAlias = nativeAliasesBySlug.get(slug); + if (!nativeAlias || collisionSkipped.has(nativeAlias)) { + if (!suppressedBareNativeSlugs.has(slug)) out.push(native); + continue; + } + const routed = deriveEntry( + template, + slug, + `Routed via opencodex → ${nativeAlias.provider} (${nativeAlias.owned_by ?? nativeAlias.provider}).`, + 5, + nativeAlias, + exactComboSlugs, + ); + routed.opencodex_catalog_kind = CODEX_NATIVE_ALIAS_CATALOG_KIND; + const rankHit = rank.get(slug) ?? rank.get(`${nativeAlias.provider}/${nativeAlias.id}`); + if (rankHit !== undefined) routed.priority = rankHit * priorityStride; + else if (accountSelectors.length > 0) routed.priority = 1_000 + (typeof routed.priority === "number" ? routed.priority : 5); + out.push(routed); + emittedNativeAliases.add(nativeAlias); + emittedNativeAliasSlugs.add(slug); + } + const nativeEntriesBySlug = new Map(nativeEntries.map(entry => [String(entry.slug), entry] as const)); + for (const [selectorIndex, selector] of accountSelectors.entries()) { + const selectorNativeSlugs = accountNativeSlugsBySelector?.get(selector) + ?? accountNativeSlugs + ?? gptSlugs; + const accountNativeEntries = selectorNativeSlugs.filter(slug => slug !== NATIVE_RESERVE_MODEL).map(slug => ( + nativeEntriesBySlug.get(slug) + ?? deriveEntry(template, slug, "OpenAI native model (Codex OAuth passthrough).", 9, undefined, new Set(), openaiContextCap) + )); + if (reserve?.mainSelectors.includes(selector)) accountNativeEntries.push(reserve.source); + for (const [nativeIndex, native] of accountNativeEntries.entries()) { + const nativeSlug = String(native.slug); + if (disabledNativeAccountSlugs.has(nativeSlug)) continue; + const e = JSON.parse(JSON.stringify(native)) as RawEntry; + const catalogSlug = `${selector}/${nativeSlug}`; + if (nativeSlug === NATIVE_RESERVE_MODEL && disabledNativeAccountSlugs.has(catalogSlug)) continue; + e.slug = catalogSlug; + e.display_name = accountBoundNativeDisplayName(selector, native); + // Codex ignores this OpenCodex extension; preserve the native comp_hash unchanged. + e.opencodex_catalog_kind = CODEX_ACCOUNT_BOUND_CATALOG_KIND; + const exactRank = rank.get(catalogSlug); + // A bare featured id belongs to the compatibility combo once shadowed. Exact + // account-qualified picks still rank normally, but the account clone must not + // inherit the bare alias rank and consume another top spawn_agent slot. + const inheritedRank = emittedNativeAliasSlugs.has(nativeSlug) ? undefined : rank.get(nativeSlug); + const featuredRank = exactRank ?? inheritedRank; + e.priority = featuredRank !== undefined + ? featuredRank * priorityStride + selectorIndex + : ((featured?.length ?? 0) + nativeIndex) * accountSelectors.length + selectorIndex; + e.visibility = "list"; + out.push(e); + } + } + for (const m of goModels) { + if (collisionSkipped.has(m) || emittedNativeAliases.has(m)) continue; + const slug = catalogModelSlug(m); + if (m.provider !== COMBO_NAMESPACE && comboPublicSlugs.has(slug)) { + warnComboMasqueradeCollisionOnce(slug); + continue; + } + // Provider rows use the one-slash slug codec; combo aliases intentionally override that + // public slug and may be bare. + const e = deriveEntry( + template, + slug, + `Routed via opencodex → ${m.provider} (${m.owned_by ?? m.provider}).`, + 5, + m, + exactComboSlugs, + ); + if (m.provider === COMBO_NAMESPACE && m.nativeAlias === true && !slug.includes("/")) { + e.opencodex_catalog_kind = CODEX_NATIVE_ALIAS_CATALOG_KIND; + } + // Featured picks may be stored raw (legacy) or encoded — honor both. + const rankHit = rank.get(slug) ?? rank.get(`${m.provider}/${m.id}`); + // Natural priority: what the row would get WITHOUT modelPickerOrder. This is the value the + // spawn_agent candidate window is derived from (see effectiveSubagentRoster), so it must never + // move when modelPickerOrder reorders the picker. + if (rankHit !== undefined) e.priority = rankHit * priorityStride; + else if (accountSelectors.length > 0) { + // Keep the generated account rows together in Codex's priority-sorted flat picker. + e.priority = 1_000 + (typeof e.priority === "number" ? e.priority : 5); + } + // The legacy routed-only builder pass keeps featured ranks and records natural priority + // before changing non-featured display priority. The final complete-order pass may move + // featured display rows too; OpenCodex guidance continues to use their natural ranks. + if (rankHit === undefined) { + const pickerPriority = pickerOrderPriority(slug, `${m.provider}/${m.id}`); + if (pickerPriority !== undefined) { + e[SPAWN_PRIORITY_FIELD] = typeof e.priority === "number" ? e.priority : 5; + e.priority = pickerPriority; + } + } + out.push(e); + } + // Central capability override (phase 120.4): the advertised flag must match the implemented WS + // endpoint. Overrides both the routed strip (normalizeRoutedCatalogEntry) and any native template + // leak (deriveEntry clones the template as-is for native slugs). + for (const entry of out) { + if (wsEnabled) entry.supports_websockets = true; + else { + delete entry.supports_websockets; + // Snapshot-backed native entries carry prefer_websockets: never advertise a preference + // for an endpoint ocx has disabled. + delete entry.prefer_websockets; + } + } + return applyMultiAgentMode(out, multiAgentMode, multiAgentV2Enabled, { + keepNativeChatGptOnV1, + preserveDefaultMultiAgentVersion: isReserveCatalogProjection, + }); +} + +export function resetCatalogRuntimeStateForTests(): void { + resetBundledCatalogCacheForTests(); + lastDropWarnSignature.clear(); + openAiApiCollisionWarnings.clear(); + comboCatalogWarningSignatures.clear(); + slugAliasCollisionWarnings.clear(); + comboMasqueradeCollisionWarnings.clear(); + comboUnrestorableShadowWarnings.clear(); + accountSelectorShadowCollisionWarnings.clear(); + clearLastComboCatalogOmissions(); + clearModelCache(undefined, "eviction"); + clearGatherRoutedModelsInflight(); +} + +export function orderForSubagents(goModels: CatalogModel[], featured?: string[]): CatalogModel[] { + if (!featured || featured.length === 0) return goModels; + const rank = new Map(featured.map((id, i) => [id, i])); + // Featured picks may be stored raw (legacy) or encoded — match both forms. + const rankOf = (m: CatalogModel) => + (m.alias ? rank.get(m.alias) : undefined) + ?? rank.get(`${m.provider}/${m.id}`) + ?? rank.get(routedSlug(m.provider, m.id)) + ?? Number.MAX_SAFE_INTEGER; + return [...goModels].sort((a, b) => { + return rankOf(a) - rankOf(b); + }); +} + +/** Routed discovery projection; native groups and alias ownership belong to the caller. */ +export function orderForModelPicker( + models: readonly CatalogModel[], + order: readonly string[] = [], + featured: readonly string[] = [], +): CatalogModel[] { + const pickerOrder = normalizeModelPickerOrder(order); + if (pickerOrder.length === 0) return [...models]; + const pickerRank = modelPickerRank(pickerOrder); + const featuredRank = modelPickerRank(featured); + const complete = pickerOrder.some(slug => !slug.includes("/")); + const rank = (model: CatalogModel): number => { + const slug = catalogModelSlug(model); + const featuredIndex = featuredRank(slug) ?? featuredRank(`${model.provider}/${model.id}`); + const natural = featuredIndex ?? 5; + const index = pickerRank(slug) ?? pickerRank(`${model.provider}/${model.id}`); + if (complete) return index ?? pickerOrder.length + natural; + // Preserve the legacy featured/alias bands, including unlisted rows before listed rows. + if (featuredIndex !== undefined || model.nativeAlias === true) return natural; + return index === undefined ? natural : PICKER_ORDER_PRIORITY_BASE + index; + }; + return [...models].sort((a, b) => rank(a) - rank(b)); +} + +/** + * True when an existing catalog row was authored by OpenCodex routing (#855). + * Every generated routed row — current full-slug form, the June–July 2026 + * provider-name form, and legacy combo aliases — carries the stable + * description prefix `Routed via opencodex → `; foreign rows from Cursor or + * user tooling do not. `owned_by` cannot serve as the signal (upstream + * ownership), and `comp_hash` defaults to "opencodex" for every normalized + * row. + */ +function isOcxAuthoredRoutedEntry(entry: RawEntry): boolean { + if (isNativeAliasCatalogEntry(entry)) return true; + const desc = typeof entry.description === "string" ? entry.description : ""; + const slug = typeof entry.slug === "string" ? entry.slug : ""; + return slug.includes("/") && desc.startsWith("Routed via opencodex → "); +} + +function recoverableNativeSlug(entry: RawEntry): string | null { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + return SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug) + && !isNativeAliasCatalogEntry(entry) + && entry.owned_by !== COMBO_NAMESPACE + ? slug + : null; +} + +/** Undo our display overlay before native metadata normalization and template reuse. */ +function restoreNativeDisplayName(entry: RawEntry): RawEntry { + const saved = entry.opencodex_native_display_name; + delete entry.opencodex_native_display_name; + if (saved && typeof saved === "object" && !Array.isArray(saved)) { + const label = saved as Record; + if (recoverableNativeSlug(entry) === label.slug + && typeof label.original === "string" && entry.display_name === label.applied) { + entry.display_name = label.original; + } + } + return entry; +} + +/** Append missing supported native rows from trusted catalog sources only. */ +export function mergeCatalogModelsWithNativeRecovery( + primaryCatalogModels: readonly RawEntry[], + nativeRecoverySources: readonly (readonly RawEntry[])[], +): RawEntry[] { + const merged = [...primaryCatalogModels]; + const recoveredNativeSlugs = new Set(primaryCatalogModels.flatMap(entry => { + const slug = recoverableNativeSlug(entry); + return slug === null ? [] : [slug]; + })); + for (const source of nativeRecoverySources) { + for (const entry of source) { + const slug = recoverableNativeSlug(entry); + if (slug === null || recoveredNativeSlugs.has(slug)) continue; + merged.push(structuredClone(entry) as RawEntry); + recoveredNativeSlugs.add(slug); + } + } + return merged; +} + +export interface ObservedCatalogMergePolicy { + /** Required observed/fixed set; the core merge never consults ambient catalog state. */ + readonly nativeBackfillSlugs: readonly string[]; + /** Whether unsupported OpenAI-family bare rows survive the merge. */ + readonly unsupportedNativeEntries: "preserve" | "drop"; + /** Whether merge-policy collision/preservation warnings belong to this caller's flow. */ + readonly warningPolicy: "emit" | "suppress"; +} + +/** Content policy shared by every writer of the canonical Codex model catalog. */ +export const CANONICAL_NATIVE_CATALOG_CONTENT_POLICY: Readonly< + Pick +> = Object.freeze({ + nativeBackfillSlugs: Object.freeze([...NATIVE_OPENAI_MODELS]), + unsupportedNativeEntries: "drop", +}); + +function normalizeModelPickerOrder(order: unknown): string[] { + return Array.isArray(order) + ? order.filter((id): id is string => typeof id === "string" && id.trim().length > 0) + : []; +} + +/** Preserve exact-id precedence while accepting the existing raw/encoded slug spellings. */ +function modelPickerRank(order: readonly string[]): (slug: string) => number | undefined { + const exact = new Map(order.map((slug, index) => [slug, index])); + const equivalent = new Map(order.map((slug, index) => [slugEquivalenceKey(slug), index])); + return slug => exact.get(slug) ?? equivalent.get(slugEquivalenceKey(slug)); +} + +/** Complete display ordering retains natural ranks for OpenCodex's separate guidance projection. */ +export function applyFullModelPickerOrder(entries: RawEntry[], order: readonly string[]): void { + const pickerOrder = normalizeModelPickerOrder(order); + if (!pickerOrder.some(slug => !slug.includes("/"))) return; + const rankOf = modelPickerRank(pickerOrder); + for (const entry of entries) { + const natural = entry[SPAWN_PRIORITY_FIELD] ?? entry.priority ?? 9; + entry[SPAWN_PRIORITY_FIELD] = natural; + entry.priority = rankOf(String(entry.slug)) ?? pickerOrder.length + Number(natural); + } +} + +export interface ObservedCatalogMergeInput { + readonly catalogModels: readonly RawEntry[]; + readonly baselineCatalogModels: readonly RawEntry[]; + readonly routedEntries: readonly RawEntry[]; + readonly baseline: ReadonlyMap; + readonly featured: readonly string[]; + readonly modelPickerOrder?: readonly string[]; + readonly accountSelectors?: readonly string[]; + readonly wsEnabled: boolean; + readonly template: RawEntry | null; + readonly disabledModels: ReadonlySet; + readonly selectedModelsByProvider: ReadonlyMap>; + readonly gatheredProviderNames: ReadonlySet; + readonly pendingProviderNames?: ReadonlySet; + readonly degradedProviderNames: ReadonlySet; + readonly legacyCustomModelSlugs: ReadonlySet; + readonly multiAgentMode: MultiAgentMode; + readonly multiAgentV2Enabled: boolean; + readonly keepNativeChatGptOnV1?: boolean; + readonly exactComboSlugs: ReadonlySet; + readonly hasPhysicalComboProvider: boolean; + readonly includeNativeOpenAi: boolean; + readonly accountBoundEntries: readonly RawEntry[]; + readonly suppressedBareNativeSlugs?: ReadonlySet; + readonly policy: ObservedCatalogMergePolicy; + readonly openaiContextCap?: NativeContextLimitsInput; + /** Exact display-only labels for bare native OpenAI models. */ + readonly nativeDisplayNames?: Readonly>; +} + +/** + * Deterministically merge one fully observed catalog state. + * + * Every non-catalog input is explicit so evidence-bound convergence cannot + * accidentally fall back to process-ambient catalog discovery or merge-policy warnings. + */ +export function mergeCatalogEntriesFromObservedState({ + catalogModels, + baselineCatalogModels, + routedEntries, + baseline, + featured, + modelPickerOrder = [], + accountSelectors = [], + wsEnabled, + template, + disabledModels, + selectedModelsByProvider, + gatheredProviderNames, + pendingProviderNames = new Set(), + degradedProviderNames, + legacyCustomModelSlugs, + multiAgentMode, + multiAgentV2Enabled, + keepNativeChatGptOnV1, + exactComboSlugs, + hasPhysicalComboProvider, + includeNativeOpenAi, + accountBoundEntries, + suppressedBareNativeSlugs = new Set(), + policy, + openaiContextCap, + nativeDisplayNames, +}: ObservedCatalogMergeInput): RawEntry[] { + // Raw catalog rows contain nested arrays/objects that normalization mutates. Detach every row at + // the observed-core boundary so callers can safely retain evidence objects or repeat the merge. + const detachedCatalogModels = catalogModels + .map(entry => restoreNativeDisplayName(structuredClone(entry) as RawEntry)); + const detachedBaselineCatalogModels = baselineCatalogModels + .map(entry => restoreNativeDisplayName(structuredClone(entry) as RawEntry)); + const detachedRoutedEntries = routedEntries.map(entry => structuredClone(entry) as RawEntry); + // Track this invocation's generated custom rows, not ownership markers read from disk. + // Their builder already finalized exact native ladders and ordinary routed mock tiers. + const freshCustomEntries = new Set(detachedRoutedEntries.filter(entry => + entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND)); + const detachedAccountBoundEntries = accountBoundEntries + .map(entry => structuredClone(entry) as RawEntry); + const disabledModelKeys = new Set([...disabledModels].map(slugEquivalenceKey)); + const legacyCustomModelKeys = new Set( + [...legacyCustomModelSlugs].map(slugEquivalenceKey), + ); + const selectedModelKeysByProvider = new Map([...selectedModelsByProvider].map(([provider, models]) => ( + [provider, new Set([...models].map(model => slugEquivalenceKey(routedSlug(provider, model))))] as const + ))); + const freshAccountKeys = new Set(detachedAccountBoundEntries.flatMap(entry => ( + typeof entry.slug === "string" ? [slugEquivalenceKey(entry.slug)] : [] + ))); + const wouldSurviveUnreplaced = (entry: RawEntry): boolean => { + if (entry.owned_by === COMBO_NAMESPACE + || trustedAccountBoundNativeCatalogSlug(entry) !== undefined + || entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND + || isOcxAuthoredRoutedEntry(entry) + || typeof entry.slug !== "string") return false; + const slug = entry.slug; + if (!slug.includes("/")) { + if (!includeNativeOpenAi || policy.nativeBackfillSlugs.includes(slug)) return false; + return policy.unsupportedNativeEntries === "preserve" || !isUnsupportedOpenAiNativeSlug(slug); + } + if (isRoutedModelCompatibilityExcluded(slug)) return false; + if (!hasPhysicalComboProvider && slug.startsWith(`${COMBO_NAMESPACE}/`)) return false; + const key = slugEquivalenceKey(slug); + if (freshAccountKeys.has(key)) return false; + if (disabledModelKeys.has(key)) return false; + const slash = slug.indexOf("/"); + const provider = slug.slice(0, slash); + if (pendingProviderNames.has(provider)) return false; + const selected = selectedModelKeysByProvider.get(provider); + if (selected !== undefined && !selected.has(key)) return false; + return !gatheredProviderNames.has(provider) || degradedProviderNames.has(provider); + }; + const validRoutedEntries = detachedRoutedEntries.filter(entry => { + return !isExactComboCatalogEntry(entry, exactComboSlugs) + || (Array.isArray(entry.input_modalities) && entry.input_modalities.length > 0); + }); + const restorableCatalogKeys = new Set(detachedBaselineCatalogModels.flatMap(entry => ( + wouldSurviveUnreplaced(entry) && typeof entry.slug === "string" + ? [slugEquivalenceKey(entry.slug)] + : [] + ))); + const unrestorableCatalogKeys = new Set(detachedCatalogModels.flatMap(entry => { + if (!wouldSurviveUnreplaced(entry) || typeof entry.slug !== "string") return []; + const key = slugEquivalenceKey(entry.slug); + return restorableCatalogKeys.has(key) ? [] : [key]; + })); + const admittedRoutedEntries = validRoutedEntries.filter(entry => { + if (!isExactComboCatalogEntry(entry, exactComboSlugs)) return true; + const slug = entry.slug as string; + const key = slugEquivalenceKey(slug); + if (!unrestorableCatalogKeys.has(key)) return true; + if (policy.warningPolicy === "emit") warnComboUnrestorableShadowOnce(slug); + return false; + }); + // A fresh non-custom row authoritatively resolves a historically ambiguous slug as a normal + // provider model. Persist that classification so the durable deletion evidence cannot remove + // the legitimate row during a later degraded refresh. + for (const entry of admittedRoutedEntries) { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + if (!slug + || entry.opencodex_catalog_kind !== undefined + || entry.owned_by === COMBO_NAMESPACE + || !isOcxAuthoredRoutedEntry(entry) + || !legacyCustomModelKeys.has(slugEquivalenceKey(slug))) continue; + entry.opencodex_catalog_kind = CODEX_PROVIDER_MODEL_CATALOG_KIND; + } + const freshExactComboEntries = new Set(admittedRoutedEntries.filter(entry => ( + isExactComboCatalogEntry(entry, exactComboSlugs) + && typeof entry.description === "string" + && entry.description.startsWith(`Routed via opencodex → ${COMBO_NAMESPACE} (`) + ))); + const rank = new Map(featured.map((slug, i) => [slug, i] as const)); + const freshEquivalentKeys = new Set(admittedRoutedEntries.flatMap(entry => ( + typeof entry.slug === "string" ? [slugEquivalenceKey(entry.slug)] : [] + ))); + const freshEquivalent = (slug: string): boolean => ( + freshEquivalentKeys.has(slugEquivalenceKey(slug)) + ); + const freshBareComboAliases = new Set(admittedRoutedEntries.flatMap(entry => ( + typeof entry.slug === "string" + && !entry.slug.includes("/") + && entry.owned_by === COMBO_NAMESPACE + ? [entry.slug] + : [] + ))); + const staleComboKeys = new Set(detachedCatalogModels.flatMap(entry => ( + typeof entry.slug === "string" + && entry.owned_by === COMBO_NAMESPACE + && !freshEquivalent(entry.slug) + ? [slugEquivalenceKey(entry.slug)] + : [] + ))); + const currentNonComboKeys = new Set(detachedCatalogModels.flatMap(entry => ( + entry.owned_by !== COMBO_NAMESPACE && typeof entry.slug === "string" + ? [slugEquivalenceKey(entry.slug)] + : [] + ))); + const restoredComboShadows = detachedBaselineCatalogModels.filter(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + if (!slug || entry.owned_by === COMBO_NAMESPACE) return false; + const key = slugEquivalenceKey(slug); + return staleComboKeys.has(key) && !currentNonComboKeys.has(key); + }); + const catalogModelsForMerge = [...detachedCatalogModels, ...restoredComboShadows]; + const nativePriority = (slug: string, fallback: unknown): number => { + const base = baseline.get(slug) + ?? (typeof fallback === "number" ? fallback : 9); + if (rank.has(slug)) return rank.get(slug)!; + return featured.length > 0 ? Math.max(base, featured.length + 100) : base; + }; + const nativeSourceEntries = includeNativeOpenAi + ? catalogModelsForMerge + .filter(m => typeof m.slug === "string" + && !(m.slug as string).includes("/") + && m.owned_by !== COMBO_NAMESPACE + && (policy.unsupportedNativeEntries === "preserve" + || policy.nativeBackfillSlugs.includes(m.slug as string) + || !isUnsupportedOpenAiNativeSlug(m.slug as string))) + .map(m => { + const slug = m.slug as string; + // Fallback-quality entries (ocx synthesis / codex-rs model_info fallback: display_name + // stamped with the bare slug) are upgraded to the pinned upstream snapshot entry so a + // previously synthesized ladder (e.g. luna advertising ultra) self-heals on sync. A + // genuine catalog entry (real display name) is preserved untouched. + if (shouldUpgradeToUpstreamEntry(m)) { + const upstream = upstreamNativeEntry(slug)!; + const finished = finishUpstreamNativeEntry(upstream, 9, openaiContextCap); + finished.priority = nativePriority(slug, upstream.priority); + return finished; + } + const preserved = normalizeServiceTiers({ ...m, priority: nativePriority(slug, m[SPAWN_PRIORITY_FIELD] ?? m.priority) }); + // Recompute spawn rank from current featured models, not a prior picker override. + delete preserved[SPAWN_PRIORITY_FIELD]; + // Older natives kept from disk still need the mock top tiers (max + ultra always + // for subagent max spawns; wire-clamped to the model's real top rung). + if (!isGpt56NativeSlug(slug) && slug !== NATIVE_RESERVE_MODEL) ensureUltraReasoningLevel(preserved); + return preserved; + }) + : []; + const native = nativeSourceEntries.filter(entry => + typeof entry.slug !== "string" + || (!freshBareComboAliases.has(entry.slug) && !suppressedBareNativeSlugs.has(entry.slug)) + ); + + // Backfill any native OpenAI slug that the on-disk catalog is missing (e.g. gpt-5.5), so a + // routed provider exposing the same id can never delete the native OpenAI/Codex base row. + // Skip when no enabled canonical openai provider exists (#636) — bare gpt-* would 404. + const nativeSlugs = new Set(native.flatMap(m => typeof m.slug === "string" ? [m.slug] : [])); + if (includeNativeOpenAi) { + for (const slug of policy.nativeBackfillSlugs) { + if (nativeSlugs.has(slug) || freshBareComboAliases.has(slug) || suppressedBareNativeSlugs.has(slug)) continue; + nativeSlugs.add(slug); + const entry = deriveEntry( + template ? JSON.parse(JSON.stringify(template)) : null, + slug, + "OpenAI native model (Codex OAuth passthrough).", + nativePriority(slug, upstreamNativeEntry(slug)?.priority), + undefined, + new Set(), + openaiContextCap, + ); + entry.priority = nativePriority(slug, upstreamNativeEntry(slug)?.priority); + native.push(entry); + } + } + + const nativeSourceBySlug = new Map([...nativeSourceEntries, ...native].flatMap(entry => + typeof entry.slug === "string" ? [[entry.slug, entry] as const] : [] + )); + const alignedAccountBoundEntries = detachedAccountBoundEntries.map(entry => { + // The explicit Reserve source is already chosen (actual row or documented Luna adaptation). + // A generic native merge must not replace its provenance or capability ladder. + if (isReserveCatalogProjection(entry)) return entry; + const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); + const source = nativeSlug === undefined ? undefined : nativeSourceBySlug.get(nativeSlug); + if (!source) return entry; + const aligned = JSON.parse(JSON.stringify(source)) as RawEntry; + aligned.slug = entry.slug; + aligned.display_name = entry.display_name; + aligned.priority = entry.priority; + aligned.visibility = "list"; + aligned.opencodex_catalog_kind = CODEX_ACCOUNT_BOUND_CATALOG_KIND; + return aligned; + }); + + const freshSlugs = new Set( + admittedRoutedEntries.flatMap(entry => typeof entry.slug === "string" ? [entry.slug] : []), + ); + const existingRoutedEntries = catalogModelsForMerge.filter(m => + typeof m.slug === "string" + && (m.slug.includes("/") || isNativeAliasCatalogEntry(m)) + && trustedAccountBoundNativeCatalogSlug(m) === undefined + ); + const preservedRoutedEntries = existingRoutedEntries.filter(entry => { + const slug = entry.slug as string; + if (freshEquivalent(slug)) return false; + if (isNativeAliasCatalogEntry(entry)) return exactComboSlugs.has(slug); + // Current custom rows are always regenerated from config, even while provider discovery is + // degraded. A marked row absent from the fresh projection is therefore an intentional delete. + if (entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND) return false; + // Before custom rows had a marker, a config deletion could otherwise be mistaken for a + // provider outage. Only explicit save-boundary evidence may classify an unmarked OpenCodex + // row; foreign and future-marked rows fail closed and remain preserved. + if (entry.opencodex_catalog_kind === undefined + && entry.owned_by !== COMBO_NAMESPACE + && isOcxAuthoredRoutedEntry(entry) + && legacyCustomModelKeys.has(slugEquivalenceKey(slug))) return false; + const provider = slug.slice(0, slug.indexOf("/")); + if (gatheredProviderNames.has(provider)) { + // A provider-local degraded observation preserves only that namespace. Authoritative empty + // catalogs and successful removals still delete stale rows even when another provider fails. + return degradedProviderNames.has(provider); + } + // Deleted/disabled providers cannot retain OpenCodex-authored ghosts. Foreign catalog rows + // remain outside provider ownership and survive unless a fresh row replaces their exact slug. + return !isOcxAuthoredRoutedEntry(entry); + }); + // Retained rows bypass the builder. Recompute managed spawn ranks from current config + // before either display-order mode; a saved display override is not current roster authority. + const pickerOrder = normalizeModelPickerOrder(modelPickerOrder); + const fullPickerOrder = pickerOrder.some(slug => !slug.includes("/")); + const rankOf = modelPickerRank(pickerOrder); + const featuredRankOf = modelPickerRank(featured); + const priorityStride = Math.max(accountSelectors.length, 1); + for (const entry of preservedRoutedEntries) { + const natural = entry[SPAWN_PRIORITY_FIELD]; + if (typeof natural === "number") { + entry.priority = natural; + delete entry[SPAWN_PRIORITY_FIELD]; + } + const slug = String(entry.slug); + if (!isOcxAuthoredRoutedEntry(entry) || isNativeAliasCatalogEntry(entry)) continue; + const featuredRank = featuredRankOf(slug); + entry.priority = featuredRank !== undefined + ? featuredRank * priorityStride + : (accountSelectors.length > 0 ? 1_000 : 0) + 5; + if (featuredRank !== undefined || fullPickerOrder) continue; + const pickerIndex = rankOf(slug); + if (pickerIndex !== undefined) { + entry[SPAWN_PRIORITY_FIELD] = entry.priority; + entry.priority = PICKER_ORDER_PRIORITY_BASE + pickerIndex * priorityStride; + } + } + let finalRoutedEntries = [...admittedRoutedEntries, ...preservedRoutedEntries]; + finalRoutedEntries = finalRoutedEntries.filter(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + if (!slug.includes("/")) return true; + if (disabledModelKeys.has(slugEquivalenceKey(slug))) return false; + // Provider allowlists own provider rows, not a current combo's public alias. Exempt only an + // identity from this gather's generated combo projection: provider discovery may supply a + // spoofed `owned_by`, and persisted combo-shaped rows are not fresh authority. + if (freshExactComboEntries.has(entry)) return true; + const slash = slug.indexOf("/"); + const provider = slug.slice(0, slash); + if (pendingProviderNames.has(provider)) return false; + const selected = selectedModelKeysByProvider.get(provider); + return selected === undefined || selected.has(slugEquivalenceKey(slug)); + }); + if (!hasPhysicalComboProvider) { + finalRoutedEntries = finalRoutedEntries.filter(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + const comboOwned = slug.startsWith(`${COMBO_NAMESPACE}/`) || entry.owned_by === COMBO_NAMESPACE; + const retainedNativeAlias = isNativeAliasCatalogEntry(entry) && exactComboSlugs.has(slug); + return !comboOwned || freshSlugs.has(slug) || retainedNativeAlias; + }); + } + finalRoutedEntries = finalRoutedEntries.filter(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + const retainedNativeAlias = isNativeAliasCatalogEntry(entry) && exactComboSlugs.has(slug); + return retainedNativeAlias + || !isExactComboCatalogEntry(entry, exactComboSlugs) + || (Array.isArray(entry.input_modalities) && entry.input_modalities.length > 0); + }); + // Reapply final catalog policy to rows preserved from disk. Those rows bypass + // gatherRoutedModels, so filtering only the freshly gathered list can resurrect an excluded id. + finalRoutedEntries = finalRoutedEntries.filter(entry => + typeof entry.slug !== "string" || !isRoutedModelCompatibilityExcluded(entry.slug) + ); + const accountBoundSlugs = new Set(alignedAccountBoundEntries.flatMap(entry => + typeof entry.slug === "string" ? [entry.slug] : [] + )); + finalRoutedEntries = finalRoutedEntries.filter(entry => { + if (typeof entry.slug !== "string" || !accountBoundSlugs.has(entry.slug)) return true; + if (freshSlugs.has(entry.slug) && policy.warningPolicy === "emit") { + warnAccountSelectorShadowedProviderOnce(entry.slug); + } + return false; + }); + const finalRoutedEntrySet = new Set(finalRoutedEntries); + const degradedPreservedCount = preservedRoutedEntries.filter(entry => { + if (!finalRoutedEntrySet.has(entry)) return false; + const slug = entry.slug as string; + const provider = slug.slice(0, slug.indexOf("/")); + return gatheredProviderNames.has(provider) && degradedProviderNames.has(provider); + }).length; + if (degradedPreservedCount > 0 && policy.warningPolicy === "emit") { + console.warn(`[opencodex] catalog sync: provider discovery degraded; preserving ${degradedPreservedCount} existing routed entr${degradedPreservedCount === 1 ? "y" : "ies"} on disk.`); + } + + const managedEntries = [...finalRoutedEntries, ...alignedAccountBoundEntries]; + const observedNativeSlugs = new Set(alignedAccountBoundEntries.flatMap(entry => { + const slug = trustedAccountBoundNativeCatalogSlug(entry); + return slug === undefined ? [] : [slug]; + })); + for (const slug of policy.nativeBackfillSlugs) observedNativeSlugs.add(slug); + const mergedEntries = [...native, ...managedEntries].map(m => { + const reserveProjection = isReserveCatalogProjection(m); + const normalized = reserveProjection ? m : normalizeServiceTiers(m); + if (!reserveProjection && !isNativeAliasCatalogEntry(normalized)) applyNativeOpenAiContextOverride(normalized, openaiContextCap); + const exactCombo = isExactComboCatalogEntry(m, exactComboSlugs); + const e = reserveProjection ? normalized : ensureStrictCatalogFields(normalized, { + preserveExactInputModalities: exactCombo, + isRouted: finalRoutedEntrySet.has(m), + }); + // Mock-max universality (260709): preserved routed entries from disk may predate + // the max rung — ensure it here so subagent max spawns validate on every + // reasoning-capable entry. max only: 5.6 exact ladders (luna: no ultra) stay intact. + if (!freshCustomEntries.has(m) && !exactCombo && !reserveProjection && !String(e.slug ?? "").startsWith("opencode-go/")) { + const levels = Array.isArray(e.supported_reasoning_levels) + ? e.supported_reasoning_levels as Array<{ effort?: string }> + : []; + if (levels.length > 0 && !levels.some(level => level.effort === "max")) { + levels.push(CODEX_REASONING_LEVELS.find(level => level.effort === "max") + ?? { effort: "max", description: "Maximum reasoning depth for the hardest problems" }); + e.supported_reasoning_levels = levels; + } + } + if (wsEnabled) e.supports_websockets = true; + else { + delete e.supports_websockets; + // Match buildCatalogEntries: never advertise a websocket preference while WS is off. + delete e.prefer_websockets; + } + return e; + }); + // Native enable/disable runs as the LAST pass so the upstream-upgrade branch above can never + // clobber a hide flag back to list. Bare ids disable every account clone; qualified ids disable + // only their generated account row. + const versionedEntries = applyMultiAgentMode( + applyNativeVisibility(mergedEntries, disabledModels, alignedAccountBoundEntries.length > 0, observedNativeSlugs), + multiAgentMode, + multiAgentV2Enabled, + { keepNativeChatGptOnV1, preserveDefaultMultiAgentVersion: isReserveCatalogProjection }, + ); + applyFullModelPickerOrder(versionedEntries, modelPickerOrder); + for (const entry of versionedEntries) { + // Templates and account clones must not inherit the native row's overlay marker. + delete entry.opencodex_native_display_name; + const slug = recoverableNativeSlug(entry); + if (slug !== null) { + const label = nativeDisplayNames && Object.hasOwn(nativeDisplayNames, slug) + ? nativeDisplayNames[slug]?.trim() : undefined; + if (label && label !== entry.display_name) { + entry.opencodex_native_display_name = { slug, original: entry.display_name, applied: label }; + entry.display_name = label; + } + } + const kind = entry.opencodex_catalog_kind; + if (trustedAccountBoundNativeCatalogSlug(entry) === undefined + && kind !== CODEX_CUSTOM_MODEL_CATALOG_KIND + && kind !== CODEX_PROVIDER_MODEL_CATALOG_KIND) continue; + // Canonicalize extension-field order after every normalizer. This keeps an unchanged catalog + // byte-idempotent whether an owned row was freshly built or retained from the prior pass. + delete entry.opencodex_catalog_kind; + entry.opencodex_catalog_kind = kind; + } + return versionedEntries; +} + +/** Merge retained-sync rows using the process-observed Codex feature state. */ +export function mergeCatalogEntriesForSync( + catalogModels: RawEntry[], + routedEntries: RawEntry[], + baseline: Map, + featured: string[], + wsEnabled: boolean, + _goIds: Set = new Set(), + template: RawEntry | null = null, + disabledModels: ReadonlySet = new Set(), + gatheredProviderNames?: Set, + multiAgentMode: MultiAgentMode = "default", + exactComboSlugs: ReadonlySet = new Set(), + hasPhysicalComboProvider = false, + includeNativeOpenAi = true, + accountBoundEntries: readonly RawEntry[] = [], + legacyCustomModelSlugs: ReadonlySet = new Set(), + suppressedBareNativeSlugs: ReadonlySet = new Set( + routedEntries.flatMap(entry => ( + isNativeAliasCatalogEntry(entry) && typeof entry.slug === "string" ? [entry.slug] : [] + )), + ), + openaiContextCap?: NativeContextLimitsInput, + keepNativeChatGptOnV1 = false, +): RawEntry[] { + // Retained for source compatibility with the original helper contract. Raw provider ids must + // not suppress same-named native rows; actual admitted combo entries own that decision now. + void _goIds; + const effectiveGatheredProviderNames = gatheredProviderNames ?? new Set( + routedEntries.flatMap(entry => { + // A slashed combo alias is not evidence that its public prefix is an authoritative provider + // namespace. Treating it as one would let the combo replace an unrestorable foreign row. + if (isExactComboCatalogEntry(entry, exactComboSlugs)) return []; + const slug = typeof entry.slug === "string" ? entry.slug : ""; + const slash = slug.indexOf("/"); + return slash > 0 ? [slug.slice(0, slash)] : []; + }), + ); + return mergeCatalogEntriesFromObservedState({ + catalogModels, + baselineCatalogModels: [], + routedEntries, + baseline, + featured, + wsEnabled, + template, + disabledModels, + selectedModelsByProvider: new Map(), + gatheredProviderNames: effectiveGatheredProviderNames, + degradedProviderNames: new Set(), + legacyCustomModelSlugs, + multiAgentMode, + multiAgentV2Enabled: isMultiAgentV2Enabled(), + keepNativeChatGptOnV1, + exactComboSlugs, + hasPhysicalComboProvider, + includeNativeOpenAi, + accountBoundEntries, + suppressedBareNativeSlugs, + openaiContextCap, + policy: { + ...CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, + warningPolicy: "emit", + }, + }); +} diff --git a/src/codex/catalog/derive-entry.ts b/src/codex/catalog/derive-entry.ts new file mode 100644 index 0000000000..8b1d5290c3 --- /dev/null +++ b/src/codex/catalog/derive-entry.ts @@ -0,0 +1,229 @@ +import type { OcxConfig } from "../../types"; +import { effectiveProviderAlias } from "../../providers/default-aliases"; +import { identifyRoutedModel } from "../../adapters/identity"; +import { COMBO_NAMESPACE } from "../../combos"; +import { + CODEX_CUSTOM_MODEL_CATALOG_KIND, + applyCatalogMetadata, + applyNativeOpenAiContextOverride, + applyRoutedCodexToolMode, + catalogModelSlug, + ensureStrictCatalogFields, + normalizeRoutedCatalogEntry, + normalizeServiceTiers, +} from "./parsing"; +import type { CatalogModel, RawEntry } from "./parsing"; +import { + hasNativeOpenAiCapabilityMetadata, + upstreamNativeEntry, + type NativeContextLimitsInput, +} from "./metadata"; +import { + applyCatalogModelMetadata, + applyReasoningLevels, + ensureGpt56ReasoningLevels, + ensureUltraReasoningLevel, + isGpt56NativeSlug, +} from "./effort"; +import { CATALOG_INACTIVE_REASON_FIELD, SPAWN_PRIORITY_FIELD } from "./subagent-roster"; + +export function finishUpstreamNativeEntry(clone: RawEntry, priority: number, contextCap?: NativeContextLimitsInput): RawEntry { + if (priority !== 9) clone.priority = priority; + applyNativeOpenAiContextOverride(clone, contextCap); + // GPT-5.6 natives keep their exact upstream ladders (e.g. luna has max but no ultra). + // Older natives (gpt-5.5) get mock max + ultra + // (wire-clamped to xhigh). Ultra is always advertised regardless of v2 toggle. + if (!isGpt56NativeSlug(String(clone.slug ?? ""))) ensureUltraReasoningLevel(clone); + return ensureStrictCatalogFields(normalizeServiceTiers(clone)); +} + +export function isExactComboCatalogModel( + model: CatalogModel | undefined, + exactComboSlugs: ReadonlySet, +): boolean { + return model?.provider === COMBO_NAMESPACE && exactComboSlugs.has(catalogModelSlug(model)); +} + +export function isExactComboCatalogEntry( + entry: RawEntry, + exactComboSlugs: ReadonlySet, +): boolean { + return entry.owned_by === COMBO_NAMESPACE + && typeof entry.slug === "string" + && exactComboSlugs.has(entry.slug); +} + +/** + * Friendly Codex-picker label for a routed `provider/model` slug. Command Code's two config + * ids differ by a single dash (`command-code` vs `commandcode`), so relabel them to the + * lowercase-dash style the opencode presets use: `commandcode-auth/x` and `commandcode-api/x`. + * The model-id portion also carries a redundant `-` prefix (`deepseek-deepseek-v4-flash`) + * that is dropped for display. Google Antigravity is relabeled to the compact `agy/` prefix for + * the same reason: `google-antigravity/` alone consumes most of the picker row. That prefix comes + * from the row's own `providerAlias`, decided once per gather flight; `null` means a cross-provider + * collision suppressed it and the canonical slug stands. This is the raw-slug path only -- a + * configured `modelAliases` entry is labeled by the effective-alias path in + * catalog/provider-fetch.ts (#2960) and keeps the canonical provider name. All other providers + * keep the raw slug exactly as before. + */ +function routedDisplayName(slug: string, model?: CatalogModel, config?: Pick): string { + const slash = slug.indexOf("/"); + if (slash <= 0) return slug; + const provider = slug.slice(0, slash); + let modelId = slug.slice(slash + 1); + if (provider === "google-antigravity") { + if (model?.providerAlias === null) return slug; + const alias = (typeof model?.providerAlias === "string" && model.providerAlias.trim().length > 0) + ? model.providerAlias.trim() + : effectiveProviderAlias(provider, undefined, config); + return alias ? `${alias}/${modelId}` : slug; + } + if (provider === "command-code" || provider === "commandcode") { + const m = modelId.match(/^([a-z0-9]+)-([a-z0-9]+(?:-[a-z0-9]+)+)$/i); + if (m && modelId.startsWith(`${m[1]}-${m[1]}-`)) modelId = modelId.slice(m[1]!.length + 1); + return `${provider === "command-code" ? "commandcode-auth" : "commandcode-api"}/${modelId}`; + } + return slug; +} + +function preservePinnedNativeCustomReasoning(model?: CatalogModel): boolean { + return model !== undefined + && model.catalogKind === CODEX_CUSTOM_MODEL_CATALOG_KIND + && hasNativeOpenAiCapabilityMetadata(model.id) + && Array.isArray(model.reasoningEfforts); +} + +/** + * Cria uma entrada nativa ou roteada a partir do snapshot upstream, de um clone + * do template ou de campos mínimos. Aplica os metadados e limites pertinentes + * sem alterar o template nem herdar sua marca de nome ou histórico de prioridade. + */ +export function deriveEntry( + template: RawEntry | null, + slug: string, + desc: string, + priority: number, + model?: CatalogModel, + exactComboSlugs: ReadonlySet = new Set(), + contextCap?: NativeContextLimitsInput, +): RawEntry { + const preserveExact = isExactComboCatalogModel(model, exactComboSlugs); + // Go exposes model-specific upstream enums; synthetic tiers mislead subagent overrides. + const preserveExactReasoning = preserveExact || model?.provider === "opencode-go"; + const codexForwardNativeCapabilityAlias = model?.codexForwardNativeCapabilityAlias === true + ? upstreamNativeEntry(model.id) + : null; + const isRouted = model !== undefined; + if (!isRouted && !slug.includes("/")) { + // Supported native slug covered by the upstream snapshot: use the REAL entry (exact + // reasoning ladder — e.g. luna has no ultra — default effort, identity, model_messages) + // instead of cloning an older template. + const upstream = upstreamNativeEntry(slug); + if (upstream) return finishUpstreamNativeEntry(upstream, priority, contextCap); + } + if (template || codexForwardNativeCapabilityAlias) { + const e = JSON.parse(JSON.stringify(codexForwardNativeCapabilityAlias ?? template)) as RawEntry; + delete e.opencodex_native_display_name; + // A cached template may carry display-order history; each new row owns its natural rank. + delete e[SPAWN_PRIORITY_FIELD]; + e.slug = slug; + e.display_name = routedDisplayName(slug, model); + e.description = desc; + e.priority = priority; + e.visibility = "list"; + if ("upgrade" in e) e.upgrade = null; + delete e.availability_nux; // don't replay another model's "now available" NUX + // Routed (namespaced) models inherit the gpt template — correct its OpenAI/GPT identity + // and advertise the reasoning ladder Codex accepts. + if (isRouted) { + // A routed model is NOT the native template: never inherit its context + // window when /models omits context metadata (#992). Known metadata + // restores exact values below; an enabled Context cap fills the gap; + // otherwise the strict-fields fallback supplies the 128k triple. + if (!codexForwardNativeCapabilityAlias) { + delete e.context_window; + delete e.max_context_window; + delete e.auto_compact_token_limit; + } + // Native id for identity text + metadata lookups — the slug may be an encoded + // alias (`provider/vendor-model`); the model object carries the native id. + const modelName = model?.id ?? slug.slice(slug.indexOf("/") + 1); + if (typeof e.base_instructions === "string") { + // Proxy-neutral: keep the GPT-5/OpenAI disclaimer but never advertise the opencodex proxy + // (leaking that into base_instructions is a non-first-party signature → ToS risk). + e.base_instructions = identifyRoutedModel(e.base_instructions, modelName); + } + applyReasoningLevels( + e, + model?.reasoningEfforts, + model?.defaultReasoningEffort, + preserveExactReasoning + || codexForwardNativeCapabilityAlias !== null + || preservePinnedNativeCustomReasoning(model), + ); + // This exact provider/model pair is the ChatGPT/Codex forward surface. Keep the pinned + // native tool/search/responses-lite contract while preserving the routed slug and wire id. + if (!codexForwardNativeCapabilityAlias) { + normalizeRoutedCatalogEntry(e, model?.parallelToolCalls === true, model?.codexToolMode); + } else if (model?.codexToolMode !== undefined) { + applyRoutedCodexToolMode(e, model.codexToolMode); + } + if (model) applyCatalogMetadata(e, model.provider, model.id, model.contextCap); + applyCatalogModelMetadata(e, model); + if (model?.catalogKind) e.opencodex_catalog_kind = model.catalogKind; + // Additive only. `visibility` is untouched: an inactive row must still be OFFERED, which is + // the whole point of #1711 — operator disable is what removes rows, and it stays a separate + // path from this one. + if (model?.quotaInactiveReason) e[CATALOG_INACTIVE_REASON_FIELD] = model.quotaInactiveReason; + } else { + applyNativeOpenAiContextOverride(e, contextCap); + if (isGpt56NativeSlug(slug)) ensureGpt56ReasoningLevels(e); + else ensureUltraReasoningLevel(e); + // Older natives do not support Responses Lite. A newer template must not enable + // reasoning.context or WebSockets on those models. + if (!isGpt56NativeSlug(slug)) { + delete e.use_responses_lite; + delete e.supports_websockets; + } + } + return ensureStrictCatalogFields(normalizeServiceTiers(e), { + preserveExactInputModalities: preserveExact, + isRouted, + }); + } + // Fallback when no template is available (best-effort; strict parser may need more). + // Routed fallbacks default to code-mode tool exposure (or shell mode when codexToolMode === "shell"); + // otherwise the nested catalog expands into `exec.description` and can exceed Cursor's 120 KB serialized tool limit (#1830). + // Cursor still omits hosted web-search metadata because runTurn bypasses that separate sidecar. + const isCursorFallback = isRouted && model?.provider === "cursor"; + const entry: RawEntry = { + slug, display_name: routedDisplayName(slug, model), description: desc, + shell_type: "unified_exec", visibility: "list", supported_in_api: true, + priority, base_instructions: "You are a helpful coding assistant.", + ...(isRouted + ? isCursorFallback + ? { supports_search_tool: true } + : { web_search_tool_type: "text_and_image", supports_search_tool: true } + : {}), + }; + if (isRouted) { + applyRoutedCodexToolMode(entry, model?.codexToolMode); + applyReasoningLevels(entry, model?.reasoningEfforts, model?.defaultReasoningEffort, preserveExactReasoning || preservePinnedNativeCustomReasoning(model)); + } + else { + applyReasoningLevels(entry, isGpt56NativeSlug(slug) ? undefined : ["low", "medium", "high", "xhigh"]); + if (isGpt56NativeSlug(slug)) ensureGpt56ReasoningLevels(entry); + } + if (model && isRouted) applyCatalogMetadata(entry, model.provider, model.id, model.contextCap); + applyCatalogModelMetadata(entry, model); + if (model?.catalogKind) entry.opencodex_catalog_kind = model.catalogKind; + // Same additive stamp as the templated path above. A routed row that reaches the no-template + // fallback is still a served row, so omitting it here would make the field depend on whether a + // template happened to be cached — which is exactly what the regression test caught. + if (model?.quotaInactiveReason) entry[CATALOG_INACTIVE_REASON_FIELD] = model.quotaInactiveReason; + if (!isRouted) applyNativeOpenAiContextOverride(entry, contextCap); + return ensureStrictCatalogFields(normalizeServiceTiers(entry), { + preserveExactInputModalities: preserveExact, + isRouted, + }); +} diff --git a/src/codex/catalog/effort.ts b/src/codex/catalog/effort.ts index e3909263c0..0e5901c2a4 100644 --- a/src/codex/catalog/effort.ts +++ b/src/codex/catalog/effort.ts @@ -39,7 +39,6 @@ import { nativeOpenAiCapabilitySourceSlug, SELF_DESCRIBED_NATIVE_OPENAI_MODELS, import { isReserveCatalogProjection } from "./reserve"; import { loadBundledCodexCatalog } from "./bundled"; import type { BundledCatalogDeps, ReadonlyRawCatalog } from "./bundled"; -import { deriveEntry } from "./sync"; import { formatClampLogLines, formatRuntimeLogLine, diff --git a/src/codex/catalog/gated-native-warn.ts b/src/codex/catalog/gated-native-warn.ts new file mode 100644 index 0000000000..cc12834e54 --- /dev/null +++ b/src/codex/catalog/gated-native-warn.ts @@ -0,0 +1,63 @@ +import type { OcxConfig } from "../../types"; +import { codexModelEntitlementStateForAccount, type CodexModelEntitlementSnapshot } from "../model-entitlements"; +import { codexAccountLogLabel, fallbackCodexAccountLogLabel } from "../account-label"; +import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; + +export function gatedNativeReauthSuppressionReason(args: { + snapshot: CodexModelEntitlementSnapshot; + slug: string; + eligibleAccountIds?: ReadonlySet; + needsReauth: (accountId: string) => boolean; + label: (accountId: string) => string; +}): string | undefined { + const observed = [...args.snapshot.modelsByAccount.keys()] + .filter(accountId => !args.eligibleAccountIds || args.eligibleAccountIds.has(accountId)) + // Only accounts that could actually have served THIS model. An account upstream positively + // denied is not why the model is missing, and blaming it would send the operator to repair a + // credential that was never going to help. `unknown` has to stay in: an account whose roster + // could not be confirmed reports `unknown` rather than `granted`, and a credential stuck on + // a failed refresh is exactly that account. + .filter(accountId => ( + codexModelEntitlementStateForAccount(args.snapshot, accountId, args.slug) !== "denied" + )); + const stuck = observed.filter(accountId => args.needsReauth(accountId)); + if (stuck.length === 0) return undefined; + const names = stuck.map(accountId => args.label(accountId)).sort().join(", "); + return stuck.length === observed.length + ? `every Codex account that could serve it needs reauthentication (${names})` + : `${stuck.length} of ${observed.length} Codex accounts that could serve it need reauthentication (${names})`; +} + +/** Durable, operator-facing label for a pool account id; never the raw id or the email. */ +export function gatedNativeAccountLabel(config: OcxConfig, accountId: string): string { + // Direct mode narrows eligibility to the native main credential, so this is the account most + // likely to be named here. `codexAuthContextLogLabel` calls it "main" everywhere else; hashing + // it into a `p`-prefixed digest would name the one account the operator cannot look up. + if (accountId === MAIN_CODEX_ACCOUNT_ID) return "main"; + const account = (config.codexAccounts ?? []).find(candidate => candidate.id === accountId); + return account ? codexAccountLogLabel(account) : fallbackCodexAccountLogLabel(accountId); +} + +const warnedGatedNativeSuppression = new Set(); + +/** Test seam: the warn-once memory is process-global, so a case needs to be able to clear it. */ +export function resetGatedNativeSuppressionWarningsForTests(): void { + warnedGatedNativeSuppression.clear(); +} + +export function warnGatedNativeSuppressedOnce(slug: string, reason: string): void { + const signature = `${slug}\u0000${reason}`; + if (warnedGatedNativeSuppression.has(signature)) return; + warnedGatedNativeSuppression.add(signature); + console.warn( + `[opencodex] catalog sync: ${slug} is not being offered because ${reason}. ` + + "Sign in again to restore it.", + ); +} + +/** + * Mescla o catálogo retido com os modelos visíveis e as configurações atuais, + * incluindo os nomes nativos. Tenta preservar o backup original e usa a permissão + * de escrita para publicar o resultado apenas se os bytes mudarem, retornando + * a contagem de entradas roteadas e por conta, o caminho e o estado da gravação. + */ diff --git a/src/codex/catalog/restore.ts b/src/codex/catalog/restore.ts new file mode 100644 index 0000000000..5393d58410 --- /dev/null +++ b/src/codex/catalog/restore.ts @@ -0,0 +1,132 @@ +import { readConfigDiagnostics } from "../../config"; +import { getCodexHome } from "../paths"; +import { readCatalog, readCatalogBackup, readCodexCatalogPath } from "./parsing"; +import type { RawEntry } from "./parsing"; +import { RETIRED_NATIVE_OPENAI_MODELS, SUPPORTED_NATIVE_OPENAI_SLUGS } from "./metadata"; +import { trustedAccountBoundNativeCatalogSlug } from "./account-models"; +import { + withCatalogWriteSerialization, + type CatalogWritePermit, +} from "../catalog-write-serialization"; +import { replaceActiveCodexCatalog } from "../internal/catalog-writer"; + +function visibleAccountReplacementNatives( + models: readonly RawEntry[], + disabledModels: ReadonlySet | null, +): Map { + const replacements = new Map(); + for (const entry of models) { + const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); + if (nativeSlug === undefined || !SUPPORTED_NATIVE_OPENAI_SLUGS.has(nativeSlug)) continue; + const exactSlug = typeof entry.slug === "string" ? entry.slug : ""; + const visible = entry.visibility === "list" + || (disabledModels !== null + && (disabledModels.has(nativeSlug) || disabledModels.has(exactSlug))); + replacements.set(nativeSlug, (replacements.get(nativeSlug) ?? true) && visible); + } + return replacements; +} + +function restoreAccountHiddenBareNatives( + entries: readonly RawEntry[], + replacementVisibility: ReadonlyMap, + disabledModels: ReadonlySet | null, +): RawEntry[] { + return entries.map(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + if ( + entry.visibility !== "hide" + || !SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug) + || replacementVisibility.get(slug) !== true + || disabledModels === null + || disabledModels.has(slug) + ) { + return entry; + } + return { ...entry, visibility: "list" }; + }); +} + +function currentDisabledModelsForRestore(): Set | null { + try { + const diagnostics = readConfigDiagnostics(); + if (diagnostics.source === "fallback" || diagnostics.error !== null) return null; + return new Set(diagnostics.config.disabledModels ?? []); + } catch { + // An unreadable config cannot safely authorize a visibility change during restore. + return null; + } +} + +export function restoreCodexCatalogWithPermit( + permit: CatalogWritePermit, + owningCodexHome: string, + /** + * The catalog this injection actually wrote, when it is known (#1798). + * + * Re-resolving from the CURRENT config is wrong after a Codex app rewrite that dropped + * `model_catalog_json`: that sends restore to the default catalog while the routed file we + * really wrote is left untouched. The recorded path is the file whose routing is ours. + */ + injectedCatalogPath?: string | null, +): { removed: number; kept: number; path: string } { + const catalogPath = injectedCatalogPath ?? readCodexCatalogPath(); + const catalog = readCatalog(catalogPath); + if (!catalog || !Array.isArray(catalog.models)) return { removed: 0, kept: 0, path: catalogPath }; + const disabledModels = currentDisabledModelsForRestore(); + const replacementVisibility = visibleAccountReplacementNatives(catalog.models, disabledModels); + const backup = readCatalogBackup(catalogPath); + if (backup && Array.isArray(backup.models)) { + const removed = (catalog.models ?? []).filter(m => typeof m.slug === "string" + && (m.slug.includes("/") || RETIRED_NATIVE_OPENAI_MODELS.has(m.slug))).length; + const backupSlugs = new Set(backup.models.flatMap(m => typeof m.slug === "string" ? [m.slug] : [])); + const userNativeAdditions = restoreAccountHiddenBareNatives( + (catalog.models ?? []).filter(m => + typeof m.slug === "string" && !m.slug.includes("/") && !backupSlugs.has(m.slug) + && !RETIRED_NATIVE_OPENAI_MODELS.has(m.slug) + ), + replacementVisibility, + disabledModels, + ); + const restored = { + ...backup, + // A pristine backup predates retirement; it must not revive withdrawn native rows. + models: [...backup.models.filter(m => typeof m.slug !== "string" + || !RETIRED_NATIVE_OPENAI_MODELS.has(trustedAccountBoundNativeCatalogSlug(m) ?? m.slug)), ...userNativeAdditions], + }; + replaceActiveCodexCatalog(permit, owningCodexHome, { + path: catalogPath, + content: `${JSON.stringify(restored, null, 2)}\n`, + }); + return { removed, kept: restored.models.length, path: catalogPath }; + } + const before = catalog.models.length; + const native = restoreAccountHiddenBareNatives( + catalog.models.filter(m => !(typeof m.slug === "string" + && (m.slug.includes("/") || RETIRED_NATIVE_OPENAI_MODELS.has(m.slug)))), + replacementVisibility, + disabledModels, + ); + const removed = before - native.length; + if (removed > 0) { + catalog.models = native; + replaceActiveCodexCatalog(permit, owningCodexHome, { + path: catalogPath, + content: `${JSON.stringify(catalog, null, 2)}\n`, + }); + } + return { removed, kept: native.length, path: catalogPath }; +} + +export function restoreCodexCatalog(): { removed: number; kept: number; path: string } { + const owningCodexHome = getCodexHome(); + const outcome = withCatalogWriteSerialization( + owningCodexHome, + permit => restoreCodexCatalogWithPermit(permit, owningCodexHome), + ); + return outcome.kind === "completed" + ? outcome.value + : { removed: 0, kept: 0, path: readCodexCatalogPath() }; +} + +/** Force Codex's models_cache stale from the on-disk catalog. Returns whether a cache write occurred. */ diff --git a/src/codex/catalog/retained-sync.ts b/src/codex/catalog/retained-sync.ts new file mode 100644 index 0000000000..21daf32714 --- /dev/null +++ b/src/codex/catalog/retained-sync.ts @@ -0,0 +1,706 @@ +import { existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { loadConfig, websocketsEnabled } from "../../config"; +import { shouldSyncCodexOnStart } from "../desired-state"; +import { legacyCustomModelCatalogSlugs } from "../custom-model-catalog-migration"; +import { getCodexHome } from "../paths"; +import type { OcxConfig } from "../../types"; +import { pendingModelSelectionProviders } from "../../providers/initial-model-selection"; +import { OPENAI_CODEX_PROVIDER_ID } from "../../providers/openai-tiers"; +import { providerCodexAccountMode } from "../../providers/registry"; +import { COMBO_NAMESPACE } from "../../combos"; +import { codexAccountNamespaceEntries, isMainCodexAccountTarget } from "../account-namespaces"; +import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; +import { + availableAccountGatedNativeModels, + codexModelEntitlementStateForAccount, + isCodexModelEntitlementSnapshotCurrent, + resolveCodexModelEntitlements, + type CodexModelEntitlementSnapshot, +} from "../model-entitlements"; +import { isAccountNeedsReauth } from "../account-runtime-state"; +import { codexRuntimeStatePath } from "../runtime"; +import { + activeCodexModelsCachePath, + catalogBackupPathFor, + catalogHasRoutedEntries, + findNativeTemplate, + findSupportedNativeTemplate, + isDefaultCatalogPath, + legacyCatalogBackupPath, + readCatalog, + readCatalogBackup, + readCodexCatalogPath, + readCodexCatalogPathForHome, + readNativeBaseline, +} from "./parsing"; +import type { CatalogModel, MultiAgentMode, RawCatalog, RawEntry } from "./parsing"; +import { + accountBoundNativeOpenAiSlugsBySelector, + desktopAllowlistSuppressedNativeSlugs, + disabledNativeSlugs, + nativeContextLimits, + observedAccountBoundNativeEntries, + observedReserveCatalogSource, + shouldIncludeAccountBoundNativeOpenAi, + shouldIncludeNativeOpenAi, + upstreamNativeEntry, +} from "./metadata"; +import { trustedAccountBoundNativeCatalogSlug } from "./account-models"; +import { bundledCatalogCacheState, loadBundledCodexCatalog } from "./bundled"; +import { isMultiAgentV2Enabled } from "../features"; +import { clampCatalogModelsToCodexSupport } from "./effort"; +import { filterCatalogVisibleModels, gatherRoutedModels, type CatalogGatherProviderModelOutcome } from "./provider-fetch"; +import { exactComboCatalogSlugs, type ComboCatalogOmission } from "./aggregation"; +import { + withCatalogWriteSerialization, + type CatalogWritePermit, +} from "../catalog-write-serialization"; +import { + publishHashedCodexCatalogBackup, + publishLegacyCodexCatalogBackup, + replaceActiveCodexCatalog, + replaceCodexModelsCache, +} from "../internal/catalog-writer"; +import { visibleCodexAccountSelectors } from "./account-models"; +import { ACCOUNT_GATED_NATIVE_OPENAI_MODELS, NATIVE_OPENAI_MODELS, NATIVE_RESERVE_MODEL } from "./native-models"; +import { createReserveCatalogProjection, RESERVE_LUNA_METADATA_SOURCE, RESERVE_SOURCE_CATALOG_FIELD } from "./reserve"; +import { + CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, + buildCatalogEntriesFromObservedState, + mergeCatalogEntriesFromObservedState, + mergeCatalogModelsWithNativeRecovery, + orderForSubagents, +} from "./build-entries"; +import { finishUpstreamNativeEntry } from "./derive-entry"; +import { finalizeAutoReviewModelOverride } from "./auto-review"; +import { gatedNativeAccountLabel, gatedNativeReauthSuppressionReason, warnGatedNativeSuppressedOnce } from "./gated-native-warn"; + +interface RetainedCatalogSyncRead { + readonly catalogPath: string; + readonly catalog: RawCatalog; + readonly onDiskCatalog: RawCatalog | null; + readonly modelsCache: RawCatalog | null; + readonly evidence: string; + /** + * Process-local epochs, baselined AFTER our own gather rather than with the + * filesystem bytes above. See `retainedCatalogProcessEvidence`. + */ + readonly processEvidence: string; +} + +interface RetainedCatalogSyncResult { + added: number; + path: string; + catalogWritten: boolean; + comboOmissions: ComboCatalogOmission[]; + /** Validated catalog commit (including identical bytes), or a refused refresh. */ + refreshOutcome?: "committed" | "refused"; + /** `desired_disabled` observed under K after the provider await; nothing was written. */ + skippedReason?: "desired_disabled"; +} + +/** + * Catalog/cache commit overrides. + * + * An explicit `ocx sync` is also the refresh path for side profiles that consume + * the OpenCodex catalog without injection (for example a custom `model_provider` + * that routes to the proxy). In that mode the Codex integration toggle only + * governs config/history injection; the catalog and models cache may still be + * refreshed, so `allowWhenDesiredDisabled` lets the commit path ignore the OFF + * gate that otherwise protects a fully native home. + */ +export interface CodexCatalogSyncOptions { + allowWhenDesiredDisabled?: boolean; +} + +interface RetainedCatalogSyncWrite { + readonly config: OcxConfig; + readonly goModels: CatalogModel[]; + readonly providerModelOutcomes: readonly CatalogGatherProviderModelOutcome[]; + readonly comboOmissions: ComboCatalogOmission[]; + readonly read: RetainedCatalogSyncRead; + readonly permit: CatalogWritePermit; + readonly owningCodexHome: string; + readonly modelEntitlements: CodexModelEntitlementSnapshot; +} + +function optionalFileBytes(path: string): string | null { + try { + return readFileSync(path).toString("base64"); + } catch (error) { + if ((error as NodeJS.ErrnoException | undefined)?.code === "ENOENT") return null; + throw error; + } +} + +function loadCatalogForRetainedSync(path: string): RawCatalog | null { + const bundled = isDefaultCatalogPath(path) ? loadBundledCodexCatalog() : null; + if (bundled) return JSON.parse(JSON.stringify(bundled)) as RawCatalog; + const active = readCatalog(path); + // A valid configured custom file remains the content authority even when it has no bare native + // template. The null-template builder is deliberate; a stale backup must not replace active + // custom root metadata merely because the current file contains only routed rows. + if (active && (!isDefaultCatalogPath(path) || findNativeTemplate(active))) return active; + return readCatalog(catalogBackupPathFor(path)) + ?? (isDefaultCatalogPath(path) ? readCatalog(legacyCatalogBackupPath()) : null) + ?? readCatalog(activeCodexModelsCachePath()) + ?? active; +} + +function retainedCatalogSyncEvidence( + config: OcxConfig, + catalogPath: string, + catalog: RawCatalog, +): string { + return JSON.stringify({ + config, + catalogPath, + catalog, + catalogBytes: optionalFileBytes(catalogPath), + hashedBackupBytes: optionalFileBytes(catalogBackupPathFor(catalogPath)), + legacyBackupBytes: isDefaultCatalogPath(catalogPath) + ? optionalFileBytes(legacyCatalogBackupPath()) : null, + modelsCacheBytes: optionalFileBytes(activeCodexModelsCachePath()), + // The persisted runtime selection is a pre-await filesystem input, not a + // process epoch: another PROCESS can move runtime authority by rewriting this + // file, and that move is invisible to our in-process memo. Recorded PRESENT or + // ABSENT, because its absence is what makes the resolver fall back. + runtimeStateBytes: optionalFileBytes(codexRuntimeStatePath()), + }); +} + +/** + * The bundled-template half of the same evidence, observed separately. + * + * The runtime process memo is deliberately NOT here, and that exclusion took three + * attempts to get honest. Gathering resolves the Codex runtime lazily and under its + * own cache key, so this path cannot pre-settle that memo: baselining it before the + * await always detected our own side effect and refused every write, and baselining + * it after the await captured a runtime that ANOTHER process had moved as though it + * were ours — a catalog prepared from R1 committing after authority reached R2. + * + * Runtime authority is covered where it is actually durable instead: the persisted + * `codex-runtime.json` bytes sit in the pre-await filesystem evidence, PRESENT or + * ABSENT, so a cross-process runtime move is caught. What is left uncovered, and is + * written down rather than papered over, is a same-process in-memory runtime swap + * that never touches that file — WP11 owns the lock that makes that case decidable. + */ +function retainedCatalogProcessEvidence(): string { + return JSON.stringify({ + bundledCatalogCache: bundledCatalogCacheState(), + }); +} + +/** + * Capture every local catalog input the retained sync path consults before its + * provider await. The exact evidence is compared after K acquisition; a newer + * catalog/backup/cache or target selection makes this attempt a no-write. + */ +function readRetainedCatalogSync(config: OcxConfig): RetainedCatalogSyncRead | null { + const catalogPath = readCodexCatalogPath(); + const catalog = loadCatalogForRetainedSync(catalogPath); + if (!catalog) return null; + + // The bundled catalog is a reliable native template on the default path, but it is not the + // merge source. Preservation must inspect the file that this sync is about to overwrite; + // otherwise an empty/partial provider gather cannot see routed or user-native rows on disk. + const onDiskCatalog = readCatalog(catalogPath); + const modelsCache = readCatalog(activeCodexModelsCachePath()); + const evidence = retainedCatalogSyncEvidence(config, catalogPath, catalog); + // `processEvidence` is filled in after the provider await, not here. + return { catalogPath, catalog, onDiskCatalog, modelsCache, evidence, processEvidence: "" }; +} + +function revalidateRetainedCatalogSync( + config: OcxConfig, + prepared: RetainedCatalogSyncRead, +): RetainedCatalogSyncRead | null { + const catalogPath = readCodexCatalogPath(); + if (catalogPath !== prepared.catalogPath) return null; + const evidence = retainedCatalogSyncEvidence(config, catalogPath, prepared.catalog); + if (evidence !== prepared.evidence) return null; + if (retainedCatalogProcessEvidence() !== prepared.processEvidence) return null; + return { + catalogPath, + catalog: JSON.parse(JSON.stringify(prepared.catalog)) as RawCatalog, + onDiskCatalog: readCatalog(catalogPath), + modelsCache: readCatalog(activeCodexModelsCachePath()), + evidence, + processEvidence: prepared.processEvidence, + }; +} + +/** + * Exact bytes currently on disk at `path`, or null when unreadable/absent. + * + * Deliberately a Buffer rather than a decoded string: `readFileSync(path, "utf8")` + * substitutes U+FFFD for every invalid byte, so a file holding a raw 0x80 decodes + * equal to prepared content holding a legitimately encoded U+FFFD. Comparing the + * decoded strings would then classify a malformed catalog as identical, skip the + * atomic repair write, and leave the corruption on disk while reporting + * `catalogWritten: false`. + */ +function currentCatalogFileContent(path: string): Buffer | null { + try { + return readFileSync(path); + } catch { + return null; + } +} + +function pristineCatalogBytes(read: RetainedCatalogSyncRead): string | null { + if (read.onDiskCatalog && !catalogHasRoutedEntries(read.onDiskCatalog)) { + try { + return readFileSync(read.catalogPath, "utf8"); + } catch { + return null; + } + } + return catalogHasRoutedEntries(read.catalog) + ? null + : `${JSON.stringify(read.catalog, null, 2)}\n`; +} + +function catalogModelsForMergeWithNativeRecovery( + catalogPath: string, + catalog: RawCatalog, + onDiskCatalog: RawCatalog | null, +): RawEntry[] { + const primaryCatalogModels = onDiskCatalog?.models ?? catalog.models ?? []; + // Native-alias compatibility can omit disabled native rows from the effective catalog because + // Desktop's remote allowlist ignores `visibility: "hide"`. Keep current/pristine native recovery + // sources beside the on-disk rows so re-enabling a model restores its real metadata. Routed and + // user-authored rows still come only from the on-disk catalog. + return mergeCatalogModelsWithNativeRecovery(primaryCatalogModels, [ + catalog.models ?? [], + readCatalogBackup(catalogPath)?.models ?? [], + ]); +} + +function writeRetainedCatalogSync({ + config, + goModels, + providerModelOutcomes, + comboOmissions, + read, + permit, + owningCodexHome, + modelEntitlements, +}: RetainedCatalogSyncWrite): RetainedCatalogSyncResult { + const { catalogPath, catalog, onDiskCatalog } = read; + const catalogModelsForMerge = catalogModelsForMergeWithNativeRecovery( + catalogPath, + catalog, + onDiskCatalog, + ); + // Strict selector for template inheritance; the validity gate above keeps the broad one. + const template = findSupportedNativeTemplate(catalog); + + try { + // Once-only: preserve the PRISTINE pre-opencodex catalog as the native-priority baseline + // (later syncs would otherwise overwrite it with featured-modified priorities). + const pristine = pristineCatalogBytes(read); + if (pristine !== null) { + publishHashedCodexCatalogBackup(permit, owningCodexHome, { + path: catalogBackupPathFor(catalogPath), + content: pristine, + }); + if (isDefaultCatalogPath(catalogPath)) { + publishLegacyCodexCatalogBackup(permit, owningCodexHome, { + path: legacyCatalogBackupPath(), + content: pristine, + }); + } + } + } catch { /* backup best-effort */ } + + // Hide disabled models from Codex, then feature the chosen subagent models (native OR routed) + // by giving them the lowest priority — see buildCatalogEntries for why priority, not array order. + const enabledGo = filterCatalogVisibleModels(goModels, config); + const featured = config.subagentModels ?? []; + const orderedGoModels = orderForSubagents(enabledGo, featured); // stable tie-break among equal priorities + const modelPickerOrder = config.modelPickerOrder ?? []; + const multiAgentMode: MultiAgentMode = config.multiAgentMode === "v1" || config.multiAgentMode === "v2" ? config.multiAgentMode : "default"; + const exactComboSlugs = exactComboCatalogSlugs(config); + const bareEligibleAccountIds = providerCodexAccountMode( + OPENAI_CODEX_PROVIDER_ID, + config.providers[OPENAI_CODEX_PROVIDER_ID], + ) === "direct" ? new Set([MAIN_CODEX_ACCOUNT_ID]) : undefined; + const availableBareGatedNativeSlugs = availableAccountGatedNativeModels( + modelEntitlements, + bareEligibleAccountIds, + ); + const availableAccountGatedNativeSlugs = availableAccountGatedNativeModels(modelEntitlements); + const availableBareNativeSlugs = NATIVE_OPENAI_MODELS.filter(slug => ( + !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) || availableBareGatedNativeSlugs.has(slug) + )); + const availableAccountNativeSlugs = NATIVE_OPENAI_MODELS.filter(slug => ( + !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) || availableAccountGatedNativeSlugs.has(slug) + )); + const unavailableGatedNativeSlugs = new Set([...ACCOUNT_GATED_NATIVE_OPENAI_MODELS].filter(slug => ( + !availableBareGatedNativeSlugs.has(slug) + ))); + // #4212: this set is the whole record of a model vanishing, and it is a set of strings that + // nothing downstream ever asks a question of. Explain it here, while the entitlement snapshot + // that produced it is still in scope, because after this point the model is simply absent and + // no later surface can tell "never entitled" apart from "the account broke this morning". + for (const slug of unavailableGatedNativeSlugs) { + const reason = gatedNativeReauthSuppressionReason({ + snapshot: modelEntitlements, + slug, + eligibleAccountIds: bareEligibleAccountIds, + needsReauth: isAccountNeedsReauth, + label: accountId => gatedNativeAccountLabel(config, accountId), + }); + if (reason) warnGatedNativeSuppressedOnce(slug, reason); + } + const suppressedBareNativeSlugs = new Set([ + ...desktopAllowlistSuppressedNativeSlugs(config), + ...unavailableGatedNativeSlugs, + ]); + const hasPhysicalComboProvider = Object.hasOwn(config.providers, COMBO_NAMESPACE); + const includeNativeOpenAi = shouldIncludeNativeOpenAi(config); + const includeAccountBoundNativeOpenAi = shouldIncludeAccountBoundNativeOpenAi(config); + // Both user levers. Passing only the cap here is what let a per-model window the dashboard + // had accepted get written back at full width in the on-disk catalog. + const openaiContextCap = nativeContextLimits(config); + const accountSelectors = includeAccountBoundNativeOpenAi + ? visibleCodexAccountSelectors(config) + : []; + const observedAccountNativeEntries = [ + ...(read.modelsCache?.models ?? []), + ...(onDiskCatalog?.models ?? []).filter(entry => + trustedAccountBoundNativeCatalogSlug(entry) !== undefined), + ]; + const accountTargets = new Map(codexAccountNamespaceEntries(config)); + const reserveMainSelectors = accountSelectors.filter(selector => + isMainCodexAccountTarget(accountTargets.get(selector) ?? "")); + // The active file can own a bare source even when the bundled catalog is the build base. + // A previously clamped qualified projection must not shorten a retained genuine ladder. + const reserveObservations = [ + ...(onDiskCatalog?.models ?? []), + ...(read.modelsCache?.models ?? []), + ...(catalog.models ?? []), + ]; + const retainedReserve = onDiskCatalog?.[RESERVE_SOURCE_CATALOG_FIELD]; + const retainedReserveSource = retainedReserve && typeof retainedReserve === "object" && !Array.isArray(retainedReserve) + ? observedReserveCatalogSource([retainedReserve as RawEntry], []) + : null; + const observedReserveSource = observedReserveCatalogSource( + // Cache invalidation carries historical bare observations alongside emitted models. + // Only unmarked observations are fresh enough to supersede the retained source. + reserveObservations.filter(entry => entry.slug === NATIVE_RESERVE_MODEL + && entry.opencodex_account_observed_native === undefined), reserveMainSelectors, + ) ?? retainedReserveSource ?? observedReserveCatalogSource(reserveObservations, reserveMainSelectors); + // This root is read only by OCX. Upstream ModelsResponse ignores unknown root fields. + // Retain before final runtime clamping: an omitted row must not turn into Luna next sync. + if (observedReserveSource) catalog[RESERVE_SOURCE_CATALOG_FIELD] = structuredClone(observedReserveSource); + else delete catalog[RESERVE_SOURCE_CATALOG_FIELD]; + const lunaSource = upstreamNativeEntry(RESERVE_LUNA_METADATA_SOURCE); + const reserve = createReserveCatalogProjection( + config, + reserveMainSelectors, + observedReserveSource, + lunaSource ? finishUpstreamNativeEntry(lunaSource, 9, openaiContextCap) : null, + ); + const accountNativeSlugsBySelector = accountSelectors.length > 0 + ? new Map([...accountBoundNativeOpenAiSlugsBySelector(config, observedAccountNativeEntries)].map(([selector, slugs]) => { + const target = accountTargets.get(selector); + const accountId = target && isMainCodexAccountTarget(target) ? MAIN_CODEX_ACCOUNT_ID : target; + return [selector, slugs.filter(slug => ( + !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) + || (accountId !== undefined + && codexModelEntitlementStateForAccount(modelEntitlements, accountId, slug) === "granted") + ))] as const; + })) + : new Map(); + const accountNativeSlugs = accountSelectors.length > 0 + ? [...new Set([...accountNativeSlugsBySelector.values()].flatMap(slugs => [...slugs]))] + : []; + // Unknown account-native ids have no safe bare/global identity. They are only projected through + // the selector map above; the no-selector catalog remains the static native/API-key surface. + const observedNativeSlugs: string[] = []; + const wsEnabled = websocketsEnabled(config); + const multiAgentV2Enabled = isMultiAgentV2Enabled(); + const goEntries = buildCatalogEntriesFromObservedState({ + template: template ? JSON.parse(JSON.stringify(template)) : null, + gptSlugs: [], + goModels: orderedGoModels, + featured, + modelPickerOrder, + wsEnabled, + multiAgentMode, + exactComboSlugs, + accountSelectors, + suppressedBareNativeSlugs, + disabledNativeAccountSlugs: new Set(), + multiAgentV2Enabled, + openaiContextCap, + }); + // Keep genuine native entries (gpt-*, codex-*) with their real per-model fields and append + // routed providers as namespaced slugs. Cursor and other adopted providers can expose model ids + // like `gpt-5.5`; those must not delete the native OpenAI/Codex base row. + const baselineCatalog = readCatalogBackup(catalogPath); + const baseline = readNativeBaseline(catalogPath); + const gatheredProviderNames = new Set( + Object.entries(config.providers ?? {}) + .filter(([, prov]) => prov.disabled !== true) + .map(([name]) => name), + ); + const degradedProviderNames = new Set( + providerModelOutcomes + .filter(outcome => outcome.state === "degraded") + .map(outcome => outcome.provider), + ); + const selectedModelsByProvider = new Map>( + Object.entries(config.providers ?? {}).flatMap(([name, provider]) => ( + provider.disabled !== true + && Array.isArray(provider.selectedModels) + && provider.selectedModels.length > 0 + ? [[name, new Set(provider.selectedModels)] as const] + : [] + )), + ); + // Central WS capability override on the FINAL on-disk catalog (the file Codex reads). Applies to + // native AND routed so the advertised flag matches the implemented endpoint (phase 120.4) and a + // native template can never leak supports_websockets while the flag is off. + // #636: when the user only configured non-OpenAI providers (e.g. kimi), do not advertise + // bare gpt-* rows that hard-404 via NoEnabledOpenAiProviderError. Keep natives when no + // providers are configured yet (fresh install / catalog bootstrap tests). + const accountBoundEntries = includeAccountBoundNativeOpenAi && accountSelectors.length > 0 + ? buildCatalogEntriesFromObservedState({ + template: template ? JSON.parse(JSON.stringify(template)) : null, + gptSlugs: availableAccountNativeSlugs, + goModels: [], + featured, + wsEnabled, + multiAgentMode, + exactComboSlugs, + accountSelectors, + suppressedBareNativeSlugs, + disabledNativeAccountSlugs: new Set([...disabledNativeSlugs(config)].filter(slug => suppressedBareNativeSlugs.has(slug))), + multiAgentV2Enabled, + keepNativeChatGptOnV1: config.keepNativeChatGptOnV1 === true, + openaiContextCap, + accountNativeSlugs, + accountNativeSlugsBySelector, + reserve, + }).filter(entry => trustedAccountBoundNativeCatalogSlug(entry) !== undefined) + : []; + catalog.models = mergeCatalogEntriesFromObservedState({ + modelPickerOrder, + accountSelectors, + catalogModels: catalogModelsForMerge, + baselineCatalogModels: baselineCatalog?.models ?? [], + routedEntries: goEntries, + baseline, + featured, + wsEnabled, + template, + disabledModels: new Set(config.disabledModels ?? []), + selectedModelsByProvider, + gatheredProviderNames, + pendingProviderNames: pendingModelSelectionProviders(config), + degradedProviderNames, + legacyCustomModelSlugs: legacyCustomModelCatalogSlugs(config), + multiAgentMode, + multiAgentV2Enabled, + keepNativeChatGptOnV1: config.keepNativeChatGptOnV1 === true, + exactComboSlugs, + hasPhysicalComboProvider, + includeNativeOpenAi, + accountBoundEntries, + suppressedBareNativeSlugs, + openaiContextCap, + nativeDisplayNames: config.providers[OPENAI_CODEX_PROVIDER_ID]?.modelDisplayNames, + policy: { + ...CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, + nativeBackfillSlugs: [...availableBareNativeSlugs, ...observedNativeSlugs], + warningPolicy: "emit", + }, + }); + clampCatalogModelsToCodexSupport(catalog.models); + finalizeAutoReviewModelOverride(catalog.models, catalogModelsForMerge, config); + + const added = goEntries.length + accountBoundEntries.length; + const content = `${JSON.stringify(catalog, null, 2)}\n`; + // A byte-identical rewrite is not a catalog change, but every mtime-keyed reader + // has to treat it as one. The app-server staleness classifier (#857) is the one + // that matters: it compares this file's mtime against each running Codex's start + // time, so an ordinary `ocx start` — or any dashboard action that re-syncs an + // unchanged model set — marked every already-running Codex as holding an outdated + // in-memory catalog. Since #1407 that verdict silences opencodex's own model + // guidance entirely (no preferred model, no roster) for the rest of that Codex's + // lifetime, so a configured injectionModel stops reaching the session even though + // nothing about the catalog changed. Skipping the no-op write keeps both the mtime + // and `catalogWritten` honest; `added` still reports the routed rows the catalog + // carries, because they are on disk either way. + const onDiskBytes = currentCatalogFileContent(catalogPath); + if (onDiskBytes !== null && onDiskBytes.equals(Buffer.from(content, "utf8"))) { + return { added, path: catalogPath, catalogWritten: false, comboOmissions }; + } + + replaceActiveCodexCatalog(permit, owningCodexHome, { + path: catalogPath, + content, + }); + return { + added, + path: catalogPath, + catalogWritten: true, + comboOmissions, + }; +} + +export async function syncCatalogModels( + config: OcxConfig, + options?: CodexCatalogSyncOptions, +): Promise { + if (pendingModelSelectionProviders(config).size) { + const { resolvePendingInitialModelSelection } = await import("../../providers/initial-model-selection-runtime"); + await resolvePendingInitialModelSelection(config); + } + const owningCodexHome = getCodexHome(); + const preflightRead = readRetainedCatalogSync(config); + if (preflightRead === null) { + return { + added: 0, + path: readCodexCatalogPath(), + catalogWritten: false, + comboOmissions: [], + refreshOutcome: "refused", + }; + } + + const comboOmissions: ComboCatalogOmission[] = []; + const providerModelOutcomes: CatalogGatherProviderModelOutcome[] = []; + // Settle the bundled template, then baseline, and only then await. Reading it + // here makes the memo ours before anyone else can move it, so a bundled swap + // during the await is an outside change rather than our own side effect. + // + // The persisted runtime selection is covered by the filesystem evidence above + // rather than by a process epoch; see `retainedCatalogProcessEvidence` for why + // the in-memory runtime memo cannot be baselined honestly from this path. + loadBundledCodexCatalog(); + const prepared: RetainedCatalogSyncRead = { + ...preflightRead, + evidence: retainedCatalogSyncEvidence(config, preflightRead.catalogPath, preflightRead.catalog), + processEvidence: retainedCatalogProcessEvidence(), + }; + const [goModels, modelEntitlements] = await Promise.all([ + gatherRoutedModels(config, { + comboOmissions, + providerModelOutcomes, + }), + resolveCodexModelEntitlements(config), + ]); + const committed = withCatalogWriteSerialization(owningCodexHome, permit => { + // Desired state can flip OFF during the provider await above. The catalog + // evidence revalidation below cannot see that — intent lives in our config, + // not in the catalog files — so the policy is re-read here, under K, right + // before the only write. A lost race becomes the discriminated skip instead + // of a routed catalog/cache surviving a completed disable. An explicit + // catalog-only sync opts out of that gate: the user asked for a refresh even + // when injection is OFF, and the toggle only protects config/history writes. + if (!shouldSyncCodexOnStart(loadConfig()) && options?.allowWhenDesiredDisabled !== true) { + return { + added: 0, + path: prepared.catalogPath, + catalogWritten: false, + comboOmissions, + skippedReason: "desired_disabled" as const, + }; + } + const current = revalidateRetainedCatalogSync(config, prepared); + if (current === null) return null; + if (!isCodexModelEntitlementSnapshotCurrent(modelEntitlements)) return null; + return writeRetainedCatalogSync({ + config, + goModels, + providerModelOutcomes, + comboOmissions, + read: current, + permit, + owningCodexHome, + modelEntitlements, + }); + }); + if (committed.kind === "completed" && committed.value !== null) { + return { + ...committed.value, + refreshOutcome: committed.value.skippedReason ? "refused" : "committed", + }; + } + return { + added: 0, + path: prepared.catalogPath, + catalogWritten: false, + comboOmissions, + refreshOutcome: "refused", + }; +} + +export function invalidateCodexModelsCacheWithPermit( + permit: CatalogWritePermit, + owningCodexHome: string, + options?: CodexCatalogSyncOptions, +): boolean { + try { + // This permit is a REACQUISITION: refreshCodexModelCatalog's commit released + // K before this rewrite runs, so the commit-path desired-state check cannot + // cover it. A disable landing in that gap must not be overwritten by a + // routed cache write — re-read intent under this permit, same as the commit. + // The catalog-only sync override applies here too so an explicit refresh + // keeps the cache consistent with the catalog it just wrote. + if (!shouldSyncCodexOnStart(loadConfig()) && options?.allowWhenDesiredDisabled !== true) return false; + const catalogPath = readCodexCatalogPathForHome(owningCodexHome); + if (!existsSync(catalogPath)) return false; + const catalog = JSON.parse(readFileSync(catalogPath, "utf8")); + const models = catalog.models ?? catalog; + const cachePath = join(owningCodexHome, "models_cache.json"); + const currentCache = readCatalog(cachePath); + const existingSlugs = new Set(models.flatMap((entry: RawEntry) => + typeof entry.slug === "string" ? [entry.slug] : [])); + const currentConfig = loadConfig(); + const mainSelectors = visibleCodexAccountSelectors(currentConfig).filter(selector => { + const target = new Map(codexAccountNamespaceEntries(currentConfig)).get(selector); + return isMainCodexAccountTarget(target ?? ""); + }); + const observedAccountModels = observedAccountBoundNativeEntries(currentCache?.models ?? []) + .filter(entry => { + const slug = typeof entry.slug === "string" ? entry.slug : ""; + return !existingSlugs.has(slug); + }) + .map(entry => ({ + ...entry, + // Keep the observation in Codex's cache without advertising a new bare picker row. The + // next OpenCodex catalog sync consumes this marker and creates only selector-qualified + // rows for the currently configured public account selectors. + visibility: "hide", + opencodex_account_observed_native: true, + opencodex_account_observed_selectors: mainSelectors, + })); + const wrapper = { + fetched_at: "2000-01-01T00:00:00Z", + client_version: "0.0.0", + models: [...models, ...observedAccountModels], + }; + replaceCodexModelsCache(permit, owningCodexHome, { + path: cachePath, + content: `${JSON.stringify(wrapper, null, 2)}\n`, + }); + return true; + } catch { + return false; + } +} + +export function invalidateCodexModelsCache(options?: CodexCatalogSyncOptions): boolean { + const owningCodexHome = getCodexHome(); + const outcome = withCatalogWriteSerialization( + owningCodexHome, + permit => invalidateCodexModelsCacheWithPermit(permit, owningCodexHome, options), + ); + return outcome.kind === "completed" && outcome.value; +} diff --git a/src/codex/catalog/subagent-roster.ts b/src/codex/catalog/subagent-roster.ts new file mode 100644 index 0000000000..69951d70b9 --- /dev/null +++ b/src/codex/catalog/subagent-roster.ts @@ -0,0 +1,176 @@ +// Holds INV-AGENT-01 from structure/overview.md; keep the id here if this file is split or renamed. +import { slugsEquivalent } from "../../providers/slug-codec"; +import { readCatalog, readCodexCatalogPath } from "./parsing"; +import type { RawEntry } from "./parsing"; +import { SUPPORTED_NATIVE_OPENAI_SLUGS } from "./metadata"; +import { trustedAccountBoundNativeCatalogSlug } from "./account-models"; +import { catalogEntryEfforts } from "./effort"; + +export const MAX_SPAWN_AGENT_MODEL_OVERRIDES = 5; + +// Base for config.modelPickerOrder display priorities (#1649). modelPickerOrder is a DISPLAY-ONLY +// reordering of the Codex model picker: it rewrites a row's Codex-visible `priority` but not +// OpenCodex's natural-priority guidance window. Native Codex advertisements still follow the +// visible priority and can differ from that guidance window. +export const PICKER_ORDER_PRIORITY_BASE = 1_000; + +// OpenCodex-private catalog field: the guidance candidate priority a row would have WITHOUT +// modelPickerOrder. Codex ignores unknown catalog fields (same as opencodex_catalog_kind), so this +// is invisible to Codex; effectiveSubagentRoster reads it to keep OpenCodex guidance candidates +// independent of display order. It does not freeze native advertisements. Absent on unmoved rows. +export const SPAWN_PRIORITY_FIELD = "opencodex_spawn_priority"; + +// OpenCodex-private catalog field: this row is listed but currently unable to serve (#1711). +// Codex ignores unknown catalog fields (same as opencodex_catalog_kind and the spawn priority +// above) and ensureStrictCatalogFields does not strip extras, so this is invisible to the native +// picker and cannot change what Codex offers. It never touches `visibility`. +export const CATALOG_INACTIVE_REASON_FIELD = "opencodex_inactive_reason"; + +export type SpawnAgentSurface = "v1" | "v2"; + +export type SubagentRosterExclusionReason = + | "missing_catalog_entry" + | "picker_hidden" + | "surface_incompatible" + | "outside_display_limit"; + +/** + * Whether a catalog entry may be offered as a V2 subagent model. + * + * Upstream changed this rule in codex-rs `6d4d9442c` ("Support leaf models in + * multi-agent v2"). `model_supports_multi_agent_backend` + * (core/src/tools/handlers/multi_agents_common.rs:36-42) now admits EVERY model + * except one explicitly marked `disabled`; the older `== Some(V2)` equality that + * `92938d880` introduced is gone. + * + * The field no longer answers "may I be a delegation target". It answers "does the + * CHILD get collaboration tools": `collab_tools_enabled` + * (core/src/tools/spec_plan.rs:599-610) grants a child recursive tools only when its + * own catalog value is exactly `Some(V2)`. The three-way distinction survives, but it + * now means eligible-recursive / eligible-LEAF / excluded: + * + * - `"v2"` -> eligible, and the child may itself delegate. + * - `"v1"` -> eligible LEAF worker. This is upstream's pin for `gpt-5.6-luna` + * (models-manager/models.json); excluding it here is exactly what + * kept Luna out of opencodex's roster. + * - absent/null -> eligible LEAF worker (routed or unpinned-native model). + * - `"disabled"` -> the sole capability-based exclusion. + * + * This is the roster filter only. Catalog STAMPING is a separate concern owned by + * `applyMultiAgentMode`, including the `keepNativeChatGptOnV1` policy (#1728) that + * keeps ChatGPT-native rows on `v1` so a native parent can still spawn a routed child + * despite backend-encrypted NEW_TASK bodies (#92). Recognizing those `v1` rows as + * eligible leaves here is what makes that policy usable, not a contradiction of it. + * + * Devlog: 260816_codexrs_multiagent_v2_and_history_perf/011 (C1), superseding the + * option-B decision in 260730_codex_rs_upstream_v2_live_handoff/060. + */ +export function isEligibleV2SubagentEntry(entry: RawEntry): boolean { + return entry.multi_agent_version !== "disabled"; +} + +export interface EffectiveSubagentModel { + model: string; + efforts: string[]; +} + +export interface SubagentRosterExclusion { + configured: string; + reason: SubagentRosterExclusionReason; + catalogModel?: string; +} + +export interface EffectiveSubagentRoster { + /** OpenCodex's natural-priority guidance projection, not captured native tool text. */ + candidates: EffectiveSubagentModel[]; + /** Configured models within that projection; exact-name eligibility is a separate check. */ + advertised: EffectiveSubagentModel[]; + excluded: SubagentRosterExclusion[]; +} + +export function configuredCatalogEntry(entries: readonly RawEntry[], configured: string): RawEntry | undefined { + return entries.find(entry => entry.slug === configured) + ?? entries.find(entry => typeof entry.slug === "string" && slugsEquivalent(configured, entry.slug)); +} + +function configuredSubagentModelMatchesEntry(configured: string, entry: RawEntry): boolean { + if (typeof entry.slug !== "string") return false; + if (slugsEquivalent(configured, entry.slug)) return true; + const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); + return !configured.includes("/") + && nativeSlug !== undefined + && SUPPORTED_NATIVE_OPENAI_SLUGS.has(nativeSlug) + && slugsEquivalent(configured, nativeSlug); +} + +export function effectiveSubagentRoster( + configuredModels: readonly string[], + surface: SpawnAgentSurface, + catalogEntries?: readonly RawEntry[], +): EffectiveSubagentRoster { + const configured = configuredModels + .filter(model => model.trim().length > 0) + .filter((model, index, all) => + !all.slice(0, index).some(previous => slugsEquivalent(previous, model)) + ); + const entries = catalogEntries ?? readCatalog(readCodexCatalogPath())?.models ?? []; + const ordered = entries + .map((entry, index) => ({ entry, index })) + .filter(({ entry }) => typeof entry.slug === "string") + .filter(({ entry }) => entry.visibility === "list") + .filter(({ entry }) => surface !== "v2" || isEligibleV2SubagentEntry(entry)) + .sort((left, right) => { + // OpenCodex guidance candidates rank by natural priority (SPAWN_PRIORITY_FIELD when present), + // so modelPickerOrder does not change this projection. Native tool advertisements differ. Rows the + // override did not move fall back to their Codex-visible `priority`. + const spawnPriorityOf = (entry: RawEntry): number => { + const spawn = entry[SPAWN_PRIORITY_FIELD]; + if (typeof spawn === "number" && Number.isFinite(spawn)) return spawn; + return typeof entry.priority === "number" && Number.isFinite(entry.priority) + ? entry.priority : Number.MAX_SAFE_INTEGER; + }; + const leftPriority = spawnPriorityOf(left.entry); + const rightPriority = spawnPriorityOf(right.entry); + return leftPriority - rightPriority || left.index - right.index; + }) + .slice(0, MAX_SPAWN_AGENT_MODEL_OVERRIDES); + const orderedEntries = new Set(ordered.map(({ entry }) => entry)); + + const candidates = ordered.map(({ entry }) => ({ + model: entry.slug as string, + efforts: catalogEntryEfforts(entry), + })); + const advertised = ordered + .filter(({ entry }) => configured.some(model => configuredSubagentModelMatchesEntry(model, entry))) + .map(({ entry }) => ({ + model: entry.slug as string, + efforts: catalogEntryEfforts(entry), + })); + const excluded = configured.flatMap((model): SubagentRosterExclusion[] => { + const matchingEntries = entries.filter(entry => configuredSubagentModelMatchesEntry(model, entry)); + if (matchingEntries.some(entry => orderedEntries.has(entry))) return []; + if (matchingEntries.length === 0) return [{ configured: model, reason: "missing_catalog_entry" }]; + const visibleCompatible = matchingEntries.find(entry => + entry.visibility === "list" + && (surface !== "v2" || isEligibleV2SubagentEntry(entry)) + ); + if (visibleCompatible) { + return [{ + configured: model, + catalogModel: visibleCompatible.slug as string, + reason: "outside_display_limit", + }]; + } + const visible = matchingEntries.find(entry => entry.visibility === "list"); + if (visible) { + return [{ + configured: model, + catalogModel: visible.slug as string, + reason: "surface_incompatible", + }]; + } + const hidden = configuredCatalogEntry(entries, model) ?? matchingEntries[0]!; + return [{ configured: model, catalogModel: hidden.slug as string, reason: "picker_hidden" }]; + }); + return { candidates, advertised, excluded }; +} diff --git a/src/codex/catalog/sync.ts b/src/codex/catalog/sync.ts index d3380020b0..373bcb4377 100644 --- a/src/codex/catalog/sync.ts +++ b/src/codex/catalog/sync.ts @@ -1,2698 +1,52 @@ -import { effectiveProviderAlias } from "../../providers/default-aliases"; -import { pendingModelSelectionProviders } from "../../providers/initial-model-selection"; -import { execFileSync } from "node:child_process"; -import { createHash } from "node:crypto"; -import { existsSync, readFileSync } from "node:fs"; -import { delimiter, dirname, join, resolve } from "node:path"; -import { expandUserPath, loadConfig, readConfigDiagnostics, websocketsEnabled } from "../../config"; -import { shouldSyncCodexOnStart } from "../desired-state"; -import { legacyCustomModelCatalogSlugs } from "../custom-model-catalog-migration"; -import { CODEX_CONFIG_PATH, CODEX_MODELS_CACHE_PATH, DEFAULT_CATALOG_PATH, getCodexHome, readRootTomlString, resolveCodexConfigPath } from "../paths"; -import { clearModelCache, DEFAULT_MODEL_CACHE_TTL_MS, getFreshCached, getStaleCached, isModelsFetchCoolingDown, markModelsFetchFailure, setCached } from "../model-cache"; -import { buildModelsRequest, resolveModelsAuthToken } from "../../oauth"; -import type { OcxConfig, OcxProviderConfig } from "../../types"; -import { modelInList } from "../../types"; -import { CODEX_REASONING_LEVELS, codexEffortRank, configuredReasoningEfforts, modelRecordValue, sanitizeCodexReasoningEfforts } from "../../reasoning-effort"; -import { getModelMetadata, getModelMetadataCaseInsensitive, listModelMetadata, resolveMetadataProvider } from "../../generated/model-metadata"; -import { enrichProviderFromRegistry, shouldCaseFoldMetadataModelId } from "../../providers/derive"; -import { applyProviderContextCap, providerContextCap } from "../../providers/context-cap"; -import { encodeRoutedModelId, routedSlug, slugEquals, slugEquivalenceKey, slugsEquivalent } from "../../providers/slug-codec"; -import { canonicalAutoReviewModelKey, isValidAutoReviewModel as isValidAutoReviewTarget } from "../../config/provider-validation"; -import { identifyRoutedModel } from "../../adapters/identity"; -import { filterCursorConfiguredModelsByLiveDiscovery } from "../../adapters/cursor/discovery"; -import { fetchCursorUsableModels } from "../../adapters/cursor/live-models"; -import { - COMBO_NAMESPACE, - comboModelId, - getCombo, - listComboIds, - targetKey, -} from "../../combos"; -import type { NormalizedComboConfig } from "../../combos/types"; -import { providerDestinationResolvedError } from "../../lib/destination-policy"; -import { redactSecretString } from "../../lib/redact"; -import upstreamModelsSnapshot from "../data/upstream-models.json"; -import { OPENAI_CODEX_PROVIDER_ID } from "../../providers/openai-tiers"; -import { providerCodexAccountMode } from "../../providers/registry"; -import { codexAccountNamespaceEntries, isMainCodexAccountTarget } from "../account-namespaces"; -import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; -import { - availableAccountGatedNativeModels, - codexModelEntitlementStateForAccount, - isCodexModelEntitlementSnapshotCurrent, - resolveCodexModelEntitlements, - type CodexModelEntitlementSnapshot, -} from "../model-entitlements"; -import { isAccountNeedsReauth } from "../account-runtime-state"; -import { codexAccountLogLabel, fallbackCodexAccountLogLabel } from "../account-label"; - - -import { CODEX_CUSTOM_MODEL_CATALOG_KIND, CODEX_PROVIDER_MODEL_CATALOG_KIND, activeCodexModelsCachePath, applyCatalogMetadata, applyMultiAgentMode, applyNativeOpenAiContextOverride, applyRoutedCodexToolMode, catalogBackupPathFor, catalogHasRoutedEntries, catalogModelSlug, ensureStrictCatalogFields, findNativeTemplate, findSupportedNativeTemplate, isDefaultCatalogPath, isRoutedModelCompatibilityExcluded, legacyCatalogBackupPath, normalizeRoutedCatalogEntry, normalizeServiceTiers, readCatalog, readCatalogBackup, readCodexCatalogPath, readCodexCatalogPathForHome, readConfiguredAutoReviewModel, readNativeBaseline } from "./parsing"; -import type { CatalogModel, MultiAgentMode, RawCatalog, RawEntry } from "./parsing"; -import { accountBoundNativeOpenAiSlugs, accountBoundNativeOpenAiSlugsBySelector, applyNativeVisibility, CODEX_NATIVE_ALIAS_CATALOG_KIND, desktopAllowlistSuppressedNativeSlugs, disabledNativeSlugs, hasNativeOpenAiCapabilityMetadata, isNativeAliasCatalogEntry, isUnsupportedOpenAiNativeSlug, NATIVE_OPENAI_MODELS, RETIRED_NATIVE_OPENAI_MODELS, nativeContextLimits, observedAccountBoundNativeEntries, shouldIncludeAccountBoundNativeOpenAi, shouldIncludeNativeOpenAi, shouldUpgradeToUpstreamEntry, SUPPORTED_NATIVE_OPENAI_SLUGS, upstreamNativeEntry, type NativeContextLimitsInput } from "./metadata"; -import { - bundledCatalogCacheState, - loadBundledCodexCatalog, - resetBundledCatalogCacheForTests, -} from "./bundled"; -import { isMultiAgentV2Enabled } from "../features"; -import { applyCatalogModelMetadata, applyReasoningLevels, catalogEntryEfforts, clampCatalogModelsToCodexSupport, ensureGpt56ReasoningLevels, ensureUltraReasoningLevel, isGpt56NativeSlug } from "./effort"; -import { - clearGatherRoutedModelsInflight, - filterCatalogVisibleModels, - gatherRoutedModels, - lastDropWarnSignature, - type CatalogGatherProviderModelOutcome, -} from "./provider-fetch"; -import { accountSelectorShadowCollisionWarnings, clearLastComboCatalogOmissions, comboCatalogWarningSignatures, comboMasqueradeCollisionWarnings, comboUnrestorableShadowWarnings, exactComboCatalogSlugs, openAiApiCollisionWarnings, resolveSlugAliasCollisions, slugAliasCollisionWarnings, warnAccountSelectorShadowedProviderOnce, warnComboMasqueradeCollisionOnce, warnComboUnrestorableShadowOnce } from "./aggregation"; -import type { ComboCatalogOmission } from "./aggregation"; -import { - withCatalogWriteSerialization, - type CatalogWritePermit, -} from "../catalog-write-serialization"; -import { - publishHashedCodexCatalogBackup, - publishLegacyCodexCatalogBackup, - replaceActiveCodexCatalog, - replaceCodexModelsCache, -} from "../internal/catalog-writer"; -import { codexRuntimeStatePath } from "../runtime"; -import { accountBoundNativeDisplayName, CODEX_ACCOUNT_BOUND_CATALOG_KIND, trustedAccountBoundNativeCatalogSlug, visibleCodexAccountSelectors } from "./account-models"; -import { ACCOUNT_GATED_NATIVE_OPENAI_MODELS, NATIVE_RESERVE_MODEL } from "./native-models"; -import { observedReserveCatalogSource } from "./metadata"; -import { - createReserveCatalogProjection, - isReserveCatalogProjection, - RESERVE_LUNA_METADATA_SOURCE, - RESERVE_SOURCE_CATALOG_FIELD, - type ReserveCatalogProjection, -} from "./reserve"; - -export const MAX_SPAWN_AGENT_MODEL_OVERRIDES = 5; - -// Base for config.modelPickerOrder display priorities (#1649). modelPickerOrder is a DISPLAY-ONLY -// reordering of the Codex model picker: it rewrites a row's Codex-visible `priority` but not -// OpenCodex's natural-priority guidance window. Native Codex advertisements still follow the -// visible priority and can differ from that guidance window. -export const PICKER_ORDER_PRIORITY_BASE = 1_000; - -// OpenCodex-private catalog field: the guidance candidate priority a row would have WITHOUT -// modelPickerOrder. Codex ignores unknown catalog fields (same as opencodex_catalog_kind), so this -// is invisible to Codex; effectiveSubagentRoster reads it to keep OpenCodex guidance candidates -// independent of display order. It does not freeze native advertisements. Absent on unmoved rows. -export const SPAWN_PRIORITY_FIELD = "opencodex_spawn_priority"; - -// OpenCodex-private catalog field: this row is listed but currently unable to serve (#1711). -// Codex ignores unknown catalog fields (same as opencodex_catalog_kind and the spawn priority -// above) and ensureStrictCatalogFields does not strip extras, so this is invisible to the native -// picker and cannot change what Codex offers. It never touches `visibility`. -export const CATALOG_INACTIVE_REASON_FIELD = "opencodex_inactive_reason"; - -export type SpawnAgentSurface = "v1" | "v2"; - -export type SubagentRosterExclusionReason = - | "missing_catalog_entry" - | "picker_hidden" - | "surface_incompatible" - | "outside_display_limit"; - -/** - * Whether a catalog entry may be offered as a V2 subagent model. - * - * Upstream changed this rule in codex-rs `6d4d9442c` ("Support leaf models in - * multi-agent v2"). `model_supports_multi_agent_backend` - * (core/src/tools/handlers/multi_agents_common.rs:36-42) now admits EVERY model - * except one explicitly marked `disabled`; the older `== Some(V2)` equality that - * `92938d880` introduced is gone. - * - * The field no longer answers "may I be a delegation target". It answers "does the - * CHILD get collaboration tools": `collab_tools_enabled` - * (core/src/tools/spec_plan.rs:599-610) grants a child recursive tools only when its - * own catalog value is exactly `Some(V2)`. The three-way distinction survives, but it - * now means eligible-recursive / eligible-LEAF / excluded: - * - * - `"v2"` -> eligible, and the child may itself delegate. - * - `"v1"` -> eligible LEAF worker. This is upstream's pin for `gpt-5.6-luna` - * (models-manager/models.json); excluding it here is exactly what - * kept Luna out of opencodex's roster. - * - absent/null -> eligible LEAF worker (routed or unpinned-native model). - * - `"disabled"` -> the sole capability-based exclusion. - * - * This is the roster filter only. Catalog STAMPING is a separate concern owned by - * `applyMultiAgentMode`, including the `keepNativeChatGptOnV1` policy (#1728) that - * keeps ChatGPT-native rows on `v1` so a native parent can still spawn a routed child - * despite backend-encrypted NEW_TASK bodies (#92). Recognizing those `v1` rows as - * eligible leaves here is what makes that policy usable, not a contradiction of it. - * - * Devlog: 260816_codexrs_multiagent_v2_and_history_perf/011 (C1), superseding the - * option-B decision in 260730_codex_rs_upstream_v2_live_handoff/060. - */ -export function isEligibleV2SubagentEntry(entry: RawEntry): boolean { - return entry.multi_agent_version !== "disabled"; -} - -export interface EffectiveSubagentModel { - model: string; - efforts: string[]; -} - -export interface SubagentRosterExclusion { - configured: string; - reason: SubagentRosterExclusionReason; - catalogModel?: string; -} - -export interface EffectiveSubagentRoster { - /** OpenCodex's natural-priority guidance projection, not captured native tool text. */ - candidates: EffectiveSubagentModel[]; - /** Configured models within that projection; exact-name eligibility is a separate check. */ - advertised: EffectiveSubagentModel[]; - excluded: SubagentRosterExclusion[]; -} - -export function configuredCatalogEntry(entries: readonly RawEntry[], configured: string): RawEntry | undefined { - return entries.find(entry => entry.slug === configured) - ?? entries.find(entry => typeof entry.slug === "string" && slugsEquivalent(configured, entry.slug)); -} - -function configuredSubagentModelMatchesEntry(configured: string, entry: RawEntry): boolean { - if (typeof entry.slug !== "string") return false; - if (slugsEquivalent(configured, entry.slug)) return true; - const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); - return !configured.includes("/") - && nativeSlug !== undefined - && SUPPORTED_NATIVE_OPENAI_SLUGS.has(nativeSlug) - && slugsEquivalent(configured, nativeSlug); -} - -export function effectiveSubagentRoster( - configuredModels: readonly string[], - surface: SpawnAgentSurface, - catalogEntries?: readonly RawEntry[], -): EffectiveSubagentRoster { - const configured = configuredModels - .filter(model => model.trim().length > 0) - .filter((model, index, all) => - !all.slice(0, index).some(previous => slugsEquivalent(previous, model)) - ); - const entries = catalogEntries ?? readCatalog(readCodexCatalogPath())?.models ?? []; - const ordered = entries - .map((entry, index) => ({ entry, index })) - .filter(({ entry }) => typeof entry.slug === "string") - .filter(({ entry }) => entry.visibility === "list") - .filter(({ entry }) => surface !== "v2" || isEligibleV2SubagentEntry(entry)) - .sort((left, right) => { - // OpenCodex guidance candidates rank by natural priority (SPAWN_PRIORITY_FIELD when present), - // so modelPickerOrder does not change this projection. Native tool advertisements differ. Rows the - // override did not move fall back to their Codex-visible `priority`. - const spawnPriorityOf = (entry: RawEntry): number => { - const spawn = entry[SPAWN_PRIORITY_FIELD]; - if (typeof spawn === "number" && Number.isFinite(spawn)) return spawn; - return typeof entry.priority === "number" && Number.isFinite(entry.priority) - ? entry.priority : Number.MAX_SAFE_INTEGER; - }; - const leftPriority = spawnPriorityOf(left.entry); - const rightPriority = spawnPriorityOf(right.entry); - return leftPriority - rightPriority || left.index - right.index; - }) - .slice(0, MAX_SPAWN_AGENT_MODEL_OVERRIDES); - const orderedEntries = new Set(ordered.map(({ entry }) => entry)); - - const candidates = ordered.map(({ entry }) => ({ - model: entry.slug as string, - efforts: catalogEntryEfforts(entry), - })); - const advertised = ordered - .filter(({ entry }) => configured.some(model => configuredSubagentModelMatchesEntry(model, entry))) - .map(({ entry }) => ({ - model: entry.slug as string, - efforts: catalogEntryEfforts(entry), - })); - const excluded = configured.flatMap((model): SubagentRosterExclusion[] => { - const matchingEntries = entries.filter(entry => configuredSubagentModelMatchesEntry(model, entry)); - if (matchingEntries.some(entry => orderedEntries.has(entry))) return []; - if (matchingEntries.length === 0) return [{ configured: model, reason: "missing_catalog_entry" }]; - const visibleCompatible = matchingEntries.find(entry => - entry.visibility === "list" - && (surface !== "v2" || isEligibleV2SubagentEntry(entry)) - ); - if (visibleCompatible) { - return [{ - configured: model, - catalogModel: visibleCompatible.slug as string, - reason: "outside_display_limit", - }]; - } - const visible = matchingEntries.find(entry => entry.visibility === "list"); - if (visible) { - return [{ - configured: model, - catalogModel: visible.slug as string, - reason: "surface_incompatible", - }]; - } - const hidden = configuredCatalogEntry(entries, model) ?? matchingEntries[0]!; - return [{ configured: model, catalogModel: hidden.slug as string, reason: "picker_hidden" }]; - }); - return { candidates, advertised, excluded }; -} - -export function finishUpstreamNativeEntry(clone: RawEntry, priority: number, contextCap?: NativeContextLimitsInput): RawEntry { - if (priority !== 9) clone.priority = priority; - applyNativeOpenAiContextOverride(clone, contextCap); - // GPT-5.6 natives keep their exact upstream ladders (e.g. luna has max but no ultra). - // Older natives (gpt-5.5) get mock max + ultra - // (wire-clamped to xhigh). Ultra is always advertised regardless of v2 toggle. - if (!isGpt56NativeSlug(String(clone.slug ?? ""))) ensureUltraReasoningLevel(clone); - return ensureStrictCatalogFields(normalizeServiceTiers(clone)); -} - -export function isExactComboCatalogModel( - model: CatalogModel | undefined, - exactComboSlugs: ReadonlySet, -): boolean { - return model?.provider === COMBO_NAMESPACE && exactComboSlugs.has(catalogModelSlug(model)); -} - -function isExactComboCatalogEntry( - entry: RawEntry, - exactComboSlugs: ReadonlySet, -): boolean { - return entry.owned_by === COMBO_NAMESPACE - && typeof entry.slug === "string" - && exactComboSlugs.has(entry.slug); -} - -/** - * Friendly Codex-picker label for a routed `provider/model` slug. Command Code's two config - * ids differ by a single dash (`command-code` vs `commandcode`), so relabel them to the - * lowercase-dash style the opencode presets use: `commandcode-auth/x` and `commandcode-api/x`. - * The model-id portion also carries a redundant `-` prefix (`deepseek-deepseek-v4-flash`) - * that is dropped for display. Google Antigravity is relabeled to the compact `agy/` prefix for - * the same reason: `google-antigravity/` alone consumes most of the picker row. That prefix comes - * from the row's own `providerAlias`, decided once per gather flight; `null` means a cross-provider - * collision suppressed it and the canonical slug stands. This is the raw-slug path only -- a - * configured `modelAliases` entry is labeled by the effective-alias path in - * catalog/provider-fetch.ts (#2960) and keeps the canonical provider name. All other providers - * keep the raw slug exactly as before. - */ -function routedDisplayName(slug: string, model?: CatalogModel, config?: Pick): string { - const slash = slug.indexOf("/"); - if (slash <= 0) return slug; - const provider = slug.slice(0, slash); - let modelId = slug.slice(slash + 1); - if (provider === "google-antigravity") { - if (model?.providerAlias === null) return slug; - const alias = (typeof model?.providerAlias === "string" && model.providerAlias.trim().length > 0) - ? model.providerAlias.trim() - : effectiveProviderAlias(provider, undefined, config); - return alias ? `${alias}/${modelId}` : slug; - } - if (provider === "command-code" || provider === "commandcode") { - const m = modelId.match(/^([a-z0-9]+)-([a-z0-9]+(?:-[a-z0-9]+)+)$/i); - if (m && modelId.startsWith(`${m[1]}-${m[1]}-`)) modelId = modelId.slice(m[1]!.length + 1); - return `${provider === "command-code" ? "commandcode-auth" : "commandcode-api"}/${modelId}`; - } - return slug; -} - -function preservePinnedNativeCustomReasoning(model?: CatalogModel): boolean { - return model !== undefined - && model.catalogKind === CODEX_CUSTOM_MODEL_CATALOG_KIND - && hasNativeOpenAiCapabilityMetadata(model.id) - && Array.isArray(model.reasoningEfforts); -} - -/** - * Cria uma entrada nativa ou roteada a partir do snapshot upstream, de um clone - * do template ou de campos mínimos. Aplica os metadados e limites pertinentes - * sem alterar o template nem herdar sua marca de nome ou histórico de prioridade. - */ -export function deriveEntry( - template: RawEntry | null, - slug: string, - desc: string, - priority: number, - model?: CatalogModel, - exactComboSlugs: ReadonlySet = new Set(), - contextCap?: NativeContextLimitsInput, -): RawEntry { - const preserveExact = isExactComboCatalogModel(model, exactComboSlugs); - // Go exposes model-specific upstream enums; synthetic tiers mislead subagent overrides. - const preserveExactReasoning = preserveExact || model?.provider === "opencode-go"; - const codexForwardNativeCapabilityAlias = model?.codexForwardNativeCapabilityAlias === true - ? upstreamNativeEntry(model.id) - : null; - const isRouted = model !== undefined; - if (!isRouted && !slug.includes("/")) { - // Supported native slug covered by the upstream snapshot: use the REAL entry (exact - // reasoning ladder — e.g. luna has no ultra — default effort, identity, model_messages) - // instead of cloning an older template. - const upstream = upstreamNativeEntry(slug); - if (upstream) return finishUpstreamNativeEntry(upstream, priority, contextCap); - } - if (template || codexForwardNativeCapabilityAlias) { - const e = JSON.parse(JSON.stringify(codexForwardNativeCapabilityAlias ?? template)) as RawEntry; - delete e.opencodex_native_display_name; - // A cached template may carry display-order history; each new row owns its natural rank. - delete e[SPAWN_PRIORITY_FIELD]; - e.slug = slug; - e.display_name = routedDisplayName(slug, model); - e.description = desc; - e.priority = priority; - e.visibility = "list"; - if ("upgrade" in e) e.upgrade = null; - delete e.availability_nux; // don't replay another model's "now available" NUX - // Routed (namespaced) models inherit the gpt template — correct its OpenAI/GPT identity - // and advertise the reasoning ladder Codex accepts. - if (isRouted) { - // A routed model is NOT the native template: never inherit its context - // window when /models omits context metadata (#992). Known metadata - // restores exact values below; an enabled Context cap fills the gap; - // otherwise the strict-fields fallback supplies the 128k triple. - if (!codexForwardNativeCapabilityAlias) { - delete e.context_window; - delete e.max_context_window; - delete e.auto_compact_token_limit; - } - // Native id for identity text + metadata lookups — the slug may be an encoded - // alias (`provider/vendor-model`); the model object carries the native id. - const modelName = model?.id ?? slug.slice(slug.indexOf("/") + 1); - if (typeof e.base_instructions === "string") { - // Proxy-neutral: keep the GPT-5/OpenAI disclaimer but never advertise the opencodex proxy - // (leaking that into base_instructions is a non-first-party signature → ToS risk). - e.base_instructions = identifyRoutedModel(e.base_instructions, modelName); - } - applyReasoningLevels( - e, - model?.reasoningEfforts, - model?.defaultReasoningEffort, - preserveExactReasoning - || codexForwardNativeCapabilityAlias !== null - || preservePinnedNativeCustomReasoning(model), - ); - // This exact provider/model pair is the ChatGPT/Codex forward surface. Keep the pinned - // native tool/search/responses-lite contract while preserving the routed slug and wire id. - if (!codexForwardNativeCapabilityAlias) { - normalizeRoutedCatalogEntry(e, model?.parallelToolCalls === true, model?.codexToolMode); - } else if (model?.codexToolMode !== undefined) { - applyRoutedCodexToolMode(e, model.codexToolMode); - } - if (model) applyCatalogMetadata(e, model.provider, model.id, model.contextCap); - applyCatalogModelMetadata(e, model); - if (model?.catalogKind) e.opencodex_catalog_kind = model.catalogKind; - // Additive only. `visibility` is untouched: an inactive row must still be OFFERED, which is - // the whole point of #1711 — operator disable is what removes rows, and it stays a separate - // path from this one. - if (model?.quotaInactiveReason) e[CATALOG_INACTIVE_REASON_FIELD] = model.quotaInactiveReason; - } else { - applyNativeOpenAiContextOverride(e, contextCap); - if (isGpt56NativeSlug(slug)) ensureGpt56ReasoningLevels(e); - else ensureUltraReasoningLevel(e); - // Older natives do not support Responses Lite. A newer template must not enable - // reasoning.context or WebSockets on those models. - if (!isGpt56NativeSlug(slug)) { - delete e.use_responses_lite; - delete e.supports_websockets; - } - } - return ensureStrictCatalogFields(normalizeServiceTiers(e), { - preserveExactInputModalities: preserveExact, - isRouted, - }); - } - // Fallback when no template is available (best-effort; strict parser may need more). - // Routed fallbacks default to code-mode tool exposure (or shell mode when codexToolMode === "shell"); - // otherwise the nested catalog expands into `exec.description` and can exceed Cursor's 120 KB serialized tool limit (#1830). - // Cursor still omits hosted web-search metadata because runTurn bypasses that separate sidecar. - const isCursorFallback = isRouted && model?.provider === "cursor"; - const entry: RawEntry = { - slug, display_name: routedDisplayName(slug, model), description: desc, - shell_type: "unified_exec", visibility: "list", supported_in_api: true, - priority, base_instructions: "You are a helpful coding assistant.", - ...(isRouted - ? isCursorFallback - ? { supports_search_tool: true } - : { web_search_tool_type: "text_and_image", supports_search_tool: true } - : {}), - }; - if (isRouted) { - applyRoutedCodexToolMode(entry, model?.codexToolMode); - applyReasoningLevels(entry, model?.reasoningEfforts, model?.defaultReasoningEffort, preserveExactReasoning || preservePinnedNativeCustomReasoning(model)); - } - else { - applyReasoningLevels(entry, isGpt56NativeSlug(slug) ? undefined : ["low", "medium", "high", "xhigh"]); - if (isGpt56NativeSlug(slug)) ensureGpt56ReasoningLevels(entry); - } - if (model && isRouted) applyCatalogMetadata(entry, model.provider, model.id, model.contextCap); - applyCatalogModelMetadata(entry, model); - if (model?.catalogKind) entry.opencodex_catalog_kind = model.catalogKind; - // Same additive stamp as the templated path above. A routed row that reaches the no-template - // fallback is still a served row, so omitting it here would make the field depend on whether a - // template happened to be cached — which is exactly what the regression test caught. - if (model?.quotaInactiveReason) entry[CATALOG_INACTIVE_REASON_FIELD] = model.quotaInactiveReason; - if (!isRouted) applyNativeOpenAiContextOverride(entry, contextCap); - return ensureStrictCatalogFields(normalizeServiceTiers(entry), { - preserveExactInputModalities: preserveExact, - isRouted, - }); -} - -export interface ObservedCatalogEntryBuildInput { - readonly template: RawEntry | null; - readonly gptSlugs: readonly string[]; - readonly goModels: readonly CatalogModel[]; - readonly featured?: readonly string[]; - /** Optional full picker ordering (config.modelPickerOrder); orders non-featured rows. */ - readonly modelPickerOrder?: readonly string[]; - readonly wsEnabled: boolean; - readonly multiAgentMode: MultiAgentMode; - readonly exactComboSlugs: ReadonlySet; - readonly accountSelectors: readonly string[]; - readonly suppressedBareNativeSlugs: ReadonlySet; - readonly disabledNativeAccountSlugs: ReadonlySet; - readonly multiAgentV2Enabled: boolean; - readonly keepNativeChatGptOnV1?: boolean; - readonly openaiContextCap?: NativeContextLimitsInput; - /** Additional native ids to clone under account selectors, without creating bare rows. */ - readonly accountNativeSlugs?: readonly string[]; - /** Per-selector account ids; unknown observations must not be copied to unrelated accounts. */ - readonly accountNativeSlugsBySelector?: ReadonlyMap; - /** Codex-only manual selector metadata; deliberately independent of live permission. */ - readonly reserve?: ReserveCatalogProjection; -} - -/** Build entries with the process-observed Codex feature state. */ -export function buildCatalogEntries( - template: RawEntry | null, - gptSlugs: string[], - goModels: CatalogModel[], - featured?: string[], - wsEnabled = false, - multiAgentMode: MultiAgentMode = "default", - exactComboSlugs: ReadonlySet = new Set(), - accountSelectors: readonly string[] = [], - suppressedBareNativeSlugs: ReadonlySet = new Set(), - disabledNativeAccountSlugs: ReadonlySet = new Set(), - contextCap?: NativeContextLimitsInput, - accountNativeSlugs?: readonly string[], - accountNativeSlugsBySelector?: ReadonlyMap, - keepNativeChatGptOnV1 = false, - modelPickerOrder: readonly string[] = [], -): RawEntry[] { - const entries = buildCatalogEntriesFromObservedState({ - template, - gptSlugs, - goModels, - featured, - modelPickerOrder, - wsEnabled, - multiAgentMode, - exactComboSlugs, - accountSelectors, - suppressedBareNativeSlugs, - disabledNativeAccountSlugs, - multiAgentV2Enabled: isMultiAgentV2Enabled(), - keepNativeChatGptOnV1, - openaiContextCap: contextCap, - accountNativeSlugs, - accountNativeSlugsBySelector, - }); - applyFullModelPickerOrder(entries, modelPickerOrder); - return entries; -} - -/** Build entries solely from caller-observed inputs, with no feature-state filesystem read. */ -export function buildCatalogEntriesFromObservedState({ - template, - gptSlugs, - goModels, - featured, - modelPickerOrder, - wsEnabled, - multiAgentMode, - exactComboSlugs, - accountSelectors, - suppressedBareNativeSlugs, - disabledNativeAccountSlugs, - multiAgentV2Enabled, - keepNativeChatGptOnV1, - openaiContextCap, - accountNativeSlugs, - accountNativeSlugsBySelector, - reserve, -}: ObservedCatalogEntryBuildInput): RawEntry[] { - // Codex's models-manager sorts by `priority` ASC and advertises the first 5 picker-visible - // models to spawn_agent (sort_by_key(priority) + MAX_MODEL_OVERRIDES_IN_SPAWN_AGENT=5). Catalog - // ARRAY order is discarded — so "featuring" a model = giving it the LOWEST priority (0..N-1) so - // it sorts to the front. This works for native gpt slugs AND routed slugs alike. - const rank = new Map((featured ?? []).map((slug, i) => [slug, i] as const)); - const priorityStride = Math.max(accountSelectors.length, 1); - // Optional full picker order (#1649). Independent of the 5-slot spawn_agent cap: it only - // rewrites the Codex-visible display `priority` of listed non-featured routed rows so a >5 - // catalog stays put across rebuilds. Featured rows keep their existing 0..N-1 band; when - // modelPickerOrder is unset the helper is a no-op and every priority below is byte-identical to - // before. The spawn_agent candidate window is derived separately from SPAWN_PRIORITY_FIELD, so - // this display reorder does not change OpenCodex's guidance candidate calculation. - const pickerOrder = normalizeModelPickerOrder(modelPickerOrder); - const pickerOrderRank = new Map(pickerOrder.map((slug, i) => [slug, i] as const)); - const pickerOrderActive = pickerOrder.length > 0; - // The display band reuses the existing high priority tier (>= PICKER_ORDER_PRIORITY_BASE, the - // same 1_000+ neighborhood account rows occupy), keeping listed rows visually after the featured - // band. OpenCodex guidance membership does not depend on this — see SPAWN_PRIORITY_FIELD. - /** - * Priority for a non-featured routed row that is explicitly LISTED in modelPickerOrder. Listed - * slugs sort in declared order within the high picker-order display tier - * (>= PICKER_ORDER_PRIORITY_BASE). This sets the Codex-visible `priority` only; the caller records - * the row's natural priority in SPAWN_PRIORITY_FIELD for OpenCodex's unchanged guidance window. - * Returns undefined when the feature is off or the row is not listed, so those rows - * keep their original assignment (default 5 / account 1_000+) untouched. - * - * Scope: only the generic routed `/` rows call this (see the goModels loop - * below). Native passthrough rows and account-qualified native rows keep their own priority - * logic and are intentionally not reordered in this legacy builder pass. The final merge can - * apply complete ordering when the configured list includes a bare id. - */ - const pickerOrderPriority = (slug: string, altSlug?: string): number | undefined => { - if (!pickerOrderActive) return undefined; - const hit = pickerOrderRank.get(slug) ?? (altSlug !== undefined ? pickerOrderRank.get(altSlug) : undefined); - if (hit === undefined) return undefined; - return PICKER_ORDER_PRIORITY_BASE + hit * priorityStride; - }; - const out: RawEntry[] = []; - const nativeEntries: RawEntry[] = []; - const collisionSkipped = resolveSlugAliasCollisions([...goModels]); - const emittedNativeAliases = new Set(); - const emittedNativeAliasSlugs = new Set(); - const nativeAliasesBySlug = new Map(); - for (const model of goModels) { - if (model.provider !== COMBO_NAMESPACE - || model.nativeAlias !== true - || typeof model.alias !== "string" - || model.alias.includes("/")) continue; - if (nativeAliasesBySlug.has(model.alias)) { - collisionSkipped.add(model); - if (!slugAliasCollisionWarnings.has(model.alias)) { - slugAliasCollisionWarnings.add(model.alias); - console.warn( - `[opencodex] native combo alias collision on "${model.alias}": keeping the first configured combo and omitting later duplicates from the catalog.`, - ); - } - continue; - } - nativeAliasesBySlug.set(model.alias, model); - } - const comboPublicSlugs = new Set(goModels - .filter(model => model.provider === COMBO_NAMESPACE) - .map(catalogModelSlug)); - for (const slug of gptSlugs) { - const native = deriveEntry(template, slug, "OpenAI native model (Codex OAuth passthrough).", 9, undefined, new Set(), openaiContextCap); - if (rank.has(slug)) native.priority = rank.get(slug)!; - nativeEntries.push(native); - const nativeAlias = nativeAliasesBySlug.get(slug); - if (!nativeAlias || collisionSkipped.has(nativeAlias)) { - if (!suppressedBareNativeSlugs.has(slug)) out.push(native); - continue; - } - const routed = deriveEntry( - template, - slug, - `Routed via opencodex → ${nativeAlias.provider} (${nativeAlias.owned_by ?? nativeAlias.provider}).`, - 5, - nativeAlias, - exactComboSlugs, - ); - routed.opencodex_catalog_kind = CODEX_NATIVE_ALIAS_CATALOG_KIND; - const rankHit = rank.get(slug) ?? rank.get(`${nativeAlias.provider}/${nativeAlias.id}`); - if (rankHit !== undefined) routed.priority = rankHit * priorityStride; - else if (accountSelectors.length > 0) routed.priority = 1_000 + (typeof routed.priority === "number" ? routed.priority : 5); - out.push(routed); - emittedNativeAliases.add(nativeAlias); - emittedNativeAliasSlugs.add(slug); - } - const nativeEntriesBySlug = new Map(nativeEntries.map(entry => [String(entry.slug), entry] as const)); - for (const [selectorIndex, selector] of accountSelectors.entries()) { - const selectorNativeSlugs = accountNativeSlugsBySelector?.get(selector) - ?? accountNativeSlugs - ?? gptSlugs; - const accountNativeEntries = selectorNativeSlugs.filter(slug => slug !== NATIVE_RESERVE_MODEL).map(slug => ( - nativeEntriesBySlug.get(slug) - ?? deriveEntry(template, slug, "OpenAI native model (Codex OAuth passthrough).", 9, undefined, new Set(), openaiContextCap) - )); - if (reserve?.mainSelectors.includes(selector)) accountNativeEntries.push(reserve.source); - for (const [nativeIndex, native] of accountNativeEntries.entries()) { - const nativeSlug = String(native.slug); - if (disabledNativeAccountSlugs.has(nativeSlug)) continue; - const e = JSON.parse(JSON.stringify(native)) as RawEntry; - const catalogSlug = `${selector}/${nativeSlug}`; - if (nativeSlug === NATIVE_RESERVE_MODEL && disabledNativeAccountSlugs.has(catalogSlug)) continue; - e.slug = catalogSlug; - e.display_name = accountBoundNativeDisplayName(selector, native); - // Codex ignores this OpenCodex extension; preserve the native comp_hash unchanged. - e.opencodex_catalog_kind = CODEX_ACCOUNT_BOUND_CATALOG_KIND; - const exactRank = rank.get(catalogSlug); - // A bare featured id belongs to the compatibility combo once shadowed. Exact - // account-qualified picks still rank normally, but the account clone must not - // inherit the bare alias rank and consume another top spawn_agent slot. - const inheritedRank = emittedNativeAliasSlugs.has(nativeSlug) ? undefined : rank.get(nativeSlug); - const featuredRank = exactRank ?? inheritedRank; - e.priority = featuredRank !== undefined - ? featuredRank * priorityStride + selectorIndex - : ((featured?.length ?? 0) + nativeIndex) * accountSelectors.length + selectorIndex; - e.visibility = "list"; - out.push(e); - } - } - for (const m of goModels) { - if (collisionSkipped.has(m) || emittedNativeAliases.has(m)) continue; - const slug = catalogModelSlug(m); - if (m.provider !== COMBO_NAMESPACE && comboPublicSlugs.has(slug)) { - warnComboMasqueradeCollisionOnce(slug); - continue; - } - // Provider rows use the one-slash slug codec; combo aliases intentionally override that - // public slug and may be bare. - const e = deriveEntry( - template, - slug, - `Routed via opencodex → ${m.provider} (${m.owned_by ?? m.provider}).`, - 5, - m, - exactComboSlugs, - ); - if (m.provider === COMBO_NAMESPACE && m.nativeAlias === true && !slug.includes("/")) { - e.opencodex_catalog_kind = CODEX_NATIVE_ALIAS_CATALOG_KIND; - } - // Featured picks may be stored raw (legacy) or encoded — honor both. - const rankHit = rank.get(slug) ?? rank.get(`${m.provider}/${m.id}`); - // Natural priority: what the row would get WITHOUT modelPickerOrder. This is the value the - // spawn_agent candidate window is derived from (see effectiveSubagentRoster), so it must never - // move when modelPickerOrder reorders the picker. - if (rankHit !== undefined) e.priority = rankHit * priorityStride; - else if (accountSelectors.length > 0) { - // Keep the generated account rows together in Codex's priority-sorted flat picker. - e.priority = 1_000 + (typeof e.priority === "number" ? e.priority : 5); - } - // The legacy routed-only builder pass keeps featured ranks and records natural priority - // before changing non-featured display priority. The final complete-order pass may move - // featured display rows too; OpenCodex guidance continues to use their natural ranks. - if (rankHit === undefined) { - const pickerPriority = pickerOrderPriority(slug, `${m.provider}/${m.id}`); - if (pickerPriority !== undefined) { - e[SPAWN_PRIORITY_FIELD] = typeof e.priority === "number" ? e.priority : 5; - e.priority = pickerPriority; - } - } - out.push(e); - } - // Central capability override (phase 120.4): the advertised flag must match the implemented WS - // endpoint. Overrides both the routed strip (normalizeRoutedCatalogEntry) and any native template - // leak (deriveEntry clones the template as-is for native slugs). - for (const entry of out) { - if (wsEnabled) entry.supports_websockets = true; - else { - delete entry.supports_websockets; - // Snapshot-backed native entries carry prefer_websockets: never advertise a preference - // for an endpoint ocx has disabled. - delete entry.prefer_websockets; - } - } - return applyMultiAgentMode(out, multiAgentMode, multiAgentV2Enabled, { - keepNativeChatGptOnV1, - preserveDefaultMultiAgentVersion: isReserveCatalogProjection, - }); -} - -export function resetCatalogRuntimeStateForTests(): void { - resetBundledCatalogCacheForTests(); - lastDropWarnSignature.clear(); - openAiApiCollisionWarnings.clear(); - comboCatalogWarningSignatures.clear(); - slugAliasCollisionWarnings.clear(); - comboMasqueradeCollisionWarnings.clear(); - comboUnrestorableShadowWarnings.clear(); - accountSelectorShadowCollisionWarnings.clear(); - clearLastComboCatalogOmissions(); - clearModelCache(undefined, "eviction"); - clearGatherRoutedModelsInflight(); -} - -export function orderForSubagents(goModels: CatalogModel[], featured?: string[]): CatalogModel[] { - if (!featured || featured.length === 0) return goModels; - const rank = new Map(featured.map((id, i) => [id, i])); - // Featured picks may be stored raw (legacy) or encoded — match both forms. - const rankOf = (m: CatalogModel) => - (m.alias ? rank.get(m.alias) : undefined) - ?? rank.get(`${m.provider}/${m.id}`) - ?? rank.get(routedSlug(m.provider, m.id)) - ?? Number.MAX_SAFE_INTEGER; - return [...goModels].sort((a, b) => { - return rankOf(a) - rankOf(b); - }); -} - -/** Routed discovery projection; native groups and alias ownership belong to the caller. */ -export function orderForModelPicker( - models: readonly CatalogModel[], - order: readonly string[] = [], - featured: readonly string[] = [], -): CatalogModel[] { - const pickerOrder = normalizeModelPickerOrder(order); - if (pickerOrder.length === 0) return [...models]; - const pickerRank = modelPickerRank(pickerOrder); - const featuredRank = modelPickerRank(featured); - const complete = pickerOrder.some(slug => !slug.includes("/")); - const rank = (model: CatalogModel): number => { - const slug = catalogModelSlug(model); - const featuredIndex = featuredRank(slug) ?? featuredRank(`${model.provider}/${model.id}`); - const natural = featuredIndex ?? 5; - const index = pickerRank(slug) ?? pickerRank(`${model.provider}/${model.id}`); - if (complete) return index ?? pickerOrder.length + natural; - // Preserve the legacy featured/alias bands, including unlisted rows before listed rows. - if (featuredIndex !== undefined || model.nativeAlias === true) return natural; - return index === undefined ? natural : PICKER_ORDER_PRIORITY_BASE + index; - }; - return [...models].sort((a, b) => rank(a) - rank(b)); -} - -/** - * True when an existing catalog row was authored by OpenCodex routing (#855). - * Every generated routed row — current full-slug form, the June–July 2026 - * provider-name form, and legacy combo aliases — carries the stable - * description prefix `Routed via opencodex → `; foreign rows from Cursor or - * user tooling do not. `owned_by` cannot serve as the signal (upstream - * ownership), and `comp_hash` defaults to "opencodex" for every normalized - * row. - */ -function isOcxAuthoredRoutedEntry(entry: RawEntry): boolean { - if (isNativeAliasCatalogEntry(entry)) return true; - const desc = typeof entry.description === "string" ? entry.description : ""; - const slug = typeof entry.slug === "string" ? entry.slug : ""; - return slug.includes("/") && desc.startsWith("Routed via opencodex → "); -} - -function recoverableNativeSlug(entry: RawEntry): string | null { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - return SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug) - && !isNativeAliasCatalogEntry(entry) - && entry.owned_by !== COMBO_NAMESPACE - ? slug - : null; -} - -/** Undo our display overlay before native metadata normalization and template reuse. */ -function restoreNativeDisplayName(entry: RawEntry): RawEntry { - const saved = entry.opencodex_native_display_name; - delete entry.opencodex_native_display_name; - if (saved && typeof saved === "object" && !Array.isArray(saved)) { - const label = saved as Record; - if (recoverableNativeSlug(entry) === label.slug - && typeof label.original === "string" && entry.display_name === label.applied) { - entry.display_name = label.original; - } - } - return entry; -} - -/** Append missing supported native rows from trusted catalog sources only. */ -export function mergeCatalogModelsWithNativeRecovery( - primaryCatalogModels: readonly RawEntry[], - nativeRecoverySources: readonly (readonly RawEntry[])[], -): RawEntry[] { - const merged = [...primaryCatalogModels]; - const recoveredNativeSlugs = new Set(primaryCatalogModels.flatMap(entry => { - const slug = recoverableNativeSlug(entry); - return slug === null ? [] : [slug]; - })); - for (const source of nativeRecoverySources) { - for (const entry of source) { - const slug = recoverableNativeSlug(entry); - if (slug === null || recoveredNativeSlugs.has(slug)) continue; - merged.push(structuredClone(entry) as RawEntry); - recoveredNativeSlugs.add(slug); - } - } - return merged; -} - -export interface ObservedCatalogMergePolicy { - /** Required observed/fixed set; the core merge never consults ambient catalog state. */ - readonly nativeBackfillSlugs: readonly string[]; - /** Whether unsupported OpenAI-family bare rows survive the merge. */ - readonly unsupportedNativeEntries: "preserve" | "drop"; - /** Whether merge-policy collision/preservation warnings belong to this caller's flow. */ - readonly warningPolicy: "emit" | "suppress"; -} - -/** Content policy shared by every writer of the canonical Codex model catalog. */ -export const CANONICAL_NATIVE_CATALOG_CONTENT_POLICY: Readonly< - Pick -> = Object.freeze({ - nativeBackfillSlugs: Object.freeze([...NATIVE_OPENAI_MODELS]), - unsupportedNativeEntries: "drop", -}); - -function normalizeModelPickerOrder(order: unknown): string[] { - return Array.isArray(order) - ? order.filter((id): id is string => typeof id === "string" && id.trim().length > 0) - : []; -} - -/** Preserve exact-id precedence while accepting the existing raw/encoded slug spellings. */ -function modelPickerRank(order: readonly string[]): (slug: string) => number | undefined { - const exact = new Map(order.map((slug, index) => [slug, index])); - const equivalent = new Map(order.map((slug, index) => [slugEquivalenceKey(slug), index])); - return slug => exact.get(slug) ?? equivalent.get(slugEquivalenceKey(slug)); -} - -/** Complete display ordering retains natural ranks for OpenCodex's separate guidance projection. */ -export function applyFullModelPickerOrder(entries: RawEntry[], order: readonly string[]): void { - const pickerOrder = normalizeModelPickerOrder(order); - if (!pickerOrder.some(slug => !slug.includes("/"))) return; - const rankOf = modelPickerRank(pickerOrder); - for (const entry of entries) { - const natural = entry[SPAWN_PRIORITY_FIELD] ?? entry.priority ?? 9; - entry[SPAWN_PRIORITY_FIELD] = natural; - entry.priority = rankOf(String(entry.slug)) ?? pickerOrder.length + Number(natural); - } -} - -export interface ObservedCatalogMergeInput { - readonly catalogModels: readonly RawEntry[]; - readonly baselineCatalogModels: readonly RawEntry[]; - readonly routedEntries: readonly RawEntry[]; - readonly baseline: ReadonlyMap; - readonly featured: readonly string[]; - readonly modelPickerOrder?: readonly string[]; - readonly accountSelectors?: readonly string[]; - readonly wsEnabled: boolean; - readonly template: RawEntry | null; - readonly disabledModels: ReadonlySet; - readonly selectedModelsByProvider: ReadonlyMap>; - readonly gatheredProviderNames: ReadonlySet; - readonly pendingProviderNames?: ReadonlySet; - readonly degradedProviderNames: ReadonlySet; - readonly legacyCustomModelSlugs: ReadonlySet; - readonly multiAgentMode: MultiAgentMode; - readonly multiAgentV2Enabled: boolean; - readonly keepNativeChatGptOnV1?: boolean; - readonly exactComboSlugs: ReadonlySet; - readonly hasPhysicalComboProvider: boolean; - readonly includeNativeOpenAi: boolean; - readonly accountBoundEntries: readonly RawEntry[]; - readonly suppressedBareNativeSlugs?: ReadonlySet; - readonly policy: ObservedCatalogMergePolicy; - readonly openaiContextCap?: NativeContextLimitsInput; - /** Exact display-only labels for bare native OpenAI models. */ - readonly nativeDisplayNames?: Readonly>; -} - -/** - * Deterministically merge one fully observed catalog state. - * - * Every non-catalog input is explicit so evidence-bound convergence cannot - * accidentally fall back to process-ambient catalog discovery or merge-policy warnings. - */ -export function mergeCatalogEntriesFromObservedState({ - catalogModels, - baselineCatalogModels, - routedEntries, - baseline, - featured, - modelPickerOrder = [], - accountSelectors = [], - wsEnabled, - template, - disabledModels, - selectedModelsByProvider, - gatheredProviderNames, - pendingProviderNames = new Set(), - degradedProviderNames, - legacyCustomModelSlugs, - multiAgentMode, - multiAgentV2Enabled, - keepNativeChatGptOnV1, - exactComboSlugs, - hasPhysicalComboProvider, - includeNativeOpenAi, - accountBoundEntries, - suppressedBareNativeSlugs = new Set(), - policy, - openaiContextCap, - nativeDisplayNames, -}: ObservedCatalogMergeInput): RawEntry[] { - // Raw catalog rows contain nested arrays/objects that normalization mutates. Detach every row at - // the observed-core boundary so callers can safely retain evidence objects or repeat the merge. - const detachedCatalogModels = catalogModels - .map(entry => restoreNativeDisplayName(structuredClone(entry) as RawEntry)); - const detachedBaselineCatalogModels = baselineCatalogModels - .map(entry => restoreNativeDisplayName(structuredClone(entry) as RawEntry)); - const detachedRoutedEntries = routedEntries.map(entry => structuredClone(entry) as RawEntry); - // Track this invocation's generated custom rows, not ownership markers read from disk. - // Their builder already finalized exact native ladders and ordinary routed mock tiers. - const freshCustomEntries = new Set(detachedRoutedEntries.filter(entry => - entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND)); - const detachedAccountBoundEntries = accountBoundEntries - .map(entry => structuredClone(entry) as RawEntry); - const disabledModelKeys = new Set([...disabledModels].map(slugEquivalenceKey)); - const legacyCustomModelKeys = new Set( - [...legacyCustomModelSlugs].map(slugEquivalenceKey), - ); - const selectedModelKeysByProvider = new Map([...selectedModelsByProvider].map(([provider, models]) => ( - [provider, new Set([...models].map(model => slugEquivalenceKey(routedSlug(provider, model))))] as const - ))); - const freshAccountKeys = new Set(detachedAccountBoundEntries.flatMap(entry => ( - typeof entry.slug === "string" ? [slugEquivalenceKey(entry.slug)] : [] - ))); - const wouldSurviveUnreplaced = (entry: RawEntry): boolean => { - if (entry.owned_by === COMBO_NAMESPACE - || trustedAccountBoundNativeCatalogSlug(entry) !== undefined - || entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND - || isOcxAuthoredRoutedEntry(entry) - || typeof entry.slug !== "string") return false; - const slug = entry.slug; - if (!slug.includes("/")) { - if (!includeNativeOpenAi || policy.nativeBackfillSlugs.includes(slug)) return false; - return policy.unsupportedNativeEntries === "preserve" || !isUnsupportedOpenAiNativeSlug(slug); - } - if (isRoutedModelCompatibilityExcluded(slug)) return false; - if (!hasPhysicalComboProvider && slug.startsWith(`${COMBO_NAMESPACE}/`)) return false; - const key = slugEquivalenceKey(slug); - if (freshAccountKeys.has(key)) return false; - if (disabledModelKeys.has(key)) return false; - const slash = slug.indexOf("/"); - const provider = slug.slice(0, slash); - if (pendingProviderNames.has(provider)) return false; - const selected = selectedModelKeysByProvider.get(provider); - if (selected !== undefined && !selected.has(key)) return false; - return !gatheredProviderNames.has(provider) || degradedProviderNames.has(provider); - }; - const validRoutedEntries = detachedRoutedEntries.filter(entry => { - return !isExactComboCatalogEntry(entry, exactComboSlugs) - || (Array.isArray(entry.input_modalities) && entry.input_modalities.length > 0); - }); - const restorableCatalogKeys = new Set(detachedBaselineCatalogModels.flatMap(entry => ( - wouldSurviveUnreplaced(entry) && typeof entry.slug === "string" - ? [slugEquivalenceKey(entry.slug)] - : [] - ))); - const unrestorableCatalogKeys = new Set(detachedCatalogModels.flatMap(entry => { - if (!wouldSurviveUnreplaced(entry) || typeof entry.slug !== "string") return []; - const key = slugEquivalenceKey(entry.slug); - return restorableCatalogKeys.has(key) ? [] : [key]; - })); - const admittedRoutedEntries = validRoutedEntries.filter(entry => { - if (!isExactComboCatalogEntry(entry, exactComboSlugs)) return true; - const slug = entry.slug as string; - const key = slugEquivalenceKey(slug); - if (!unrestorableCatalogKeys.has(key)) return true; - if (policy.warningPolicy === "emit") warnComboUnrestorableShadowOnce(slug); - return false; - }); - // A fresh non-custom row authoritatively resolves a historically ambiguous slug as a normal - // provider model. Persist that classification so the durable deletion evidence cannot remove - // the legitimate row during a later degraded refresh. - for (const entry of admittedRoutedEntries) { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - if (!slug - || entry.opencodex_catalog_kind !== undefined - || entry.owned_by === COMBO_NAMESPACE - || !isOcxAuthoredRoutedEntry(entry) - || !legacyCustomModelKeys.has(slugEquivalenceKey(slug))) continue; - entry.opencodex_catalog_kind = CODEX_PROVIDER_MODEL_CATALOG_KIND; - } - const freshExactComboEntries = new Set(admittedRoutedEntries.filter(entry => ( - isExactComboCatalogEntry(entry, exactComboSlugs) - && typeof entry.description === "string" - && entry.description.startsWith(`Routed via opencodex → ${COMBO_NAMESPACE} (`) - ))); - const rank = new Map(featured.map((slug, i) => [slug, i] as const)); - const freshEquivalentKeys = new Set(admittedRoutedEntries.flatMap(entry => ( - typeof entry.slug === "string" ? [slugEquivalenceKey(entry.slug)] : [] - ))); - const freshEquivalent = (slug: string): boolean => ( - freshEquivalentKeys.has(slugEquivalenceKey(slug)) - ); - const freshBareComboAliases = new Set(admittedRoutedEntries.flatMap(entry => ( - typeof entry.slug === "string" - && !entry.slug.includes("/") - && entry.owned_by === COMBO_NAMESPACE - ? [entry.slug] - : [] - ))); - const staleComboKeys = new Set(detachedCatalogModels.flatMap(entry => ( - typeof entry.slug === "string" - && entry.owned_by === COMBO_NAMESPACE - && !freshEquivalent(entry.slug) - ? [slugEquivalenceKey(entry.slug)] - : [] - ))); - const currentNonComboKeys = new Set(detachedCatalogModels.flatMap(entry => ( - entry.owned_by !== COMBO_NAMESPACE && typeof entry.slug === "string" - ? [slugEquivalenceKey(entry.slug)] - : [] - ))); - const restoredComboShadows = detachedBaselineCatalogModels.filter(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - if (!slug || entry.owned_by === COMBO_NAMESPACE) return false; - const key = slugEquivalenceKey(slug); - return staleComboKeys.has(key) && !currentNonComboKeys.has(key); - }); - const catalogModelsForMerge = [...detachedCatalogModels, ...restoredComboShadows]; - const nativePriority = (slug: string, fallback: unknown): number => { - const base = baseline.get(slug) - ?? (typeof fallback === "number" ? fallback : 9); - if (rank.has(slug)) return rank.get(slug)!; - return featured.length > 0 ? Math.max(base, featured.length + 100) : base; - }; - const nativeSourceEntries = includeNativeOpenAi - ? catalogModelsForMerge - .filter(m => typeof m.slug === "string" - && !(m.slug as string).includes("/") - && m.owned_by !== COMBO_NAMESPACE - && (policy.unsupportedNativeEntries === "preserve" - || policy.nativeBackfillSlugs.includes(m.slug as string) - || !isUnsupportedOpenAiNativeSlug(m.slug as string))) - .map(m => { - const slug = m.slug as string; - // Fallback-quality entries (ocx synthesis / codex-rs model_info fallback: display_name - // stamped with the bare slug) are upgraded to the pinned upstream snapshot entry so a - // previously synthesized ladder (e.g. luna advertising ultra) self-heals on sync. A - // genuine catalog entry (real display name) is preserved untouched. - if (shouldUpgradeToUpstreamEntry(m)) { - const upstream = upstreamNativeEntry(slug)!; - const finished = finishUpstreamNativeEntry(upstream, 9, openaiContextCap); - finished.priority = nativePriority(slug, upstream.priority); - return finished; - } - const preserved = normalizeServiceTiers({ ...m, priority: nativePriority(slug, m[SPAWN_PRIORITY_FIELD] ?? m.priority) }); - // Recompute spawn rank from current featured models, not a prior picker override. - delete preserved[SPAWN_PRIORITY_FIELD]; - // Older natives kept from disk still need the mock top tiers (max + ultra always - // for subagent max spawns; wire-clamped to the model's real top rung). - if (!isGpt56NativeSlug(slug) && slug !== NATIVE_RESERVE_MODEL) ensureUltraReasoningLevel(preserved); - return preserved; - }) - : []; - const native = nativeSourceEntries.filter(entry => - typeof entry.slug !== "string" - || (!freshBareComboAliases.has(entry.slug) && !suppressedBareNativeSlugs.has(entry.slug)) - ); - - // Backfill any native OpenAI slug that the on-disk catalog is missing (e.g. gpt-5.5), so a - // routed provider exposing the same id can never delete the native OpenAI/Codex base row. - // Skip when no enabled canonical openai provider exists (#636) — bare gpt-* would 404. - const nativeSlugs = new Set(native.flatMap(m => typeof m.slug === "string" ? [m.slug] : [])); - if (includeNativeOpenAi) { - for (const slug of policy.nativeBackfillSlugs) { - if (nativeSlugs.has(slug) || freshBareComboAliases.has(slug) || suppressedBareNativeSlugs.has(slug)) continue; - nativeSlugs.add(slug); - const entry = deriveEntry( - template ? JSON.parse(JSON.stringify(template)) : null, - slug, - "OpenAI native model (Codex OAuth passthrough).", - nativePriority(slug, upstreamNativeEntry(slug)?.priority), - undefined, - new Set(), - openaiContextCap, - ); - entry.priority = nativePriority(slug, upstreamNativeEntry(slug)?.priority); - native.push(entry); - } - } - - const nativeSourceBySlug = new Map([...nativeSourceEntries, ...native].flatMap(entry => - typeof entry.slug === "string" ? [[entry.slug, entry] as const] : [] - )); - const alignedAccountBoundEntries = detachedAccountBoundEntries.map(entry => { - // The explicit Reserve source is already chosen (actual row or documented Luna adaptation). - // A generic native merge must not replace its provenance or capability ladder. - if (isReserveCatalogProjection(entry)) return entry; - const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); - const source = nativeSlug === undefined ? undefined : nativeSourceBySlug.get(nativeSlug); - if (!source) return entry; - const aligned = JSON.parse(JSON.stringify(source)) as RawEntry; - aligned.slug = entry.slug; - aligned.display_name = entry.display_name; - aligned.priority = entry.priority; - aligned.visibility = "list"; - aligned.opencodex_catalog_kind = CODEX_ACCOUNT_BOUND_CATALOG_KIND; - return aligned; - }); - - const freshSlugs = new Set( - admittedRoutedEntries.flatMap(entry => typeof entry.slug === "string" ? [entry.slug] : []), - ); - const existingRoutedEntries = catalogModelsForMerge.filter(m => - typeof m.slug === "string" - && (m.slug.includes("/") || isNativeAliasCatalogEntry(m)) - && trustedAccountBoundNativeCatalogSlug(m) === undefined - ); - const preservedRoutedEntries = existingRoutedEntries.filter(entry => { - const slug = entry.slug as string; - if (freshEquivalent(slug)) return false; - if (isNativeAliasCatalogEntry(entry)) return exactComboSlugs.has(slug); - // Current custom rows are always regenerated from config, even while provider discovery is - // degraded. A marked row absent from the fresh projection is therefore an intentional delete. - if (entry.opencodex_catalog_kind === CODEX_CUSTOM_MODEL_CATALOG_KIND) return false; - // Before custom rows had a marker, a config deletion could otherwise be mistaken for a - // provider outage. Only explicit save-boundary evidence may classify an unmarked OpenCodex - // row; foreign and future-marked rows fail closed and remain preserved. - if (entry.opencodex_catalog_kind === undefined - && entry.owned_by !== COMBO_NAMESPACE - && isOcxAuthoredRoutedEntry(entry) - && legacyCustomModelKeys.has(slugEquivalenceKey(slug))) return false; - const provider = slug.slice(0, slug.indexOf("/")); - if (gatheredProviderNames.has(provider)) { - // A provider-local degraded observation preserves only that namespace. Authoritative empty - // catalogs and successful removals still delete stale rows even when another provider fails. - return degradedProviderNames.has(provider); - } - // Deleted/disabled providers cannot retain OpenCodex-authored ghosts. Foreign catalog rows - // remain outside provider ownership and survive unless a fresh row replaces their exact slug. - return !isOcxAuthoredRoutedEntry(entry); - }); - // Retained rows bypass the builder. Recompute managed spawn ranks from current config - // before either display-order mode; a saved display override is not current roster authority. - const pickerOrder = normalizeModelPickerOrder(modelPickerOrder); - const fullPickerOrder = pickerOrder.some(slug => !slug.includes("/")); - const rankOf = modelPickerRank(pickerOrder); - const featuredRankOf = modelPickerRank(featured); - const priorityStride = Math.max(accountSelectors.length, 1); - for (const entry of preservedRoutedEntries) { - const natural = entry[SPAWN_PRIORITY_FIELD]; - if (typeof natural === "number") { - entry.priority = natural; - delete entry[SPAWN_PRIORITY_FIELD]; - } - const slug = String(entry.slug); - if (!isOcxAuthoredRoutedEntry(entry) || isNativeAliasCatalogEntry(entry)) continue; - const featuredRank = featuredRankOf(slug); - entry.priority = featuredRank !== undefined - ? featuredRank * priorityStride - : (accountSelectors.length > 0 ? 1_000 : 0) + 5; - if (featuredRank !== undefined || fullPickerOrder) continue; - const pickerIndex = rankOf(slug); - if (pickerIndex !== undefined) { - entry[SPAWN_PRIORITY_FIELD] = entry.priority; - entry.priority = PICKER_ORDER_PRIORITY_BASE + pickerIndex * priorityStride; - } - } - let finalRoutedEntries = [...admittedRoutedEntries, ...preservedRoutedEntries]; - finalRoutedEntries = finalRoutedEntries.filter(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - if (!slug.includes("/")) return true; - if (disabledModelKeys.has(slugEquivalenceKey(slug))) return false; - // Provider allowlists own provider rows, not a current combo's public alias. Exempt only an - // identity from this gather's generated combo projection: provider discovery may supply a - // spoofed `owned_by`, and persisted combo-shaped rows are not fresh authority. - if (freshExactComboEntries.has(entry)) return true; - const slash = slug.indexOf("/"); - const provider = slug.slice(0, slash); - if (pendingProviderNames.has(provider)) return false; - const selected = selectedModelKeysByProvider.get(provider); - return selected === undefined || selected.has(slugEquivalenceKey(slug)); - }); - if (!hasPhysicalComboProvider) { - finalRoutedEntries = finalRoutedEntries.filter(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - const comboOwned = slug.startsWith(`${COMBO_NAMESPACE}/`) || entry.owned_by === COMBO_NAMESPACE; - const retainedNativeAlias = isNativeAliasCatalogEntry(entry) && exactComboSlugs.has(slug); - return !comboOwned || freshSlugs.has(slug) || retainedNativeAlias; - }); - } - finalRoutedEntries = finalRoutedEntries.filter(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - const retainedNativeAlias = isNativeAliasCatalogEntry(entry) && exactComboSlugs.has(slug); - return retainedNativeAlias - || !isExactComboCatalogEntry(entry, exactComboSlugs) - || (Array.isArray(entry.input_modalities) && entry.input_modalities.length > 0); - }); - // Reapply final catalog policy to rows preserved from disk. Those rows bypass - // gatherRoutedModels, so filtering only the freshly gathered list can resurrect an excluded id. - finalRoutedEntries = finalRoutedEntries.filter(entry => - typeof entry.slug !== "string" || !isRoutedModelCompatibilityExcluded(entry.slug) - ); - const accountBoundSlugs = new Set(alignedAccountBoundEntries.flatMap(entry => - typeof entry.slug === "string" ? [entry.slug] : [] - )); - finalRoutedEntries = finalRoutedEntries.filter(entry => { - if (typeof entry.slug !== "string" || !accountBoundSlugs.has(entry.slug)) return true; - if (freshSlugs.has(entry.slug) && policy.warningPolicy === "emit") { - warnAccountSelectorShadowedProviderOnce(entry.slug); - } - return false; - }); - const finalRoutedEntrySet = new Set(finalRoutedEntries); - const degradedPreservedCount = preservedRoutedEntries.filter(entry => { - if (!finalRoutedEntrySet.has(entry)) return false; - const slug = entry.slug as string; - const provider = slug.slice(0, slug.indexOf("/")); - return gatheredProviderNames.has(provider) && degradedProviderNames.has(provider); - }).length; - if (degradedPreservedCount > 0 && policy.warningPolicy === "emit") { - console.warn(`[opencodex] catalog sync: provider discovery degraded; preserving ${degradedPreservedCount} existing routed entr${degradedPreservedCount === 1 ? "y" : "ies"} on disk.`); - } - - const managedEntries = [...finalRoutedEntries, ...alignedAccountBoundEntries]; - const observedNativeSlugs = new Set(alignedAccountBoundEntries.flatMap(entry => { - const slug = trustedAccountBoundNativeCatalogSlug(entry); - return slug === undefined ? [] : [slug]; - })); - for (const slug of policy.nativeBackfillSlugs) observedNativeSlugs.add(slug); - const mergedEntries = [...native, ...managedEntries].map(m => { - const reserveProjection = isReserveCatalogProjection(m); - const normalized = reserveProjection ? m : normalizeServiceTiers(m); - if (!reserveProjection && !isNativeAliasCatalogEntry(normalized)) applyNativeOpenAiContextOverride(normalized, openaiContextCap); - const exactCombo = isExactComboCatalogEntry(m, exactComboSlugs); - const e = reserveProjection ? normalized : ensureStrictCatalogFields(normalized, { - preserveExactInputModalities: exactCombo, - isRouted: finalRoutedEntrySet.has(m), - }); - // Mock-max universality (260709): preserved routed entries from disk may predate - // the max rung — ensure it here so subagent max spawns validate on every - // reasoning-capable entry. max only: 5.6 exact ladders (luna: no ultra) stay intact. - if (!freshCustomEntries.has(m) && !exactCombo && !reserveProjection && !String(e.slug ?? "").startsWith("opencode-go/")) { - const levels = Array.isArray(e.supported_reasoning_levels) - ? e.supported_reasoning_levels as Array<{ effort?: string }> - : []; - if (levels.length > 0 && !levels.some(level => level.effort === "max")) { - levels.push(CODEX_REASONING_LEVELS.find(level => level.effort === "max") - ?? { effort: "max", description: "Maximum reasoning depth for the hardest problems" }); - e.supported_reasoning_levels = levels; - } - } - if (wsEnabled) e.supports_websockets = true; - else { - delete e.supports_websockets; - // Match buildCatalogEntries: never advertise a websocket preference while WS is off. - delete e.prefer_websockets; - } - return e; - }); - // Native enable/disable runs as the LAST pass so the upstream-upgrade branch above can never - // clobber a hide flag back to list. Bare ids disable every account clone; qualified ids disable - // only their generated account row. - const versionedEntries = applyMultiAgentMode( - applyNativeVisibility(mergedEntries, disabledModels, alignedAccountBoundEntries.length > 0, observedNativeSlugs), - multiAgentMode, - multiAgentV2Enabled, - { keepNativeChatGptOnV1, preserveDefaultMultiAgentVersion: isReserveCatalogProjection }, - ); - applyFullModelPickerOrder(versionedEntries, modelPickerOrder); - for (const entry of versionedEntries) { - // Templates and account clones must not inherit the native row's overlay marker. - delete entry.opencodex_native_display_name; - const slug = recoverableNativeSlug(entry); - if (slug !== null) { - const label = nativeDisplayNames && Object.hasOwn(nativeDisplayNames, slug) - ? nativeDisplayNames[slug]?.trim() : undefined; - if (label && label !== entry.display_name) { - entry.opencodex_native_display_name = { slug, original: entry.display_name, applied: label }; - entry.display_name = label; - } - } - const kind = entry.opencodex_catalog_kind; - if (trustedAccountBoundNativeCatalogSlug(entry) === undefined - && kind !== CODEX_CUSTOM_MODEL_CATALOG_KIND - && kind !== CODEX_PROVIDER_MODEL_CATALOG_KIND) continue; - // Canonicalize extension-field order after every normalizer. This keeps an unchanged catalog - // byte-idempotent whether an owned row was freshly built or retained from the prior pass. - delete entry.opencodex_catalog_kind; - entry.opencodex_catalog_kind = kind; - } - return versionedEntries; -} - -/** Merge retained-sync rows using the process-observed Codex feature state. */ -export function mergeCatalogEntriesForSync( - catalogModels: RawEntry[], - routedEntries: RawEntry[], - baseline: Map, - featured: string[], - wsEnabled: boolean, - _goIds: Set = new Set(), - template: RawEntry | null = null, - disabledModels: ReadonlySet = new Set(), - gatheredProviderNames?: Set, - multiAgentMode: MultiAgentMode = "default", - exactComboSlugs: ReadonlySet = new Set(), - hasPhysicalComboProvider = false, - includeNativeOpenAi = true, - accountBoundEntries: readonly RawEntry[] = [], - legacyCustomModelSlugs: ReadonlySet = new Set(), - suppressedBareNativeSlugs: ReadonlySet = new Set( - routedEntries.flatMap(entry => ( - isNativeAliasCatalogEntry(entry) && typeof entry.slug === "string" ? [entry.slug] : [] - )), - ), - openaiContextCap?: NativeContextLimitsInput, - keepNativeChatGptOnV1 = false, -): RawEntry[] { - // Retained for source compatibility with the original helper contract. Raw provider ids must - // not suppress same-named native rows; actual admitted combo entries own that decision now. - void _goIds; - const effectiveGatheredProviderNames = gatheredProviderNames ?? new Set( - routedEntries.flatMap(entry => { - // A slashed combo alias is not evidence that its public prefix is an authoritative provider - // namespace. Treating it as one would let the combo replace an unrestorable foreign row. - if (isExactComboCatalogEntry(entry, exactComboSlugs)) return []; - const slug = typeof entry.slug === "string" ? entry.slug : ""; - const slash = slug.indexOf("/"); - return slash > 0 ? [slug.slice(0, slash)] : []; - }), - ); - return mergeCatalogEntriesFromObservedState({ - catalogModels, - baselineCatalogModels: [], - routedEntries, - baseline, - featured, - wsEnabled, - template, - disabledModels, - selectedModelsByProvider: new Map(), - gatheredProviderNames: effectiveGatheredProviderNames, - degradedProviderNames: new Set(), - legacyCustomModelSlugs, - multiAgentMode, - multiAgentV2Enabled: isMultiAgentV2Enabled(), - keepNativeChatGptOnV1, - exactComboSlugs, - hasPhysicalComboProvider, - includeNativeOpenAi, - accountBoundEntries, - suppressedBareNativeSlugs, - openaiContextCap, - policy: { - ...CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, - warningPolicy: "emit", - }, - }); -} - -interface RetainedCatalogSyncRead { - readonly catalogPath: string; - readonly catalog: RawCatalog; - readonly onDiskCatalog: RawCatalog | null; - readonly modelsCache: RawCatalog | null; - readonly evidence: string; - /** - * Process-local epochs, baselined AFTER our own gather rather than with the - * filesystem bytes above. See `retainedCatalogProcessEvidence`. - */ - readonly processEvidence: string; -} - -interface RetainedCatalogSyncResult { - added: number; - path: string; - catalogWritten: boolean; - comboOmissions: ComboCatalogOmission[]; - /** Validated catalog commit (including identical bytes), or a refused refresh. */ - refreshOutcome?: "committed" | "refused"; - /** `desired_disabled` observed under K after the provider await; nothing was written. */ - skippedReason?: "desired_disabled"; -} - -/** - * Catalog/cache commit overrides. - * - * An explicit `ocx sync` is also the refresh path for side profiles that consume - * the OpenCodex catalog without injection (for example a custom `model_provider` - * that routes to the proxy). In that mode the Codex integration toggle only - * governs config/history injection; the catalog and models cache may still be - * refreshed, so `allowWhenDesiredDisabled` lets the commit path ignore the OFF - * gate that otherwise protects a fully native home. - */ -export interface CodexCatalogSyncOptions { - allowWhenDesiredDisabled?: boolean; -} - -interface RetainedCatalogSyncWrite { - readonly config: OcxConfig; - readonly goModels: CatalogModel[]; - readonly providerModelOutcomes: readonly CatalogGatherProviderModelOutcome[]; - readonly comboOmissions: ComboCatalogOmission[]; - readonly read: RetainedCatalogSyncRead; - readonly permit: CatalogWritePermit; - readonly owningCodexHome: string; - readonly modelEntitlements: CodexModelEntitlementSnapshot; -} - -function optionalFileBytes(path: string): string | null { - try { - return readFileSync(path).toString("base64"); - } catch (error) { - if ((error as NodeJS.ErrnoException | undefined)?.code === "ENOENT") return null; - throw error; - } -} - -function loadCatalogForRetainedSync(path: string): RawCatalog | null { - const bundled = isDefaultCatalogPath(path) ? loadBundledCodexCatalog() : null; - if (bundled) return JSON.parse(JSON.stringify(bundled)) as RawCatalog; - const active = readCatalog(path); - // A valid configured custom file remains the content authority even when it has no bare native - // template. The null-template builder is deliberate; a stale backup must not replace active - // custom root metadata merely because the current file contains only routed rows. - if (active && (!isDefaultCatalogPath(path) || findNativeTemplate(active))) return active; - return readCatalog(catalogBackupPathFor(path)) - ?? (isDefaultCatalogPath(path) ? readCatalog(legacyCatalogBackupPath()) : null) - ?? readCatalog(activeCodexModelsCachePath()) - ?? active; -} - -function retainedCatalogSyncEvidence( - config: OcxConfig, - catalogPath: string, - catalog: RawCatalog, -): string { - return JSON.stringify({ - config, - catalogPath, - catalog, - catalogBytes: optionalFileBytes(catalogPath), - hashedBackupBytes: optionalFileBytes(catalogBackupPathFor(catalogPath)), - legacyBackupBytes: isDefaultCatalogPath(catalogPath) - ? optionalFileBytes(legacyCatalogBackupPath()) : null, - modelsCacheBytes: optionalFileBytes(activeCodexModelsCachePath()), - // The persisted runtime selection is a pre-await filesystem input, not a - // process epoch: another PROCESS can move runtime authority by rewriting this - // file, and that move is invisible to our in-process memo. Recorded PRESENT or - // ABSENT, because its absence is what makes the resolver fall back. - runtimeStateBytes: optionalFileBytes(codexRuntimeStatePath()), - }); -} - -/** - * The bundled-template half of the same evidence, observed separately. - * - * The runtime process memo is deliberately NOT here, and that exclusion took three - * attempts to get honest. Gathering resolves the Codex runtime lazily and under its - * own cache key, so this path cannot pre-settle that memo: baselining it before the - * await always detected our own side effect and refused every write, and baselining - * it after the await captured a runtime that ANOTHER process had moved as though it - * were ours — a catalog prepared from R1 committing after authority reached R2. - * - * Runtime authority is covered where it is actually durable instead: the persisted - * `codex-runtime.json` bytes sit in the pre-await filesystem evidence, PRESENT or - * ABSENT, so a cross-process runtime move is caught. What is left uncovered, and is - * written down rather than papered over, is a same-process in-memory runtime swap - * that never touches that file — WP11 owns the lock that makes that case decidable. - */ -function retainedCatalogProcessEvidence(): string { - return JSON.stringify({ - bundledCatalogCache: bundledCatalogCacheState(), - }); -} - -/** - * Capture every local catalog input the retained sync path consults before its - * provider await. The exact evidence is compared after K acquisition; a newer - * catalog/backup/cache or target selection makes this attempt a no-write. - */ -function readRetainedCatalogSync(config: OcxConfig): RetainedCatalogSyncRead | null { - const catalogPath = readCodexCatalogPath(); - const catalog = loadCatalogForRetainedSync(catalogPath); - if (!catalog) return null; - - // The bundled catalog is a reliable native template on the default path, but it is not the - // merge source. Preservation must inspect the file that this sync is about to overwrite; - // otherwise an empty/partial provider gather cannot see routed or user-native rows on disk. - const onDiskCatalog = readCatalog(catalogPath); - const modelsCache = readCatalog(activeCodexModelsCachePath()); - const evidence = retainedCatalogSyncEvidence(config, catalogPath, catalog); - // `processEvidence` is filled in after the provider await, not here. - return { catalogPath, catalog, onDiskCatalog, modelsCache, evidence, processEvidence: "" }; -} - -function revalidateRetainedCatalogSync( - config: OcxConfig, - prepared: RetainedCatalogSyncRead, -): RetainedCatalogSyncRead | null { - const catalogPath = readCodexCatalogPath(); - if (catalogPath !== prepared.catalogPath) return null; - const evidence = retainedCatalogSyncEvidence(config, catalogPath, prepared.catalog); - if (evidence !== prepared.evidence) return null; - if (retainedCatalogProcessEvidence() !== prepared.processEvidence) return null; - return { - catalogPath, - catalog: JSON.parse(JSON.stringify(prepared.catalog)) as RawCatalog, - onDiskCatalog: readCatalog(catalogPath), - modelsCache: readCatalog(activeCodexModelsCachePath()), - evidence, - processEvidence: prepared.processEvidence, - }; -} - -/** - * Exact bytes currently on disk at `path`, or null when unreadable/absent. - * - * Deliberately a Buffer rather than a decoded string: `readFileSync(path, "utf8")` - * substitutes U+FFFD for every invalid byte, so a file holding a raw 0x80 decodes - * equal to prepared content holding a legitimately encoded U+FFFD. Comparing the - * decoded strings would then classify a malformed catalog as identical, skip the - * atomic repair write, and leave the corruption on disk while reporting - * `catalogWritten: false`. - */ -function currentCatalogFileContent(path: string): Buffer | null { - try { - return readFileSync(path); - } catch { - return null; - } -} - -function pristineCatalogBytes(read: RetainedCatalogSyncRead): string | null { - if (read.onDiskCatalog && !catalogHasRoutedEntries(read.onDiskCatalog)) { - try { - return readFileSync(read.catalogPath, "utf8"); - } catch { - return null; - } - } - return catalogHasRoutedEntries(read.catalog) - ? null - : `${JSON.stringify(read.catalog, null, 2)}\n`; -} - -function catalogModelsForMergeWithNativeRecovery( - catalogPath: string, - catalog: RawCatalog, - onDiskCatalog: RawCatalog | null, -): RawEntry[] { - const primaryCatalogModels = onDiskCatalog?.models ?? catalog.models ?? []; - // Native-alias compatibility can omit disabled native rows from the effective catalog because - // Desktop's remote allowlist ignores `visibility: "hide"`. Keep current/pristine native recovery - // sources beside the on-disk rows so re-enabling a model restores its real metadata. Routed and - // user-authored rows still come only from the on-disk catalog. - return mergeCatalogModelsWithNativeRecovery(primaryCatalogModels, [ - catalog.models ?? [], - readCatalogBackup(catalogPath)?.models ?? [], - ]); -} - -const AUTO_REVIEW_ROOT_MARKER = "opencodex_auto_review_root"; - -interface RootAutoReviewStamp { - slug: string; - original: string | null; - applied: string; -} - -function rootAutoReviewStamp(entry: RawEntry): RootAutoReviewStamp | undefined { - const value = entry[AUTO_REVIEW_ROOT_MARKER]; - if (!value || typeof value !== "object" || Array.isArray(value)) return undefined; - const stamp = value as Record; - if (stamp.slug !== entry.slug || typeof stamp.slug !== "string" - || typeof stamp.applied !== "string" - || (stamp.original !== null && typeof stamp.original !== "string")) return undefined; - return stamp as unknown as RootAutoReviewStamp; -} - - -/** True when the value is a valid Codex catalog auto-review selector. */ -export function isValidAutoReviewModel(value: unknown): value is string { - return isValidAutoReviewTarget(value); -} - -export type AutoReviewModelOverrideResult = "absent" | "applied" | "invalid" | "unresolved"; - -/** True when a catalog row was synthesized by opencodex instead of coming from upstream. */ -function isRoutedCatalogEntry(entry: RawEntry): boolean { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - return slug.includes("/") - || (typeof entry.description === "string" && entry.description.startsWith("Routed via opencodex → ")); -} - -/** Restore an owned native value, retaining provenance to avoid legacy reclassification. */ -function clearAutoReviewOverrideValue(entry: RawEntry): void { - const stamp = rootAutoReviewStamp(entry); - if (stamp) { - if (entry.auto_review_model_override === stamp.applied) entry.auto_review_model_override = stamp.original; - } else { - entry.auto_review_model_override = null; - delete entry[AUTO_REVIEW_ROOT_MARKER]; - } -} - -/** - * Legacy whole-catalog root stamp: releases before AUTO_REVIEW_ROOT_MARKER wrote root stamps that - * are textually identical to an upstream value, so the only way to recognize one is the uniform - * signature the no-provider path relies on — a single value that a routed row also carries. - * Returns the stamped values when the observed rows match that shape. - */ -function legacyRootStampValues(observedModels: readonly RawEntry[]): ReadonlySet | undefined { - if (observedModels.some(entry => entry?.[AUTO_REVIEW_ROOT_MARKER] !== undefined)) return undefined; - const configuredValues = new Set(observedModels.flatMap(entry => { - const value = entry?.auto_review_model_override; - return typeof value === "string" && value.trim() ? [value] : []; - })); - const globalStamp = configuredValues.size === 1 - && observedModels.some(entry => { - const value = entry.auto_review_model_override; - return isRoutedCatalogEntry(entry) - && typeof value === "string" - && value.trim().length > 0 - && configuredValues.has(value); - }) - && observedModels.every(entry => { - const value = entry?.auto_review_model_override; - return value === null - || value === undefined - || (typeof value === "string" && configuredValues.has(value)); - }); - return globalStamp ? configuredValues : undefined; -} - -/** - * Sweep legacy root stamps off the rows a root removal owns, before provider plans land. - * - * Root removal reaches marker-tagged native rows on its own, but a catalog written before the - * marker only carries the legacy signature — and provider stamping rewrites that signature before - * the root pass could read it, so the sweep has to run first. - */ -function clearLegacyRootStamps(models: readonly RawEntry[], sourceModels: readonly RawEntry[] = []): void { - const legacyStamp = legacyRootStampValues([...models, ...sourceModels]); - if (legacyStamp === undefined) return; - for (const entry of models) { - if (!entry || typeof entry !== "object") continue; - const current = entry.auto_review_model_override; - if (entry[AUTO_REVIEW_ROOT_MARKER] === undefined - && typeof current === "string" && legacyStamp.has(current)) clearAutoReviewOverrideValue(entry); - } -} - -/** - * Clear the root selector from every row this path owns: routed rows, rows stamped by a release - * that writes the provenance marker, and the legacy whole-catalog stamp that predates it. - */ -function clearAutoReviewModelOverride( - models: readonly RawEntry[], - sourceModels: readonly RawEntry[] = [], -): void { - const legacyStamp = legacyRootStampValues([...models, ...sourceModels]); - for (const entry of models) { - if (!entry || typeof entry !== "object") continue; - const current = entry.auto_review_model_override; - if (isRoutedCatalogEntry(entry) - || (entry[AUTO_REVIEW_ROOT_MARKER] === true || rootAutoReviewStamp(entry) !== undefined) - || (legacyStamp !== undefined && typeof current === "string" && legacyStamp.has(current))) { - clearAutoReviewOverrideValue(entry); - } - } -} - -/** Warn once about a malformed or unresolvable root auto-review selector. */ -function warnAutoReviewModelDiagnostic( - reason: "invalid" | "unresolved", - configured: string, -): void { - const safeConfigured = JSON.stringify(redactSecretString(configured)); - const detail = reason === "unresolved" - ? "the selector was not found in the final catalog" - : "the selector format is invalid"; - console.warn( - `[opencodex] auto_review_model ${detail} (${safeConfigured}); preserving normal upstream auto-review behavior.`, - ); -} - -/** Warn once about a malformed or unresolvable provider-scoped auto-review selector. */ -function warnProviderAutoReviewModelDiagnostic( - reason: "invalid" | "unresolved", - provider: string, - configured: string, -): void { - const safeProvider = JSON.stringify(redactSecretString(provider)); - const safeConfigured = JSON.stringify(redactSecretString(configured)); - const detail = reason === "unresolved" - ? "the selector was not found in the final catalog" - : "the selector format is invalid"; - console.warn( - `[opencodex] auto_review_model for provider ${safeProvider} ${detail} (${safeConfigured}); using the next valid provider/root selector or upstream behavior.`, - ); -} - -/** - * Note once when a bare selector resolves to a row outside the provider it was configured on. - * - * That is how a native model is named as a reviewer, so it stays usable, but a mistyped target must - * not be silent: the operator sees which catalog row actually supplies the reviewer. - */ -function warnProviderAutoReviewForeignTarget(provider: string, configured: string, target: string): void { - const safeProvider = JSON.stringify(redactSecretString(provider)); - const safeConfigured = JSON.stringify(redactSecretString(configured)); - const safeTarget = JSON.stringify(redactSecretString(target)); - console.warn( - `[opencodex] auto_review_model for provider ${safeProvider} (${safeConfigured}) resolved to ${safeTarget}, which is not a row of that provider; that catalog row supplies the reviewer.`, - ); -} - -/** Preserve native upstream overrides and the root-derived provenance marker from source rows. */ -function preserveNativeAutoReviewModelOverrides( - models: readonly RawEntry[], - sourceModels: readonly RawEntry[], -): void { - const existing = new Map(); - for (const entry of sourceModels) { - const slug = typeof entry.slug === "string" ? entry.slug : undefined; - const value = entry.auto_review_model_override; - if (!slug || isRoutedCatalogEntry(entry)) continue; - if (typeof value === "string" || value === null) { - existing.set(slug, { value, root: rootAutoReviewStamp(entry) ?? (entry[AUTO_REVIEW_ROOT_MARKER] === true ? true : undefined) }); - } - } - for (const entry of models) { - const slug = typeof entry.slug === "string" ? entry.slug : undefined; - if (!slug || isRoutedCatalogEntry(entry) || !existing.has(slug)) continue; - const saved = existing.get(slug)!; - entry.auto_review_model_override = saved.value; - if (saved.root) entry[AUTO_REVIEW_ROOT_MARKER] = structuredClone(saved.root); - else delete entry[AUTO_REVIEW_ROOT_MARKER]; - } -} - -/** Stamp a root-derived override and mark native rows so later root removal is durable. */ -function stampRootAutoReviewOverride(entry: RawEntry, target: string): void { - if (!isRoutedCatalogEntry(entry)) { - const previous = rootAutoReviewStamp(entry); - const current = entry.auto_review_model_override; - entry[AUTO_REVIEW_ROOT_MARKER] = { - slug: typeof entry.slug === "string" ? entry.slug : "", - original: previous && current === previous.applied - ? previous.original : typeof current === "string" ? current : null, - applied: target, - } satisfies RootAutoReviewStamp; - } else { - delete entry[AUTO_REVIEW_ROOT_MARKER]; - } - entry.auto_review_model_override = target; -} - -/** Stamp a provider-derived override; provider stamps never fall under root removal. */ -function stampProviderAutoReviewOverride(entry: RawEntry, target: string): void { - entry.auto_review_model_override = target; - delete entry[AUTO_REVIEW_ROOT_MARKER]; -} - -/** - * Apply the root Codex auto-review selector to every catalog row, or clear it when the value is - * absent, blank, malformed, or does not resolve against the assembled catalog. - */ -export function applyAutoReviewModelOverride( - models: RawEntry[] | undefined, - autoReviewModel: string | null | undefined, - sourceModels: readonly RawEntry[] = [], -): AutoReviewModelOverrideResult { - if (!models || !Array.isArray(models)) return "absent"; - if (autoReviewModel === null || autoReviewModel === undefined) { - clearAutoReviewModelOverride(models, sourceModels); - return "absent"; - } - const trimmed = autoReviewModel.trim(); - if (!trimmed) { - clearAutoReviewModelOverride(models, sourceModels); - return "absent"; - } - if (!isValidAutoReviewModel(trimmed)) { - clearAutoReviewModelOverride(models, sourceModels); - warnAutoReviewModelDiagnostic("invalid", trimmed); - return "invalid"; - } - if (!configuredCatalogEntry(models, trimmed)) { - clearAutoReviewModelOverride(models, sourceModels); - warnAutoReviewModelDiagnostic("unresolved", trimmed); - return "unresolved"; - } - for (const entry of models) { - if (entry && typeof entry === "object") { - stampRootAutoReviewOverride(entry, trimmed); - } - } - return "applied"; -} - -/** Validated provider-scoped target with both the configured spelling and catalog slug. */ -interface ValidProviderReviewTarget { - configured: string; - target: string; -} - -/** One provider's resolved provider-wide and per-model auto-review targets. */ -interface ProviderReviewPlan { - wide?: ValidProviderReviewTarget; - perModel: Map; -} - -/** Public provider namespace of a routed catalog row, when it has one. */ -function catalogEntryProviderName(entry: RawEntry): string | undefined { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - const slash = slug.indexOf("/"); - return slash > 0 && isRoutedCatalogEntry(entry) ? slug.slice(0, slash) : undefined; -} - -/** Encoded model-id segment of a routed catalog row, when it has one. */ -function catalogEntryModelSegment(entry: RawEntry): string | undefined { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - const slash = slug.indexOf("/"); - return slash > 0 ? slug.slice(slash + 1) : undefined; -} - -/** Case-preserving encoded key used to match per-model override maps. */ -function providerModelKey(modelId: string): string { - return canonicalAutoReviewModelKey(modelId); -} - -/** - * True when another routed row of this provider already carries `alias` as its own model id. - * - * The alias API validates against whatever ids discovery has reported so far, so on a cold start an - * alias can be persisted that later turns out to name a different row. A key using it is then not - * an alternate spelling of the aliased model — it is that row's id — and must not be propagated. - */ -function aliasNamesAnotherRoutedRow(models: readonly RawEntry[], provider: string, alias: string): boolean { - const encoded = encodeRoutedModelId(alias); - return models.some(entry => isRoutedCatalogEntry(entry) - && catalogEntryProviderName(entry) === provider - && catalogEntryModelSegment(entry) === encoded); -} - -/** Resolve one configured target against the assembled catalog; bare values name a model of the same provider. */ -function resolveProviderReviewTarget( - models: readonly RawEntry[], - provider: string, - configuredRaw: unknown, -): { kind: "valid"; value: ValidProviderReviewTarget; foreign?: boolean } | { kind: "invalid"; configured: string } | { kind: "unresolved"; configured: string } | { kind: "absent" } { - if (typeof configuredRaw !== "string") return { kind: "absent" }; - const configured = configuredRaw.trim(); - if (!configured) return { kind: "absent" }; - if (!isValidAutoReviewModel(configured)) return { kind: "invalid", configured }; - const prefix = `${provider}/`; - let match: RawEntry | undefined; - const sameProviderCandidate = (rawModelId: string): RawEntry | undefined => models.find(entry => { - if (!isRoutedCatalogEntry(entry) || typeof entry.slug !== "string" || !entry.slug.startsWith(prefix)) return false; - const segment = catalogEntryModelSegment(entry); - return segment !== undefined && segment === encodeRoutedModelId(rawModelId); - }); - // A bare selector names a model of this provider. A full selector that resolves in the - // assembled catalog already names the exact row, including a same-provider encoded slug. - if (!configured.includes("/")) { - match = sameProviderCandidate(configured); - } - match ??= configuredCatalogEntry(models, configured); - if (!match && configured.startsWith(prefix)) { - match = sameProviderCandidate(configured.slice(prefix.length)); - } - if (!match) { - // A raw model id may itself contain "/" (for example zenmux moonshotai/kimi-k3). - // After the full-selector lookup misses, try that spelling as a same-provider id. - match = sameProviderCandidate(configured); - } - if (!match) return { kind: "unresolved", configured }; - const target = typeof match.slug === "string" ? match.slug : configured; - // A qualified selector may name another provider's row on purpose; only a bare value that lands - // outside this provider is worth reporting. - const foreign = !configured.includes("/") && catalogEntryProviderName(match) !== provider; - return { kind: "valid", value: { configured, target }, ...(foreign ? { foreign: true } : {}) }; -} - -/** Build resolved per-provider plans and emit one diagnostic per bad selector. */ -function buildProviderReviewPlans( - models: readonly RawEntry[], - config: Pick, -): { plans: Map; failure?: "invalid" | "unresolved" } { - const plans = new Map(); - let failure: "invalid" | "unresolved" | undefined; - const warned = new Set(); - const recordFailure = (kind: "invalid" | "unresolved", provider: string, configured: string): void => { - const signature = `${provider}\u0000${configured}`; - if (warned.has(signature)) return; - warned.add(signature); - warnProviderAutoReviewModelDiagnostic(kind, provider, configured); - failure ??= kind; - }; - const recordForeignTarget = (provider: string, configured: string, target: string): void => { - const signature = `${provider}\u0000foreign\u0000${configured}`; - if (warned.has(signature)) return; - warned.add(signature); - warnProviderAutoReviewForeignTarget(provider, configured, target); - }; - for (const [name, provider] of Object.entries(config.providers ?? {})) { - if (provider.autoReviewModel === undefined && provider.autoReviewModelOverrides === undefined) continue; - const plan: ProviderReviewPlan = { perModel: new Map() }; - if (provider.autoReviewModel !== undefined) { - const resolved = resolveProviderReviewTarget(models, name, provider.autoReviewModel); - if (resolved.kind === "valid") { - plan.wide = resolved.value; - if (resolved.foreign) recordForeignTarget(name, resolved.value.configured, resolved.value.target); - } - else if (resolved.kind !== "absent") recordFailure(resolved.kind, name, resolved.configured); - } - if (provider.autoReviewModelOverrides !== undefined) { - for (const [modelId, rawTarget] of Object.entries(provider.autoReviewModelOverrides)) { - const resolved = resolveProviderReviewTarget(models, name, rawTarget); - if (resolved.kind === "valid") { - plan.perModel.set(providerModelKey(modelId), resolved.value); - if (resolved.foreign) recordForeignTarget(name, resolved.value.configured, resolved.value.target); - } else if (resolved.kind !== "absent") { - recordFailure(resolved.kind, name, resolved.configured); - } - } - } - // `modelAliases` publishes a second public name for a model id, and a routed row's slug always - // carries the upstream id — so accept an override key written in either spelling. - for (const [modelId, alias] of Object.entries(provider.modelAliases ?? {})) { - if (typeof alias !== "string" || !alias.trim()) continue; - if (aliasNamesAnotherRoutedRow(models, name, alias)) continue; - const idKey = providerModelKey(modelId); - const aliasKey = providerModelKey(alias); - if (idKey === aliasKey) continue; - const fromId = plan.perModel.get(idKey); - const fromAlias = plan.perModel.get(aliasKey); - if (fromId !== undefined && fromAlias === undefined) plan.perModel.set(aliasKey, fromId); - else if (fromAlias !== undefined && fromId === undefined) plan.perModel.set(idKey, fromAlias); - } - if (plan.wide !== undefined || plan.perModel.size > 0) plans.set(name, plan); - } - return { plans, failure }; -} - -/** Apply or clear the root selector only on rows without a provider stamp. */ -function applyRootSelectorToRemaining( - models: readonly RawEntry[], - rootValue: string | null | undefined, - providerStamped: ReadonlySet, -): AutoReviewModelOverrideResult { - const clearRemaining = (): void => { - for (const entry of models) { - if (!entry || providerStamped.has(entry)) continue; - // Native rows written by releases before the root marker cannot be told apart from upstream - // values once provider stamps diverge. clearLegacyRootStamps sweeps the ones the legacy - // uniform signature still recognizes before provider plans land, because provider stamping - // destroys that signature; a catalog that no longer matches it needs a one-off manual sync. - if (isRoutedCatalogEntry(entry) || entry[AUTO_REVIEW_ROOT_MARKER] === true || rootAutoReviewStamp(entry)) clearAutoReviewOverrideValue(entry); - } - }; - if (rootValue === null || rootValue === undefined) { - clearRemaining(); - return "absent"; - } - const trimmed = rootValue.trim(); - if (!trimmed) { - clearRemaining(); - return "absent"; - } - if (!isValidAutoReviewModel(trimmed)) { - clearRemaining(); - warnAutoReviewModelDiagnostic("invalid", trimmed); - return "invalid"; - } - if (!configuredCatalogEntry(models, trimmed)) { - clearRemaining(); - warnAutoReviewModelDiagnostic("unresolved", trimmed); - return "unresolved"; - } - for (const entry of models) { - if (!entry || providerStamped.has(entry)) continue; - stampRootAutoReviewOverride(entry, trimmed); - } - return "applied"; -} - -/** Provider-aware variant: provider rows win and the root selector is the fallback. */ -export function applyConfiguredAutoReviewModelOverride( - models: RawEntry[] | undefined, - rootAutoReviewModel: string | null | undefined, - config: Pick, - sourceModels: readonly RawEntry[] = [], -): AutoReviewModelOverrideResult { - if (!models || !Array.isArray(models)) return "absent"; - // Runs unconditionally because the sweep only fires on the uniform legacy signature. A resolved - // root selector restamps every row it touches below, so the call is behavior-preserving there; - // with the root absent, invalid, or unresolved those clears are final — which is the point, and - // also the limit: the legacy heuristic cannot tell a root stamp from an identical upstream value. - clearLegacyRootStamps(models, sourceModels); - const { plans, failure } = buildProviderReviewPlans(models, config); - const providerStamped = new Set(); - for (const entry of models) { - if (!entry || typeof entry !== "object") continue; - const provider = catalogEntryProviderName(entry); - if (!provider) continue; - const plan = plans.get(provider); - if (!plan) continue; - const modelSegment = catalogEntryModelSegment(entry); - const perModel = modelSegment === undefined ? undefined : plan.perModel.get(providerModelKey(modelSegment)); - const selected = perModel ?? plan.wide; - if (!selected) continue; - stampProviderAutoReviewOverride(entry, selected.target); - providerStamped.add(entry); - } - const rootResult = applyRootSelectorToRemaining(models, rootAutoReviewModel, providerStamped); - const providerApplied = [...providerStamped].some(entry => typeof entry.auto_review_model_override === "string"); - if (providerApplied) { - if (rootResult === "invalid" || rootResult === "unresolved") return rootResult; - return failure ?? "applied"; - } - return failure ?? rootResult; -} - -/** True when any provider row configures a provider-scoped auto-review selector. */ -function configHasProviderAutoReview(config: Pick): boolean { - return Object.values(config.providers ?? {}).some(provider => - provider.autoReviewModel !== undefined || provider.autoReviewModelOverrides !== undefined); -} - -/** Apply the root Codex auto-review selector after the final catalog merge. */ -export function finalizeAutoReviewModelOverride( - models: RawEntry[] | undefined, - sourceModels: readonly RawEntry[] = [], - config?: Pick, -): AutoReviewModelOverrideResult { - if (models && sourceModels.length > 0) preserveNativeAutoReviewModelOverrides(models, sourceModels); - if (config && configHasProviderAutoReview(config)) { - return applyConfiguredAutoReviewModelOverride(models, readConfiguredAutoReviewModel(), config, sourceModels); - } - return applyAutoReviewModelOverride(models, readConfiguredAutoReviewModel(), sourceModels); -} -/** - * Why an account-gated native model stopped being offered, but only when the answer is one the - * operator can act on. - * - * Suppression is an omission: the row is never built, so there is no catalog entry for a reason - * to ride on and no downstream consumer that could explain it later. #4212's reporter watched - * their models disappear and reasonably concluded the proxy was broken, because every surface - * that changed said nothing about the account that caused it. - * - * Returns `undefined` for the ordinary case — an account that is simply not entitled to a gated - * model. That is the default state for most installations, it is not news, and warning about it - * on every sync would bury the one case that matters. A credential the operator must repair is - * the case that matters, so that is the only one this speaks up about. - * - * Accounts are named with the durable `p`-prefixed log label, the same identifier the dashboard - * shows, never the raw pool id or the email. - */ -export function gatedNativeReauthSuppressionReason(args: { - snapshot: CodexModelEntitlementSnapshot; - slug: string; - eligibleAccountIds?: ReadonlySet; - needsReauth: (accountId: string) => boolean; - label: (accountId: string) => string; -}): string | undefined { - const observed = [...args.snapshot.modelsByAccount.keys()] - .filter(accountId => !args.eligibleAccountIds || args.eligibleAccountIds.has(accountId)) - // Only accounts that could actually have served THIS model. An account upstream positively - // denied is not why the model is missing, and blaming it would send the operator to repair a - // credential that was never going to help. `unknown` has to stay in: an account whose roster - // could not be confirmed reports `unknown` rather than `granted`, and a credential stuck on - // a failed refresh is exactly that account. - .filter(accountId => ( - codexModelEntitlementStateForAccount(args.snapshot, accountId, args.slug) !== "denied" - )); - const stuck = observed.filter(accountId => args.needsReauth(accountId)); - if (stuck.length === 0) return undefined; - const names = stuck.map(accountId => args.label(accountId)).sort().join(", "); - return stuck.length === observed.length - ? `every Codex account that could serve it needs reauthentication (${names})` - : `${stuck.length} of ${observed.length} Codex accounts that could serve it need reauthentication (${names})`; -} - -/** Durable, operator-facing label for a pool account id; never the raw id or the email. */ -function gatedNativeAccountLabel(config: OcxConfig, accountId: string): string { - // Direct mode narrows eligibility to the native main credential, so this is the account most - // likely to be named here. `codexAuthContextLogLabel` calls it "main" everywhere else; hashing - // it into a `p`-prefixed digest would name the one account the operator cannot look up. - if (accountId === MAIN_CODEX_ACCOUNT_ID) return "main"; - const account = (config.codexAccounts ?? []).find(candidate => candidate.id === accountId); - return account ? codexAccountLogLabel(account) : fallbackCodexAccountLogLabel(accountId); -} - -const warnedGatedNativeSuppression = new Set(); - -/** Test seam: the warn-once memory is process-global, so a case needs to be able to clear it. */ -export function resetGatedNativeSuppressionWarningsForTests(): void { - warnedGatedNativeSuppression.clear(); -} - -function warnGatedNativeSuppressedOnce(slug: string, reason: string): void { - const signature = `${slug}\u0000${reason}`; - if (warnedGatedNativeSuppression.has(signature)) return; - warnedGatedNativeSuppression.add(signature); - console.warn( - `[opencodex] catalog sync: ${slug} is not being offered because ${reason}. ` - + "Sign in again to restore it.", - ); -} - -/** - * Mescla o catálogo retido com os modelos visíveis e as configurações atuais, - * incluindo os nomes nativos. Tenta preservar o backup original e usa a permissão - * de escrita para publicar o resultado apenas se os bytes mudarem, retornando - * a contagem de entradas roteadas e por conta, o caminho e o estado da gravação. - */ -function writeRetainedCatalogSync({ - config, - goModels, - providerModelOutcomes, - comboOmissions, - read, - permit, - owningCodexHome, - modelEntitlements, -}: RetainedCatalogSyncWrite): RetainedCatalogSyncResult { - const { catalogPath, catalog, onDiskCatalog } = read; - const catalogModelsForMerge = catalogModelsForMergeWithNativeRecovery( - catalogPath, - catalog, - onDiskCatalog, - ); - // Strict selector for template inheritance; the validity gate above keeps the broad one. - const template = findSupportedNativeTemplate(catalog); - - try { - // Once-only: preserve the PRISTINE pre-opencodex catalog as the native-priority baseline - // (later syncs would otherwise overwrite it with featured-modified priorities). - const pristine = pristineCatalogBytes(read); - if (pristine !== null) { - publishHashedCodexCatalogBackup(permit, owningCodexHome, { - path: catalogBackupPathFor(catalogPath), - content: pristine, - }); - if (isDefaultCatalogPath(catalogPath)) { - publishLegacyCodexCatalogBackup(permit, owningCodexHome, { - path: legacyCatalogBackupPath(), - content: pristine, - }); - } - } - } catch { /* backup best-effort */ } - - // Hide disabled models from Codex, then feature the chosen subagent models (native OR routed) - // by giving them the lowest priority — see buildCatalogEntries for why priority, not array order. - const enabledGo = filterCatalogVisibleModels(goModels, config); - const featured = config.subagentModels ?? []; - const orderedGoModels = orderForSubagents(enabledGo, featured); // stable tie-break among equal priorities - const modelPickerOrder = config.modelPickerOrder ?? []; - const multiAgentMode: MultiAgentMode = config.multiAgentMode === "v1" || config.multiAgentMode === "v2" ? config.multiAgentMode : "default"; - const exactComboSlugs = exactComboCatalogSlugs(config); - const bareEligibleAccountIds = providerCodexAccountMode( - OPENAI_CODEX_PROVIDER_ID, - config.providers[OPENAI_CODEX_PROVIDER_ID], - ) === "direct" ? new Set([MAIN_CODEX_ACCOUNT_ID]) : undefined; - const availableBareGatedNativeSlugs = availableAccountGatedNativeModels( - modelEntitlements, - bareEligibleAccountIds, - ); - const availableAccountGatedNativeSlugs = availableAccountGatedNativeModels(modelEntitlements); - const availableBareNativeSlugs = NATIVE_OPENAI_MODELS.filter(slug => ( - !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) || availableBareGatedNativeSlugs.has(slug) - )); - const availableAccountNativeSlugs = NATIVE_OPENAI_MODELS.filter(slug => ( - !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) || availableAccountGatedNativeSlugs.has(slug) - )); - const unavailableGatedNativeSlugs = new Set([...ACCOUNT_GATED_NATIVE_OPENAI_MODELS].filter(slug => ( - !availableBareGatedNativeSlugs.has(slug) - ))); - // #4212: this set is the whole record of a model vanishing, and it is a set of strings that - // nothing downstream ever asks a question of. Explain it here, while the entitlement snapshot - // that produced it is still in scope, because after this point the model is simply absent and - // no later surface can tell "never entitled" apart from "the account broke this morning". - for (const slug of unavailableGatedNativeSlugs) { - const reason = gatedNativeReauthSuppressionReason({ - snapshot: modelEntitlements, - slug, - eligibleAccountIds: bareEligibleAccountIds, - needsReauth: isAccountNeedsReauth, - label: accountId => gatedNativeAccountLabel(config, accountId), - }); - if (reason) warnGatedNativeSuppressedOnce(slug, reason); - } - const suppressedBareNativeSlugs = new Set([ - ...desktopAllowlistSuppressedNativeSlugs(config), - ...unavailableGatedNativeSlugs, - ]); - const hasPhysicalComboProvider = Object.hasOwn(config.providers, COMBO_NAMESPACE); - const includeNativeOpenAi = shouldIncludeNativeOpenAi(config); - const includeAccountBoundNativeOpenAi = shouldIncludeAccountBoundNativeOpenAi(config); - // Both user levers. Passing only the cap here is what let a per-model window the dashboard - // had accepted get written back at full width in the on-disk catalog. - const openaiContextCap = nativeContextLimits(config); - const accountSelectors = includeAccountBoundNativeOpenAi - ? visibleCodexAccountSelectors(config) - : []; - const observedAccountNativeEntries = [ - ...(read.modelsCache?.models ?? []), - ...(onDiskCatalog?.models ?? []).filter(entry => - trustedAccountBoundNativeCatalogSlug(entry) !== undefined), - ]; - const accountTargets = new Map(codexAccountNamespaceEntries(config)); - const reserveMainSelectors = accountSelectors.filter(selector => - isMainCodexAccountTarget(accountTargets.get(selector) ?? "")); - // The active file can own a bare source even when the bundled catalog is the build base. - // A previously clamped qualified projection must not shorten a retained genuine ladder. - const reserveObservations = [ - ...(onDiskCatalog?.models ?? []), - ...(read.modelsCache?.models ?? []), - ...(catalog.models ?? []), - ]; - const retainedReserve = onDiskCatalog?.[RESERVE_SOURCE_CATALOG_FIELD]; - const retainedReserveSource = retainedReserve && typeof retainedReserve === "object" && !Array.isArray(retainedReserve) - ? observedReserveCatalogSource([retainedReserve as RawEntry], []) - : null; - const observedReserveSource = observedReserveCatalogSource( - // Cache invalidation carries historical bare observations alongside emitted models. - // Only unmarked observations are fresh enough to supersede the retained source. - reserveObservations.filter(entry => entry.slug === NATIVE_RESERVE_MODEL - && entry.opencodex_account_observed_native === undefined), reserveMainSelectors, - ) ?? retainedReserveSource ?? observedReserveCatalogSource(reserveObservations, reserveMainSelectors); - // This root is read only by OCX. Upstream ModelsResponse ignores unknown root fields. - // Retain before final runtime clamping: an omitted row must not turn into Luna next sync. - if (observedReserveSource) catalog[RESERVE_SOURCE_CATALOG_FIELD] = structuredClone(observedReserveSource); - else delete catalog[RESERVE_SOURCE_CATALOG_FIELD]; - const lunaSource = upstreamNativeEntry(RESERVE_LUNA_METADATA_SOURCE); - const reserve = createReserveCatalogProjection( - config, - reserveMainSelectors, - observedReserveSource, - lunaSource ? finishUpstreamNativeEntry(lunaSource, 9, openaiContextCap) : null, - ); - const accountNativeSlugsBySelector = accountSelectors.length > 0 - ? new Map([...accountBoundNativeOpenAiSlugsBySelector(config, observedAccountNativeEntries)].map(([selector, slugs]) => { - const target = accountTargets.get(selector); - const accountId = target && isMainCodexAccountTarget(target) ? MAIN_CODEX_ACCOUNT_ID : target; - return [selector, slugs.filter(slug => ( - !ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(slug) - || (accountId !== undefined - && codexModelEntitlementStateForAccount(modelEntitlements, accountId, slug) === "granted") - ))] as const; - })) - : new Map(); - const accountNativeSlugs = accountSelectors.length > 0 - ? [...new Set([...accountNativeSlugsBySelector.values()].flatMap(slugs => [...slugs]))] - : []; - // Unknown account-native ids have no safe bare/global identity. They are only projected through - // the selector map above; the no-selector catalog remains the static native/API-key surface. - const observedNativeSlugs: string[] = []; - const wsEnabled = websocketsEnabled(config); - const multiAgentV2Enabled = isMultiAgentV2Enabled(); - const goEntries = buildCatalogEntriesFromObservedState({ - template: template ? JSON.parse(JSON.stringify(template)) : null, - gptSlugs: [], - goModels: orderedGoModels, - featured, - modelPickerOrder, - wsEnabled, - multiAgentMode, - exactComboSlugs, - accountSelectors, - suppressedBareNativeSlugs, - disabledNativeAccountSlugs: new Set(), - multiAgentV2Enabled, - openaiContextCap, - }); - // Keep genuine native entries (gpt-*, codex-*) with their real per-model fields and append - // routed providers as namespaced slugs. Cursor and other adopted providers can expose model ids - // like `gpt-5.5`; those must not delete the native OpenAI/Codex base row. - const baselineCatalog = readCatalogBackup(catalogPath); - const baseline = readNativeBaseline(catalogPath); - const gatheredProviderNames = new Set( - Object.entries(config.providers ?? {}) - .filter(([, prov]) => prov.disabled !== true) - .map(([name]) => name), - ); - const degradedProviderNames = new Set( - providerModelOutcomes - .filter(outcome => outcome.state === "degraded") - .map(outcome => outcome.provider), - ); - const selectedModelsByProvider = new Map>( - Object.entries(config.providers ?? {}).flatMap(([name, provider]) => ( - provider.disabled !== true - && Array.isArray(provider.selectedModels) - && provider.selectedModels.length > 0 - ? [[name, new Set(provider.selectedModels)] as const] - : [] - )), - ); - // Central WS capability override on the FINAL on-disk catalog (the file Codex reads). Applies to - // native AND routed so the advertised flag matches the implemented endpoint (phase 120.4) and a - // native template can never leak supports_websockets while the flag is off. - // #636: when the user only configured non-OpenAI providers (e.g. kimi), do not advertise - // bare gpt-* rows that hard-404 via NoEnabledOpenAiProviderError. Keep natives when no - // providers are configured yet (fresh install / catalog bootstrap tests). - const accountBoundEntries = includeAccountBoundNativeOpenAi && accountSelectors.length > 0 - ? buildCatalogEntriesFromObservedState({ - template: template ? JSON.parse(JSON.stringify(template)) : null, - gptSlugs: availableAccountNativeSlugs, - goModels: [], - featured, - wsEnabled, - multiAgentMode, - exactComboSlugs, - accountSelectors, - suppressedBareNativeSlugs, - disabledNativeAccountSlugs: new Set([...disabledNativeSlugs(config)].filter(slug => suppressedBareNativeSlugs.has(slug))), - multiAgentV2Enabled, - keepNativeChatGptOnV1: config.keepNativeChatGptOnV1 === true, - openaiContextCap, - accountNativeSlugs, - accountNativeSlugsBySelector, - reserve, - }).filter(entry => trustedAccountBoundNativeCatalogSlug(entry) !== undefined) - : []; - catalog.models = mergeCatalogEntriesFromObservedState({ - modelPickerOrder, - accountSelectors, - catalogModels: catalogModelsForMerge, - baselineCatalogModels: baselineCatalog?.models ?? [], - routedEntries: goEntries, - baseline, - featured, - wsEnabled, - template, - disabledModels: new Set(config.disabledModels ?? []), - selectedModelsByProvider, - gatheredProviderNames, - pendingProviderNames: pendingModelSelectionProviders(config), - degradedProviderNames, - legacyCustomModelSlugs: legacyCustomModelCatalogSlugs(config), - multiAgentMode, - multiAgentV2Enabled, - keepNativeChatGptOnV1: config.keepNativeChatGptOnV1 === true, - exactComboSlugs, - hasPhysicalComboProvider, - includeNativeOpenAi, - accountBoundEntries, - suppressedBareNativeSlugs, - openaiContextCap, - nativeDisplayNames: config.providers[OPENAI_CODEX_PROVIDER_ID]?.modelDisplayNames, - policy: { - ...CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, - nativeBackfillSlugs: [...availableBareNativeSlugs, ...observedNativeSlugs], - warningPolicy: "emit", - }, - }); - clampCatalogModelsToCodexSupport(catalog.models); - finalizeAutoReviewModelOverride(catalog.models, catalogModelsForMerge, config); - - const added = goEntries.length + accountBoundEntries.length; - const content = `${JSON.stringify(catalog, null, 2)}\n`; - // A byte-identical rewrite is not a catalog change, but every mtime-keyed reader - // has to treat it as one. The app-server staleness classifier (#857) is the one - // that matters: it compares this file's mtime against each running Codex's start - // time, so an ordinary `ocx start` — or any dashboard action that re-syncs an - // unchanged model set — marked every already-running Codex as holding an outdated - // in-memory catalog. Since #1407 that verdict silences opencodex's own model - // guidance entirely (no preferred model, no roster) for the rest of that Codex's - // lifetime, so a configured injectionModel stops reaching the session even though - // nothing about the catalog changed. Skipping the no-op write keeps both the mtime - // and `catalogWritten` honest; `added` still reports the routed rows the catalog - // carries, because they are on disk either way. - const onDiskBytes = currentCatalogFileContent(catalogPath); - if (onDiskBytes !== null && onDiskBytes.equals(Buffer.from(content, "utf8"))) { - return { added, path: catalogPath, catalogWritten: false, comboOmissions }; - } - - replaceActiveCodexCatalog(permit, owningCodexHome, { - path: catalogPath, - content, - }); - return { - added, - path: catalogPath, - catalogWritten: true, - comboOmissions, - }; -} - -function visibleAccountReplacementNatives( - models: readonly RawEntry[], - disabledModels: ReadonlySet | null, -): Map { - const replacements = new Map(); - for (const entry of models) { - const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry); - if (nativeSlug === undefined || !SUPPORTED_NATIVE_OPENAI_SLUGS.has(nativeSlug)) continue; - const exactSlug = typeof entry.slug === "string" ? entry.slug : ""; - const visible = entry.visibility === "list" - || (disabledModels !== null - && (disabledModels.has(nativeSlug) || disabledModels.has(exactSlug))); - replacements.set(nativeSlug, (replacements.get(nativeSlug) ?? true) && visible); - } - return replacements; -} - -function restoreAccountHiddenBareNatives( - entries: readonly RawEntry[], - replacementVisibility: ReadonlyMap, - disabledModels: ReadonlySet | null, -): RawEntry[] { - return entries.map(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - if ( - entry.visibility !== "hide" - || !SUPPORTED_NATIVE_OPENAI_SLUGS.has(slug) - || replacementVisibility.get(slug) !== true - || disabledModels === null - || disabledModels.has(slug) - ) { - return entry; - } - return { ...entry, visibility: "list" }; - }); -} - -function currentDisabledModelsForRestore(): Set | null { - try { - const diagnostics = readConfigDiagnostics(); - if (diagnostics.source === "fallback" || diagnostics.error !== null) return null; - return new Set(diagnostics.config.disabledModels ?? []); - } catch { - // An unreadable config cannot safely authorize a visibility change during restore. - return null; - } -} - -export async function syncCatalogModels( - config: OcxConfig, - options?: CodexCatalogSyncOptions, -): Promise { - if (pendingModelSelectionProviders(config).size) { - const { resolvePendingInitialModelSelection } = await import("../../providers/initial-model-selection-runtime"); - await resolvePendingInitialModelSelection(config); - } - const owningCodexHome = getCodexHome(); - const preflightRead = readRetainedCatalogSync(config); - if (preflightRead === null) { - return { - added: 0, - path: readCodexCatalogPath(), - catalogWritten: false, - comboOmissions: [], - refreshOutcome: "refused", - }; - } - - const comboOmissions: ComboCatalogOmission[] = []; - const providerModelOutcomes: CatalogGatherProviderModelOutcome[] = []; - // Settle the bundled template, then baseline, and only then await. Reading it - // here makes the memo ours before anyone else can move it, so a bundled swap - // during the await is an outside change rather than our own side effect. - // - // The persisted runtime selection is covered by the filesystem evidence above - // rather than by a process epoch; see `retainedCatalogProcessEvidence` for why - // the in-memory runtime memo cannot be baselined honestly from this path. - loadBundledCodexCatalog(); - const prepared: RetainedCatalogSyncRead = { - ...preflightRead, - evidence: retainedCatalogSyncEvidence(config, preflightRead.catalogPath, preflightRead.catalog), - processEvidence: retainedCatalogProcessEvidence(), - }; - const [goModels, modelEntitlements] = await Promise.all([ - gatherRoutedModels(config, { - comboOmissions, - providerModelOutcomes, - }), - resolveCodexModelEntitlements(config), - ]); - const committed = withCatalogWriteSerialization(owningCodexHome, permit => { - // Desired state can flip OFF during the provider await above. The catalog - // evidence revalidation below cannot see that — intent lives in our config, - // not in the catalog files — so the policy is re-read here, under K, right - // before the only write. A lost race becomes the discriminated skip instead - // of a routed catalog/cache surviving a completed disable. An explicit - // catalog-only sync opts out of that gate: the user asked for a refresh even - // when injection is OFF, and the toggle only protects config/history writes. - if (!shouldSyncCodexOnStart(loadConfig()) && options?.allowWhenDesiredDisabled !== true) { - return { - added: 0, - path: prepared.catalogPath, - catalogWritten: false, - comboOmissions, - skippedReason: "desired_disabled" as const, - }; - } - const current = revalidateRetainedCatalogSync(config, prepared); - if (current === null) return null; - if (!isCodexModelEntitlementSnapshotCurrent(modelEntitlements)) return null; - return writeRetainedCatalogSync({ - config, - goModels, - providerModelOutcomes, - comboOmissions, - read: current, - permit, - owningCodexHome, - modelEntitlements, - }); - }); - if (committed.kind === "completed" && committed.value !== null) { - return { - ...committed.value, - refreshOutcome: committed.value.skippedReason ? "refused" : "committed", - }; - } - return { - added: 0, - path: prepared.catalogPath, - catalogWritten: false, - comboOmissions, - refreshOutcome: "refused", - }; -} - -export function restoreCodexCatalogWithPermit( - permit: CatalogWritePermit, - owningCodexHome: string, - /** - * The catalog this injection actually wrote, when it is known (#1798). - * - * Re-resolving from the CURRENT config is wrong after a Codex app rewrite that dropped - * `model_catalog_json`: that sends restore to the default catalog while the routed file we - * really wrote is left untouched. The recorded path is the file whose routing is ours. - */ - injectedCatalogPath?: string | null, -): { removed: number; kept: number; path: string } { - const catalogPath = injectedCatalogPath ?? readCodexCatalogPath(); - const catalog = readCatalog(catalogPath); - if (!catalog || !Array.isArray(catalog.models)) return { removed: 0, kept: 0, path: catalogPath }; - const disabledModels = currentDisabledModelsForRestore(); - const replacementVisibility = visibleAccountReplacementNatives(catalog.models, disabledModels); - const backup = readCatalogBackup(catalogPath); - if (backup && Array.isArray(backup.models)) { - const removed = (catalog.models ?? []).filter(m => typeof m.slug === "string" - && (m.slug.includes("/") || RETIRED_NATIVE_OPENAI_MODELS.has(m.slug))).length; - const backupSlugs = new Set(backup.models.flatMap(m => typeof m.slug === "string" ? [m.slug] : [])); - const userNativeAdditions = restoreAccountHiddenBareNatives( - (catalog.models ?? []).filter(m => - typeof m.slug === "string" && !m.slug.includes("/") && !backupSlugs.has(m.slug) - && !RETIRED_NATIVE_OPENAI_MODELS.has(m.slug) - ), - replacementVisibility, - disabledModels, - ); - const restored = { - ...backup, - // A pristine backup predates retirement; it must not revive withdrawn native rows. - models: [...backup.models.filter(m => typeof m.slug !== "string" - || !RETIRED_NATIVE_OPENAI_MODELS.has(trustedAccountBoundNativeCatalogSlug(m) ?? m.slug)), ...userNativeAdditions], - }; - replaceActiveCodexCatalog(permit, owningCodexHome, { - path: catalogPath, - content: `${JSON.stringify(restored, null, 2)}\n`, - }); - return { removed, kept: restored.models.length, path: catalogPath }; - } - const before = catalog.models.length; - const native = restoreAccountHiddenBareNatives( - catalog.models.filter(m => !(typeof m.slug === "string" - && (m.slug.includes("/") || RETIRED_NATIVE_OPENAI_MODELS.has(m.slug)))), - replacementVisibility, - disabledModels, - ); - const removed = before - native.length; - if (removed > 0) { - catalog.models = native; - replaceActiveCodexCatalog(permit, owningCodexHome, { - path: catalogPath, - content: `${JSON.stringify(catalog, null, 2)}\n`, - }); - } - return { removed, kept: native.length, path: catalogPath }; -} - -export function restoreCodexCatalog(): { removed: number; kept: number; path: string } { - const owningCodexHome = getCodexHome(); - const outcome = withCatalogWriteSerialization( - owningCodexHome, - permit => restoreCodexCatalogWithPermit(permit, owningCodexHome), - ); - return outcome.kind === "completed" - ? outcome.value - : { removed: 0, kept: 0, path: readCodexCatalogPath() }; -} - -/** Force Codex's models_cache stale from the on-disk catalog. Returns whether a cache write occurred. */ -export function invalidateCodexModelsCacheWithPermit( - permit: CatalogWritePermit, - owningCodexHome: string, - options?: CodexCatalogSyncOptions, -): boolean { - try { - // This permit is a REACQUISITION: refreshCodexModelCatalog's commit released - // K before this rewrite runs, so the commit-path desired-state check cannot - // cover it. A disable landing in that gap must not be overwritten by a - // routed cache write — re-read intent under this permit, same as the commit. - // The catalog-only sync override applies here too so an explicit refresh - // keeps the cache consistent with the catalog it just wrote. - if (!shouldSyncCodexOnStart(loadConfig()) && options?.allowWhenDesiredDisabled !== true) return false; - const catalogPath = readCodexCatalogPathForHome(owningCodexHome); - if (!existsSync(catalogPath)) return false; - const catalog = JSON.parse(readFileSync(catalogPath, "utf8")); - const models = catalog.models ?? catalog; - const cachePath = join(owningCodexHome, "models_cache.json"); - const currentCache = readCatalog(cachePath); - const existingSlugs = new Set(models.flatMap((entry: RawEntry) => - typeof entry.slug === "string" ? [entry.slug] : [])); - const currentConfig = loadConfig(); - const mainSelectors = visibleCodexAccountSelectors(currentConfig).filter(selector => { - const target = new Map(codexAccountNamespaceEntries(currentConfig)).get(selector); - return isMainCodexAccountTarget(target ?? ""); - }); - const observedAccountModels = observedAccountBoundNativeEntries(currentCache?.models ?? []) - .filter(entry => { - const slug = typeof entry.slug === "string" ? entry.slug : ""; - return !existingSlugs.has(slug); - }) - .map(entry => ({ - ...entry, - // Keep the observation in Codex's cache without advertising a new bare picker row. The - // next OpenCodex catalog sync consumes this marker and creates only selector-qualified - // rows for the currently configured public account selectors. - visibility: "hide", - opencodex_account_observed_native: true, - opencodex_account_observed_selectors: mainSelectors, - })); - const wrapper = { - fetched_at: "2000-01-01T00:00:00Z", - client_version: "0.0.0", - models: [...models, ...observedAccountModels], - }; - replaceCodexModelsCache(permit, owningCodexHome, { - path: cachePath, - content: `${JSON.stringify(wrapper, null, 2)}\n`, - }); - return true; - } catch { - return false; - } -} - -export function invalidateCodexModelsCache(options?: CodexCatalogSyncOptions): boolean { - const owningCodexHome = getCodexHome(); - const outcome = withCatalogWriteSerialization( - owningCodexHome, - permit => invalidateCodexModelsCacheWithPermit(permit, owningCodexHome, options), - ); - return outcome.kind === "completed" && outcome.value; -} +export { + MAX_SPAWN_AGENT_MODEL_OVERRIDES, + PICKER_ORDER_PRIORITY_BASE, + SPAWN_PRIORITY_FIELD, + CATALOG_INACTIVE_REASON_FIELD, + isEligibleV2SubagentEntry, + configuredCatalogEntry, + effectiveSubagentRoster, +} from "./subagent-roster"; +export type { + SpawnAgentSurface, + SubagentRosterExclusionReason, + EffectiveSubagentModel, + SubagentRosterExclusion, + EffectiveSubagentRoster, +} from "./subagent-roster"; +export { finishUpstreamNativeEntry, isExactComboCatalogModel, deriveEntry } from "./derive-entry"; +export { + buildCatalogEntries, + buildCatalogEntriesFromObservedState, + resetCatalogRuntimeStateForTests, + orderForSubagents, + orderForModelPicker, + mergeCatalogModelsWithNativeRecovery, + applyFullModelPickerOrder, + mergeCatalogEntriesFromObservedState, + mergeCatalogEntriesForSync, + CANONICAL_NATIVE_CATALOG_CONTENT_POLICY, +} from "./build-entries"; +export type { + ObservedCatalogEntryBuildInput, + ObservedCatalogMergeInput, + ObservedCatalogMergePolicy, +} from "./build-entries"; +export { + isValidAutoReviewModel, + applyAutoReviewModelOverride, + applyConfiguredAutoReviewModelOverride, + finalizeAutoReviewModelOverride, +} from "./auto-review"; +export type { AutoReviewModelOverrideResult } from "./auto-review"; +export { + gatedNativeReauthSuppressionReason, + resetGatedNativeSuppressionWarningsForTests, +} from "./gated-native-warn"; +export { + syncCatalogModels, + invalidateCodexModelsCache, + invalidateCodexModelsCacheWithPermit, +} from "./retained-sync"; +export type { CodexCatalogSyncOptions } from "./retained-sync"; +export { restoreCodexCatalog, restoreCodexCatalogWithPermit } from "./restore"; diff --git a/src/codex/inject.ts b/src/codex/inject.ts index 9cbddf45fe..ea6424e0eb 100644 --- a/src/codex/inject.ts +++ b/src/codex/inject.ts @@ -1,11 +1,9 @@ -import { contextCompatibleBaseLine } from "./context-compat"; -import { existsSync, readFileSync, unlinkSync } from "node:fs"; +import { existsSync, readFileSync } from "node:fs"; import { atomicWriteFile, loadConfig, observeConfigGeneration, readConfigAdmissionSnapshot, - subagentDefaultSyncEffective, websocketsEnabled, withConfigMutationLockSync, } from "../config"; @@ -30,7 +28,6 @@ import { } from "./inject-coordination"; import { readIntegrationRecord } from "./integration-record"; import { classifyNativeRoutedResidue } from "./native-residue"; -import { inspectNativeCodexOwnership } from "../integrations/native/ownership-preflight"; import { resolveCodexCoordinatorDatabasePath, resolveEffectiveUserIdentity, @@ -40,14 +37,10 @@ import { markJournalInjectedState, journaledInjectedOpenaiBaseUrl, journaledInjectedRealtimeWsBaseUrl, - journaledInjectedCatalogPath, removeJournal, - restoreJournalState, writeJournal, } from "./journal"; -import { withCatalogWriteSerialization } from "./catalog-write-serialization"; -import { restoreCodexCatalogWithPermit } from "./catalog/sync"; -import { preflightCodexHistoryInjection, syncCodexHistoryProvider, type CodexHistoryFailureReason } from "./history-provider"; +import { preflightCodexHistoryInjection } from "./history-provider"; import { describeHistoryJobFailure, deriveCodexHistoryOperation, @@ -56,36 +49,49 @@ import { type CodexHistoryJobOutcome, } from "./history-job"; import { - OCX_SECTION_MARKER, REALTIME_WS_BASE_URL_KEY, hasInjectedCodexRouting, hasInjectedOpenaiBaseUrl, - isRootOpenaiBaseUrlLine, - isRootRealtimeWsBaseUrlLine, - providerTableStart, - providerTableString, rootTomlString, stripJournaledOpenaiBaseUrl, - tomlStringPattern, } from "./injected-marker"; import { CODEX_CONFIG_PATH, CODEX_PROFILE_PATH, - DEFAULT_CATALOG_PATH, getCodexHome, - parseTomlString, - readRootTomlString, - resolveCodexConfigPath, resolveCodexStateDbPath, tomlString, } from "./paths"; -import { resolveEffectiveProjectModelProvider } from "./project-config-warnings"; -import { - transformManagedSubagentDefaults, - type ManagedSubagentDefaults, -} from "./subagent-defaults"; +import { transformManagedSubagentDefaults } from "./subagent-defaults"; import type { OcxConfig } from "../types"; -import { effectiveLoopbackListenerPort, isLoopbackHostname, shouldInjectApiAuthHeader } from "./loopback-target"; +import { + configuredManagedSubagentDefaults, + standaloneCodexRoutingTarget, + usesProviderTable, + validateCodexRoutingTarget, + type CodexRoutingTarget, +} from "./inject/routing-target"; +import { + applyEol, + buildProfileFileForTarget, + buildProviderTableBlockForTarget, + chooseCatalogPathForInjection, + dominantEol, + ensureFastModeFeature, + externalCodexModelProvider, + normalizeServiceTier, + removeProfileSection, + setRootModelCatalogPath, + setRootModelProvider, + setRootOpenaiBaseUrlForTarget, + setRootRealtimeWsBaseUrl, + stripExistingModelProvider, + stripInjectedOpenaiBaseUrl, + stripOpencodexCatalogPath, + stripRootContextWindowOverrides, +} from "./inject/config-toml"; +import { hasOcxProviderTable, removeOcxSection } from "./inject/remove"; + export { effectiveLoopbackListenerPort, isLoopbackHostname, shouldInjectApiAuthHeader } from "./loopback-target"; @@ -93,37 +99,6 @@ export { effectiveLoopbackListenerPort, isLoopbackHostname, shouldInjectApiAuthH // without importing this module back. Re-exported for existing external callers. export { hasInjectedCodexRouting, hasInjectedOpenaiBaseUrl }; -export function externalCodexModelProvider(content: string): string | null { - const provider = resolveEffectiveProjectModelProvider(content).provider; - return provider && provider !== "openai" && provider !== "opencodex" - ? provider - : null; -} - -export function currentExternalCodexModelProvider(): string | null { - if (!existsSync(CODEX_CONFIG_PATH)) return null; - return externalCodexModelProvider(readFileSync(CODEX_CONFIG_PATH, "utf8")); -} - -/** - * Detect the file's dominant line ending. Every transform in this module is LF-pure - * (split("\n") + hard "\n" joins), so CRLF configs (Windows-edited config.toml) are - * normalized to LF at the pipeline boundary and converted back on write — otherwise a - * single inject would leave a mixed-EOL file. - */ -export function dominantEol(content: string): "\r\n" | "\n" { - const crlf = (content.match(/\r\n/g) ?? []).length; - if (crlf === 0) return "\n"; - const bareLf = (content.match(/\n/g) ?? []).length - crlf; - return crlf >= bareLf ? "\r\n" : "\n"; -} - -/** Normalize all line endings to `eol` (CRLF first collapsed to LF, then expanded). */ -export function applyEol(content: string, eol: "\r\n" | "\n"): string { - const lf = content.replace(/\r\n/g, "\n"); - return eol === "\n" ? lf : lf.replace(/\n/g, "\r\n"); -} - /** * Design B (2026-07-06): loopback installs no longer re-tag the provider. Instead of * `model_provider = "opencodex"` + a `[model_providers.opencodex]` table, we set the official @@ -170,727 +145,6 @@ function runClientWriteGuard(guard: InjectCodexOptions["beforeClientWrite"]): vo } } -export interface CodexRoutingTarget { - baseUrl: string; - requiresAdmissionToken: boolean; - tokenEnv: "OPENCODEX_API_AUTH_TOKEN"; - /** - * Opt-in authless Codex Desktop mode (#1107): inject the dedicated provider table with - * `requires_openai_auth = false` so Desktop skips the ChatGPT login gate. Only ever true for - * loopback targets that need no admission token; non-loopback admission is a separate layer - * and is never weakened by this flag. - */ - desktopAuthless?: boolean; - /** Select the dedicated provider identity so Codex owns compaction locally. */ - clientCompaction?: boolean; -} - -function validateCodexRoutingTarget(target: CodexRoutingTarget): CodexRoutingTarget { - let parsed: URL; - try { - parsed = new URL(target.baseUrl); - } catch { - throw new TypeError("Codex routing target must be an absolute HTTP(S) /v1 URL"); - } - if ( - (parsed.protocol !== "http:" && parsed.protocol !== "https:") - || parsed.username - || parsed.password - || parsed.pathname !== "/v1" - || parsed.search - || parsed.hash - || target.tokenEnv !== "OPENCODEX_API_AUTH_TOKEN" - ) { - throw new TypeError("Codex routing target must be a canonical HTTP(S) /v1 URL without credentials, query, or fragment"); - } - return { ...target, baseUrl: `${parsed.origin}/v1` }; -} - -/** Provider-table form is used when auth, admission, or compaction policy needs a dedicated provider. */ -function usesProviderTable(target: CodexRoutingTarget): boolean { - return target.requiresAdmissionToken - || target.desktopAuthless === true - || target.clientCompaction === true; -} - -export function standaloneCodexRoutingTarget( - port: number, - config?: Pick< - OcxConfig, - "hostname" | "unauthenticatedLoopbackListener" | "codexDesktopAuthless" | "codexClientCompaction" - >, -): CodexRoutingTarget { - // An enabled listener with no `port` is the companion form: it answers on `port` itself, - // bound to 127.0.0.1 (#4236). Resolving it through the shared helper is what makes the - // one-port hub work without every writer repeating `?? port`. - const loopback = config?.unauthenticatedLoopbackListener; - const effectivePort = effectiveLoopbackListenerPort(config, port) ?? port; - const hostname = loopback?.enabled ? undefined : config?.hostname; - const requiresAdmissionToken = loopback?.enabled ? false : shouldInjectApiAuthHeader(config); - return { - baseUrl: `http://${providerBaseHost(hostname)}:${effectivePort}/v1`, - requiresAdmissionToken, - tokenEnv: "OPENCODEX_API_AUTH_TOKEN", - ...(config?.codexDesktopAuthless === true && !requiresAdmissionToken - ? { desktopAuthless: true } - : {}), - ...(config?.codexClientCompaction === true && !requiresAdmissionToken - ? { clientCompaction: true } - : {}), - }; -} - -function routingTargetOrigin(target: CodexRoutingTarget): string { - return target.baseUrl.slice(0, -3); -} - -function configuredManagedSubagentDefaults( - config: - | Pick< - OcxConfig, - "injectionModel" | "injectionEffort" | "syncCodexSubagentDefaults" - > - | undefined, -): ManagedSubagentDefaults | null { - if (!subagentDefaultSyncEffective(config ?? {})) return null; - return { - model: config!.injectionModel!.trim(), - ...(config!.injectionEffort?.trim() - ? { reasoningEffort: config!.injectionEffort.trim() } - : {}), - }; -} - -/** - * The `[model_providers.opencodex]` TABLE only. A table is position-independent in TOML, so it is - * safe to append at EOF. The bare root key `model_provider = "opencodex"` is NOT included here — - * it must live at the document root (before any table header) and is set separately by - * setRootModelProvider(). Appending the bare key at EOF was the original bug: it nested under - * whatever `[table]` happened to be open last (e.g. `[plugins."chrome@openai-bundled"]`), so Codex - * never saw a global model_provider and silently fell back to the `openai` (ChatGPT) provider. - */ -export function providerBaseHost(hostname: string | undefined): string { - const trimmed = (hostname ?? "127.0.0.1").trim(); - const lower = trimmed.toLowerCase(); - // Match what the server actually binds. Writing "localhost" while binding IPv4-only - // 127.0.0.1 breaks on Windows, where localhost commonly resolves to ::1 first. - if (lower === "::1" || lower === "[::1]") return "[::1]"; - if ( - isLoopbackHostname(trimmed) || - trimmed === "0.0.0.0" || - trimmed === "::" || - trimmed === "[::]" - ) - return "127.0.0.1"; - if (trimmed.startsWith("[") && trimmed.endsWith("]")) return trimmed; - return trimmed.includes(":") ? `[${trimmed}]` : trimmed; -} - -export function buildProviderTableBlock( - port: number, - supportsWebsockets?: boolean, - includeApiAuthHeader?: boolean, - hostname?: string, -): string; -export function buildProviderTableBlock( - target: CodexRoutingTarget, - supportsWebsockets?: boolean, -): string; -export function buildProviderTableBlock( - portOrTarget: number | CodexRoutingTarget, - supportsWebsockets = false, - includeApiAuthHeader = false, - hostname?: string, -): string { - const target = typeof portOrTarget === "number" - ? validateCodexRoutingTarget({ - baseUrl: `http://${providerBaseHost(hostname)}:${portOrTarget}/v1`, - requiresAdmissionToken: includeApiAuthHeader, - tokenEnv: "OPENCODEX_API_AUTH_TOKEN", - }) - : validateCodexRoutingTarget(portOrTarget); - return buildProviderTableBlockForTarget(target, supportsWebsockets); -} - -function buildProviderTableBlockForTarget( - target: CodexRoutingTarget, - supportsWebsockets = false, -): string { - const lines = [ - "", - OCX_SECTION_MARKER, - "[model_providers.opencodex]", - 'name = "OpenCodex Proxy"', - `base_url = ${tomlString(target.baseUrl)}`, - 'wire_api = "responses"', - // false only in the authless Desktop opt-in (#1107); true keeps the App/TUI account gate. - `requires_openai_auth = ${target.desktopAuthless === true ? "false" : "true"}`, - ]; - if (target.requiresAdmissionToken) { - // codex-cli 0.146+ contract (#2073): env_key sends Authorization: Bearer $VAR and - // hard-errors on a missing/empty variable instead of silently omitting auth. It - // coexists with requires_openai_auth (env_key wins wire auth; the flag keeps the - // login/account UX), and the server substitutes stored main auth for our admission - // bearer (#1686), so the modern form is strictly better than the legacy - // env_http_headers table this line used to emit. - lines.push(`env_key = ${tomlString(target.tokenEnv)}`); - } - if (supportsWebsockets) lines.push("supports_websockets = true"); - return lines.join("\n") + "\n"; -} - -export function buildOpenaiBaseUrlLine( - port: number, - hostname?: string, -): string; -export function buildOpenaiBaseUrlLine(target: CodexRoutingTarget): string; -export function buildOpenaiBaseUrlLine( - portOrTarget: number | CodexRoutingTarget, - hostname?: string, -): string { - return typeof portOrTarget === "number" - ? `openai_base_url = "http://${providerBaseHost(hostname)}:${portOrTarget}/v1"` - : buildOpenaiBaseUrlLineForTarget(validateCodexRoutingTarget(portOrTarget)); -} - -function buildOpenaiBaseUrlLineForTarget(target: CodexRoutingTarget): string { - return `openai_base_url = ${tomlString(target.baseUrl)}`; -} - -/** - * Realtime sideband override (codex-rs `experimental_realtime_ws_base_url`), written with the - * SAME value as `openai_base_url`. Desktop voice creates its WebRTC call through the proxy - * (`POST /v1/live`, answered under the Pool account the proxy selects) but, since openai/codex - * 438c9e98d (#35830), joins the sideband at `wss://api.openai.com/v1/live/{callId}` with the - * app's own login unless this key redirects it. Two accounts, one call: the join 404s. Pointing - * the key at the proxy sends the join through `GET /v1/live/{callId}` (src/server/live.ts), - * where the same Pool account is reused. codex-rs turns `http` into `ws` and appends - * `/live/{callId}` itself; the value must stay the canonical `/v1` root. - */ -export function buildRealtimeWsBaseUrlLine(target: CodexRoutingTarget): string { - return `${REALTIME_WS_BASE_URL_KEY} = ${tomlString(target.baseUrl)}`; -} - -/** - * Design B root-key injection: place `OCX_SECTION_MARKER` + `openai_base_url` at the document - * ROOT (before the first table header). Idempotent: an existing marker-owned line is rewritten - * in place. A user's OWN root `openai_base_url` (no marker above it) is respected — we keep it - * and inject nothing, reporting `keptUserBaseUrl` so the caller can surface it. - */ -export function setRootOpenaiBaseUrl( - content: string, - port: number, - hostname?: string, -): { content: string; keptUserBaseUrl: boolean }; -export function setRootOpenaiBaseUrl( - content: string, - target: CodexRoutingTarget, -): { content: string; keptUserBaseUrl: boolean }; -export function setRootOpenaiBaseUrl( - content: string, - portOrTarget: number | CodexRoutingTarget, - hostname?: string, -): { content: string; keptUserBaseUrl: boolean } { - if (typeof portOrTarget !== "number") { - return setRootOpenaiBaseUrlForTarget(content, validateCodexRoutingTarget(portOrTarget)); - } - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const key = contextCompatibleBaseLine(content, buildOpenaiBaseUrlLine(portOrTarget, hostname)); - - for (let i = 0; i < rootEnd; i++) { - if (!isRootOpenaiBaseUrlLine(lines[i])) continue; - const markerOwned = i > 0 && lines[i - 1].includes(OCX_SECTION_MARKER); - if (!markerOwned) return { content, keptUserBaseUrl: true }; - lines[i] = key; - return { content: lines.join("\n"), keptUserBaseUrl: false }; - } - - if (firstTable === -1) { - return { - content: - content.replace(/\n+$/, "") + - "\n" + - OCX_SECTION_MARKER + - "\n" + - key + - "\n", - keptUserBaseUrl: false, - }; - } - let insertAt = firstTable; - while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; - lines.splice(insertAt, 0, OCX_SECTION_MARKER, key); - return { content: lines.join("\n"), keptUserBaseUrl: false }; -} - -function setRootOpenaiBaseUrlForTarget( - content: string, - target: CodexRoutingTarget, -): { content: string; keptUserBaseUrl: boolean } { - const lines = content.split("\n"); - const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const key = contextCompatibleBaseLine(content, buildOpenaiBaseUrlLineForTarget(target)); - for (let index = 0; index < rootEnd; index += 1) { - if (!isRootOpenaiBaseUrlLine(lines[index])) continue; - const markerOwned = index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER); - if (!markerOwned) return { content, keptUserBaseUrl: true }; - lines[index] = key; - return { content: lines.join("\n"), keptUserBaseUrl: false }; - } - if (firstTable === -1) { - return { - content: `${content.replace(/\n+$/, "")}\n${OCX_SECTION_MARKER}\n${key}\n`, - keptUserBaseUrl: false, - }; - } - let insertAt = firstTable; - while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt -= 1; - lines.splice(insertAt, 0, OCX_SECTION_MARKER, key); - return { content: lines.join("\n"), keptUserBaseUrl: false }; -} - -/** - * Companion to `setRootOpenaiBaseUrlForTarget` for the realtime sideband override. Same - * ownership rule, applied per key: the line is ours only when the marker sits directly - * above it; a user's own line (no marker above it) is kept and nothing is injected. The - * key gets its OWN marker line rather than sharing the routing override's, so a user line - * that happens to sit right under our `openai_base_url` is never mistaken for ours. - * Placement: directly after the marker-owned `openai_base_url` pair. Only ever called on - * the Design B (loopback) path right after the routing override was written — the legacy - * provider-table form needs the admission-token header, which the sideband cannot carry. - */ -export function setRootRealtimeWsBaseUrl( - content: string, - target: CodexRoutingTarget, -): { content: string; keptUserRealtimeWsBaseUrl: boolean } { - const lines = content.split("\n"); - const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const key = buildRealtimeWsBaseUrlLine(validateCodexRoutingTarget(target)); - for (let index = 0; index < rootEnd; index += 1) { - if (!isRootRealtimeWsBaseUrlLine(lines[index])) continue; - const markerOwned = index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER); - if (!markerOwned) return { content, keptUserRealtimeWsBaseUrl: true }; - lines[index] = key; - return { content: lines.join("\n"), keptUserRealtimeWsBaseUrl: false }; - } - for (let index = 0; index < rootEnd; index += 1) { - if (!isRootOpenaiBaseUrlLine(lines[index])) continue; - if (!(index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER))) continue; - lines.splice(index + 1, 0, OCX_SECTION_MARKER, key); - return { content: lines.join("\n"), keptUserRealtimeWsBaseUrl: false }; - } - // No marker-owned routing override to attach to: the override has no owner, so inject nothing. - return { content, keptUserRealtimeWsBaseUrl: false }; -} - -/** - * Remove the marker-owned root `openai_base_url` (marker line + the key line right after it). - * A user's own root override (no marker) survives; an orphaned marker with no key line after - * it is dropped too so repeated strip/inject cycles cannot accumulate marker comments. - * A marker-owned `experimental_realtime_ws_base_url` pair is removed by the same rule. - */ -export function stripInjectedOpenaiBaseUrl(content: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const drop = new Set(); - for (let i = 0; i < rootEnd; i++) { - if (!lines[i].includes(OCX_SECTION_MARKER)) continue; - if (i + 1 < rootEnd && (isRootOpenaiBaseUrlLine(lines[i + 1]) || isRootRealtimeWsBaseUrlLine(lines[i + 1]))) { - drop.add(i); - drop.add(i + 1); - } else if (i + 1 >= rootEnd || lines[i + 1].trim() === "") { - drop.add(i); // orphaned marker at root - } - } - if (drop.size === 0) return content; - return lines.filter((_, i) => !drop.has(i)).join("\n"); -} - -export type CodexRoutingKind = - "native" | "opencodex-local" | "custom-local" | "custom-remote" | "unknown"; - -type RoutingEndpointKind = "local" | "remote" | "unknown"; - -function ipv4Octets(hostname: string): number[] | null { - const dotted = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(hostname); - if (dotted) { - const octets = dotted.slice(1).map(Number); - return octets.some((octet) => octet > 255) ? null : octets; - } - const mapped = /^::ffff:([0-9a-f]{1,4}):([0-9a-f]{1,4})$/i.exec(hostname); - if (!mapped) return null; - const high = Number.parseInt(mapped[1], 16); - const low = Number.parseInt(mapped[2], 16); - return [high >>> 8, high & 0xff, low >>> 8, low & 0xff]; -} - -function classifyRoutingEndpoint(value: string): RoutingEndpointKind { - try { - const url = new URL(value); - if (url.protocol !== "http:" && url.protocol !== "https:") return "unknown"; - const hostname = url.hostname - .toLowerCase() - .replace(/^\[|\]$/g, "") - .replace(/\.$/, ""); - if (!hostname) return "unknown"; - if (hostname === "localhost" || hostname.endsWith(".localhost")) - return "local"; - if (hostname === "::" || hostname === "::1" || hostname === "0.0.0.0") - return "local"; - const octets = ipv4Octets(hostname); - if (octets) { - if (octets.every((octet) => octet === 0)) return "local"; - if (octets[0] === 127) return "local"; - return "remote"; - } - if (/^::ffff:/i.test(hostname)) return "unknown"; - return "remote"; - } catch { - return "unknown"; - } -} - -/** Classify actual routing dependency separately from opencodex ownership. */ -export function classifyCodexRouting(content: string): CodexRoutingKind { - const rootBaseUrl = rootTomlString(content, "openai_base_url"); - if (rootBaseUrl) { - const endpoint = classifyRoutingEndpoint(rootBaseUrl); - if (endpoint === "unknown") return "unknown"; - if (hasInjectedOpenaiBaseUrl(content)) return "opencodex-local"; - return endpoint === "local" ? "custom-local" : "custom-remote"; - } - const rootProvider = rootTomlString(content, "model_provider"); - if (rootProvider) { - const providerTableExists = - providerTableStart(content.split("\n"), rootProvider) !== -1; - const providerBaseUrl = providerTableString( - content, - rootProvider, - "base_url", - ); - if (providerBaseUrl) { - const endpoint = classifyRoutingEndpoint(providerBaseUrl); - if (endpoint === "unknown") return "unknown"; - if (rootProvider === "opencodex") return "opencodex-local"; - return endpoint === "local" ? "custom-local" : "custom-remote"; - } - if ( - rootProvider === "opencodex" || - providerTableExists || - rootProvider !== "openai" - ) - return "unknown"; - } - return "native"; -} - -/** Read-only probe used by status, doctor, and the dashboard. */ -export function isCodexRoutingInjected(): boolean { - const path = CODEX_CONFIG_PATH; - if (!existsSync(path)) return false; - try { - return hasInjectedCodexRouting(readFileSync(path, "utf8")); - } catch { - return false; - } -} - -export function getCodexRoutingKind(): CodexRoutingKind { - const path = CODEX_CONFIG_PATH; - if (!existsSync(path)) return "native"; - try { - return classifyCodexRouting(readFileSync(path, "utf8")); - } catch { - return "unknown"; - } -} - -/** - * Strip every existing `model_provider` line that we must not duplicate: any line set to - * "opencodex" (wherever it sits — including a previously mis-nested one under a table), plus any - * ROOT-level model_provider (before the first table) of any value, since we override the global. - * A `model_provider` legitimately inside a user table/profile with a non-opencodex value is left - * untouched. - */ -function stripExistingModelProvider(content: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - const out: string[] = []; - lines.forEach((line, i) => { - if (/^\s*model_provider\s*=/.test(line)) { - const isOurs = /^\s*model_provider\s*=\s*"opencodex"\s*$/.test(line); - const isRoot = firstTable === -1 || i < firstTable; - if (isOurs || isRoot) return; // drop it - } - out.push(line); - }); - return out.join("\n"); -} - -/** - * Drop ROOT-level `model_context_window` overrides (keys before the first table header). Codex - * treats this root key as a global override that wins over the per-model catalog values, so a stale - * `model_context_window = 1000000` makes every model (e.g. gpt-5.5) report a 1M window. User-owned - * compaction limits do not alter the advertised context window and must survive reinjection. - */ -export function stripRootContextWindowOverrides(content: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - return lines - .filter((line, i) => { - const isRoot = firstTable === -1 || i < firstTable; - return !isRoot || !/^\s*model_context_window\s*=/.test(line); - }) - .join("\n"); -} - -function stripRootRoutedModel(content: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - return lines - .filter((line, i) => { - const isRoot = firstTable === -1 || i < firstTable; - if (!isRoot) return true; - const m = line.match(/^\s*model\s*=\s*("(?:\\.|[^"\\])*"|'[^']*')\s*$/); - if (!m) return true; - const model = parseTomlString(m[1]); - return !model?.includes("/"); - }) - .join("\n"); -} - -/** - * Insert `model_provider = "opencodex"` at the document ROOT — immediately before the first table - * header (TOML root keys must precede all tables). If there are no tables, append it to the root body. - */ -function setRootModelProvider(content: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - const key = 'model_provider = "opencodex"'; - if (firstTable === -1) { - return content.replace(/\n+$/, "") + "\n" + key + "\n"; - } - let insertAt = firstTable; - while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; - lines.splice(insertAt, 0, key); - return lines.join("\n"); -} - -function readRootModelCatalogPath(content: string): string | null { - const lines = content.split("\n"); - const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); - let ownedCatalogPath: string | null = null; - for (let index = 0; index < rootEnd; index += 1) { - const match = modelCatalogAssignment.exec(lines[index]); - if (!match) continue; - const catalogPath = parseTomlString(match[1]); - if (!isOpencodexCatalogPath(catalogPath)) return catalogPath; - ownedCatalogPath ??= catalogPath; - } - return ownedCatalogPath; -} - -function setRootModelCatalogPath(content: string, catalogPath: string): string { - const lines = content.split("\n"); - const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); - const key = `model_catalog_json = ${tomlString(catalogPath)}`; - const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - const ownedAssignments: number[] = []; - let hasUserAssignment = false; - for (let i = 0; i < rootEnd; i++) { - const m = modelCatalogAssignment.exec(lines[i]); - if (!m) continue; - const existing = parseTomlString(m[1]); - if (isOpencodexCatalogPath(existing)) { - ownedAssignments.push(i); - } else { - hasUserAssignment = true; - } - } - if (hasUserAssignment) { - const owned = new Set(ownedAssignments); - return lines.filter((_, index) => !owned.has(index)).join("\n"); - } - if (ownedAssignments.length > 0) { - lines[ownedAssignments[0]] = key; - const duplicates = new Set(ownedAssignments.slice(1)); - return lines.filter((_, index) => !duplicates.has(index)).join("\n"); - } - if (firstTable === -1) { - return content.replace(/\n+$/, "") + "\n" + key + "\n"; - } - let insertAt = firstTable; - while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; - lines.splice(insertAt, 0, key); - return lines.join("\n"); -} - -function removeProfileSection(content: string): string { - const lines = content.split("\n"); - const filtered: string[] = []; - let inProfile = false; - for (const line of lines) { - if (line.trim() === "[profiles.opencodex]") { - inProfile = true; - continue; - } - if (inProfile) { - if (/^\s*\[/.test(line) && line.trim() !== "[profiles.opencodex]") { - inProfile = false; - filtered.push(line); - } - continue; - } - filtered.push(line); - } - return ( - filtered - .join("\n") - .replace(/\n{3,}/g, "\n\n") - .trimEnd() + "\n" - ); -} - -function normalizeServiceTier(content: string): string { - return content.replace( - /^(\s*service_tier\s*=\s*)["']priority["']\s*$/gm, - '$1"fast"', - ); -} - -function ensureFastModeFeature(content: string, fastMode?: boolean): string { - // Tri-state fast mode (see OcxConfig.fastMode): true forces `fast_mode = true`, - // false forces `fast_mode = false`, and undefined leaves the user's config - // untouched (no [features] table is added and an existing fast_mode line is - // preserved as-is). Table and key matching accept the valid TOML spellings - // `[features] # comment`, `["features"]` / `['features']`, and quoted keys. - const lines = content.split("\n"); - const featuresHeader = /^\s*\[(["']?)\s*features\s*\1\]\s*(?:#.*)?$/; - const fastModeKey = /^\s*(?:"fast_mode"|'fast_mode'|fast_mode)\s*=/; - const featuresStart = lines.findIndex(line => featuresHeader.test(line)); - if (featuresStart === -1) { - if (fastMode === undefined) return content; - return content.trimEnd() + "\n\n[features]\nfast_mode = " + (fastMode ? "true" : "false") + "\n"; - } - - const nextTable = lines.findIndex( - (line, index) => index > featuresStart && /^\s*\[/.test(line), - ); - const featuresEnd = nextTable === -1 ? lines.length : nextTable; - for (let i = featuresStart + 1; i < featuresEnd; i++) { - if (fastModeKey.test(lines[i])) { - if (fastMode === undefined) return lines.join("\n"); - lines[i] = lines[i].replace(/^(\s*)(?:"fast_mode"|'fast_mode'|fast_mode)\s*=.*$/, `$1fast_mode = ${fastMode ? "true" : "false"}`); - return lines.join("\n"); - } - } - - if (fastMode === undefined) return lines.join("\n"); - let insertAt = featuresEnd; - while (insertAt > featuresStart + 1 && lines[insertAt - 1].trim() === "") insertAt--; - lines.splice(insertAt, 0, `fast_mode = ${fastMode ? "true" : "false"}`); - return lines.join("\n"); -} - -function isOpencodexCatalogPath(path: string): boolean { - return path.replace(/\\/g, "/").split("/").pop() === "opencodex-catalog.json"; -} - -function stripOpencodexCatalogPath(content: string): string { - const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); - const lines = content.split("\n"); - const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); - const rootEnd = firstTable === -1 ? lines.length : firstTable; - return lines - .filter((line, index) => { - if (index >= rootEnd) return true; - const m = modelCatalogAssignment.exec(line); - return !m || !isOpencodexCatalogPath(parseTomlString(m[1])); - }) - .join("\n"); -} - -export function buildProfileFile(port: number, catalogPath?: string | null, supportsWebsockets?: boolean, includeApiAuthHeader?: boolean, hostname?: string, fastMode?: boolean): string; -export function buildProfileFile(target: CodexRoutingTarget, catalogPath?: string | null, supportsWebsockets?: boolean, fastMode?: boolean): string; -export function buildProfileFile( - portOrTarget: number | CodexRoutingTarget, - catalogPath?: string | null, - supportsWebsockets = false, - includeApiAuthHeaderOrFastMode?: boolean, - hostname?: string, - fastMode?: boolean, -): string { - const target = typeof portOrTarget === "number" - ? validateCodexRoutingTarget({ - baseUrl: `http://${providerBaseHost(hostname)}:${portOrTarget}/v1`, - requiresAdmissionToken: includeApiAuthHeaderOrFastMode === true, - tokenEnv: "OPENCODEX_API_AUTH_TOKEN", - }) - : validateCodexRoutingTarget(portOrTarget); - return buildProfileFileForTarget( - target, - catalogPath, - supportsWebsockets, - typeof portOrTarget === "number" ? fastMode : includeApiAuthHeaderOrFastMode, - ); -} - -function buildProfileFileForTarget( - target: CodexRoutingTarget, - catalogPath?: string | null, - supportsWebsockets = false, - fastMode?: boolean, -): string { - const origin = routingTargetOrigin(target); - const host = new URL(origin).host; - // Design B (loopback): the reference/fallback file documents the root override form. - // Non-loopback keeps the legacy provider-table shape (built-in provider cannot carry - // the x-opencodex-api-key env header); explicit Desktop policies share that shape. - if (!usesProviderTable(target)) { - const lines = [ - "# OpenCodex proxy fallback config (Design B)", - `# Root override that points Codex's built-in openai provider at the proxy on ${host}.`, - "# Merge these root keys into ~/.codex/config.toml manually if auto-injection was removed.", - buildOpenaiBaseUrlLineForTarget(target), - ]; - if (catalogPath) lines.push(`model_catalog_json = ${tomlString(catalogPath)}`); - if (fastMode !== undefined) lines.push("", "[features]", `fast_mode = ${fastMode ? "true" : "false"}`, ""); - return lines.join("\n"); - } - const lines = [ - "# OpenCodex proxy profile — use with: codex --profile opencodex", - `# Routes all model requests through the opencodex proxy at ${host}`, - 'model_provider = "opencodex"', - ]; - if (catalogPath) lines.push(`model_catalog_json = ${tomlString(catalogPath)}`); - if (fastMode !== undefined) lines.push("", "[features]", `fast_mode = ${fastMode ? "true" : "false"}`); - lines.push(buildProviderTableBlockForTarget(target, supportsWebsockets).trimEnd(), ""); - return lines.join("\n"); -} - -export function chooseCatalogPathForInjection( - content: string, - requested?: string | null, -): string | null { - if (requested !== undefined) return requested; - - const existing = readRootModelCatalogPath(content); - if (existing) { - const resolved = resolveCodexConfigPath(existing); - if (!isOpencodexCatalogPath(resolved) || existsSync(resolved)) - return existing; - } - - return existsSync(DEFAULT_CATALOG_PATH) ? DEFAULT_CATALOG_PATH : null; -} export interface CodexInjectResult { success: boolean; @@ -916,19 +170,10 @@ export interface CodexInjectResult { const HISTORY_RELABEL_STANDS_DOWN = "history_paginated_requires_native_writer"; class CodexHistoryPreflightRefusal extends Error {} -class CodexRestoreRefusal extends Error { - constructor(readonly config: CodexRestoreConfigResult) { - super(config.message); - } -} let historyArtifactStageForTests: ((stage: string) => void) | undefined; export function setHistoryArtifactStageForTests(hook: typeof historyArtifactStageForTests): void { historyArtifactStageForTests = hook; } -let beforeRestoreConfigForTests: ((kind: string) => void) | undefined; -export function setBeforeRestoreConfigForTests(hook: typeof beforeRestoreConfigForTests): void { - beforeRestoreConfigForTests = hook; -} let beforeHistoryArtifactCommitForTests: ((kind: string) => void) | undefined; export function setBeforeHistoryArtifactCommitForTests(hook: typeof beforeHistoryArtifactCommitForTests): void { beforeHistoryArtifactCommitForTests = hook; @@ -1661,656 +906,6 @@ async function injectCodexConfigImpl( }; } -/** - * Sub-table headers like `[model_providers.opencodex.env_http_headers]` appear when a Codex app - * config rewrite re-serializes the provider's inline `env_http_headers` table. They define the - * same `model_providers.opencodex` provider, so cleanup must remove them too — otherwise the - * provider survives with no `name` and Codex rejects the whole config - * ("provider name must not be empty"). The dot terminator keeps a user's - * `[model_providers.opencodex_backup]`-style tables out of scope. - */ -function isOcxProviderHeaderLine(trimmedLine: string): boolean { - // Root form matched by regex, not equality: TOML v1.0 allows a trailing comment - // (`[model_providers.opencodex] # comment`), and an exact compare would miss that form. - // The sub-table prefix check already tolerates trailing comments by construction. - return ( - /^\[model_providers\.opencodex\]\s*(?:#.*)?$/.test(trimmedLine) || - trimmedLine.startsWith("[model_providers.opencodex.") - ); -} - -function hasOcxProviderTable(content: string): boolean { - return content - .split("\n") - .some((line) => isOcxProviderHeaderLine(line.trim())); -} - -function removeOcxSection(content: string): string { - const lines = content.split("\n"); - const filtered: string[] = []; - let inOcxSection = false; - for (const line of lines) { - if ( - line.includes(OCX_SECTION_MARKER) || - isOcxProviderHeaderLine(line.trim()) - ) { - inOcxSection = true; - continue; - } - if (inOcxSection) { - // End the injected section at the next table header that ISN'T our own. Exact match on the - // provider name (plus our own sub-tables) so a user's - // "[model_providers.opencodex_backup]" (or similar) is preserved, not swallowed. - if (/^\s*\[/.test(line) && !isOcxProviderHeaderLine(line.trim())) { - inOcxSection = false; - filtered.push(line); - } - continue; - } - filtered.push(line); - } - return ( - filtered - .join("\n") - .replace(/\n{3,}/g, "\n\n") - .trimEnd() + "\n" - ); -} - -interface StripOpencodexConfigResult { - content: string; - managedDefaultsError: string | null; -} - -/** - * Detailed form used by the on-disk restore path. A damaged ownership marker is - * ambiguous: keep the associated value, but return the transform error so the - * caller cannot report a complete restore. - */ -function stripOpencodexConfigResult( - content: string, - journaledBaseUrl: string | null = null, - journaledRealtimeWsBaseUrl: string | null = null, -): StripOpencodexConfigResult { - let out = content; - const hadRootOcxProvider = - readRootTomlString(out, "model_provider") === "opencodex"; - // #1798: marker adjacency is FORMATTING evidence, and a Codex app rewrite keeps values - // while dropping comments. Fall back to VALUE evidence -- the exact URL we recorded - // writing -- so an app-rewritten config is still recognized as ours. - const hadInjectedBaseUrl = hasInjectedOpenaiBaseUrl(out) - || (journaledBaseUrl !== null && rootTomlString(out, "openai_base_url") === journaledBaseUrl); - out = stripInjectedOpenaiBaseUrl(out); // before removeOcxSection — it keys on the marker line too - out = stripJournaledOpenaiBaseUrl(out, journaledBaseUrl, journaledRealtimeWsBaseUrl); - if (hasOcxProviderTable(out)) { - out = removeOcxSection(out); - } - out = removeProfileSection(out); - // Regex (not exact-string) removal so compact `model_provider="opencodex"` is stripped too — - // must match the detection regex above, or a detected line could survive un-removed. - out = out - .split("\n") - .filter((l) => !/^\s*model_provider\s*=\s*"opencodex"\s*$/.test(l)) - .join("\n"); - // Routed root model ids (`model = "provider/slug"`) only make sense while the proxy serves - // them — strip on both the legacy re-tag form and the Design B injected-base-url form. - if (hadRootOcxProvider || hadInjectedBaseUrl) out = stripRootRoutedModel(out); - const managedDefaults = transformManagedSubagentDefaults(out, null); - if (managedDefaults.ok) out = managedDefaults.content; - out = stripOpencodexCatalogPath(out); - return { - content: out.replace(/\n{3,}/g, "\n\n").trimEnd() + "\n", - managedDefaultsError: !managedDefaults.ok ? managedDefaults.error : null, - }; -} - -/** Pure transform: strip the opencodex provider block + `model_provider = "opencodex"` lines. */ -export function stripOpencodexConfig(content: string): string { - return stripOpencodexConfigResult(content).content; -} - -function hasOpencodexRouting(content: string): boolean { - return ( - hasOcxProviderTable(content) || - /^\s*model_provider\s*=\s*"opencodex"/m.test(content) || - hasInjectedOpenaiBaseUrl(content) - ); -} - -export function removeCodexConfig( - options: { preserveProfile?: boolean } = {}, -): { success: boolean; message: string } { - const historyError = preflightCodexHistoryInjection(false, false); - if (historyError) return { success: false, message: `Codex configuration preserved: ${historyError}. Native writer coordination is required.` }; - if (!existsSync(CODEX_CONFIG_PATH)) { - if (!options.preserveProfile && existsSync(CODEX_PROFILE_PATH)) - unlinkSync(CODEX_PROFILE_PATH); - return { - success: true, - message: `Codex config not found; no native restore was needed${options.preserveProfile ? "." : ", and the opencodex profile was removed if present."}`, - }; - } - const rawContent = readFileSync(CODEX_CONFIG_PATH, "utf-8"); - // Same EOL boundary as inject: strip in LF space, write back in the file's own ending. - // The unchanged fast path compares in LF space so an untouched file is never rewritten. - const eol = dominantEol(rawContent); - const content = applyEol(rawContent, "\n"); - // Read the recorded injection once: the strip below consumes it, and so does the - // ownership verdict, which must agree with what was actually removed. - const journaledBaseUrl = journaledInjectedOpenaiBaseUrl(); - const journaledRealtimeWsBaseUrl = journaledInjectedRealtimeWsBaseUrl(); - const had = hasOpencodexRouting(content) - || (journaledBaseUrl !== null && rootTomlString(content, "openai_base_url") === journaledBaseUrl) - || (journaledRealtimeWsBaseUrl !== null - && rootTomlString(content, REALTIME_WS_BASE_URL_KEY) === journaledRealtimeWsBaseUrl); - const stripped = stripOpencodexConfigResult(content, journaledBaseUrl, journaledRealtimeWsBaseUrl); - if (had || stripped.content !== content) { - atomicWriteFile(CODEX_CONFIG_PATH, applyEol(stripped.content, eol)); - } - if (!options.preserveProfile && existsSync(CODEX_PROFILE_PATH)) - unlinkSync(CODEX_PROFILE_PATH); - const removedMessage = had - ? `Removed opencodex routing from Codex config${options.preserveProfile ? "." : " + profile."}` - : "opencodex not present in Codex config."; - if (stripped.managedDefaultsError) { - const routingMessage = had - ? removedMessage - : "No opencodex routing was present in Codex config."; - return { - success: false, - message: - `${routingMessage} Native Codex sub-agent defaults could not be safely removed: ${stripped.managedDefaultsError}. ` + - "The ambiguous marker and adjacent value were preserved; inspect $CODEX_HOME/config.toml before using native Codex.", - }; - } - return { - success: true, - message: removedMessage, - }; -} - -export type CodexRestoreArtifactState = "ok" | "skipped" | "failed"; - -export interface CodexRestoreConfigResult { - state: CodexRestoreArtifactState; - changed: boolean; - action: "journal-restored" | "owned-fields-stripped" | "external-provider-preserved" | "failed"; - message: string; -} - -export interface CodexRestoreCatalogResult { - state: CodexRestoreArtifactState; - changed: boolean; - removed: number; - kept: number; - path: string | null; - message: string; -} - -export interface CodexRestoreHistoryResult { - state: CodexRestoreArtifactState; - changed: boolean; - reason?: CodexHistoryFailureReason; - rows: number; - files: number; - ejectedRows: number; - message: string; -} - -export interface CodexNativeRestoreResult { - success: boolean; - message: string; - externalProvider?: string; - artifacts: { - config: CodexRestoreConfigResult; - catalog: CodexRestoreCatalogResult; - history: CodexRestoreHistoryResult; - }; -} - -function failedHistoryRestore( - reason?: CodexHistoryFailureReason, - detail?: string, - progress: { rows?: number; files?: number } = {}, -): CodexRestoreHistoryResult { - const rows = progress.rows ?? 0; - const files = progress.files ?? 0; - const changed = rows > 0 || files > 0; - return { - state: "failed", - changed, - ...(reason ? { reason } : {}), - rows, - files, - ejectedRows: 0, - message: reason === "permission" - ? changed - ? "Codex resume history changed but did NOT converge because permission was denied while finalizing the backup manifest; the manifest was retained for review and safe retry." - : "Codex resume history could NOT be restored because permission was denied." - : reason === "busy" - ? changed - ? "Codex resume history changed but did NOT converge because backup-manifest finalization remained busy; the manifest was retained for review and safe retry." - : detail ?? "Codex resume history could NOT be restored — the Codex app appears to be holding the history database." - : reason === "integrity" - ? changed - ? "Codex resume history changed but did NOT converge because the backup or target changed; the manifest was retained for review and safe retry." - : "Codex resume history could NOT be restored because the backup or restore target failed integrity checks; unverified provider metadata was left unchanged." - : detail - ? `Codex resume history could NOT be restored: ${detail}` - : "Codex resume history could NOT be restored; the reason was not recorded. Run 'ocx doctor'.", - }; -} - -/** - * Restore failure wording for a Worker outcome. - * - * Only a genuine busy result blames the Codex app. An unsafe-path refusal, an - * unavailable coordinator database, a permission denial, or a dead/timed-out - * worker is a different problem; the old collapse made every one of those read - * as "the Codex app is holding the database" (issue #1191). `busy` and - * `permission` keep the restore-specific sentence built by - * `failedHistoryRestore`; every other reason reuses the single formatter so - * the two modules cannot drift apart. - */ -export function failedHistoryRestoreFromOutcome( - outcome: Extract, -): CodexRestoreHistoryResult { - if (outcome.kind === "blocked" && outcome.reason === "busy") return failedHistoryRestore("busy"); - if (outcome.kind === "failed" && outcome.historyFailureReason === "busy") { - return failedHistoryRestore( - "busy", - describeHistoryJobFailure(outcome, "restore"), - { rows: outcome.rows, files: outcome.files }, - ); - } - if (outcome.kind === "failed" && outcome.historyFailureReason === "permission") { - return failedHistoryRestore("permission", undefined, { rows: outcome.rows, files: outcome.files }); - } - if (outcome.kind === "failed" && outcome.historyFailureReason === "integrity") { - return failedHistoryRestore("integrity", undefined, { rows: outcome.rows, files: outcome.files }); - } - return failedHistoryRestore(undefined, describeHistoryJobFailure(outcome, "restore")); -} - -function externalProviderRestoreResult(activeProvider: string): CodexNativeRestoreResult { - const message = `External Codex provider ${tomlString(activeProvider)} preserved; no native restore was needed.`; - return { - success: true, - message, - externalProvider: activeProvider, - artifacts: { - config: { state: "skipped", changed: false, action: "external-provider-preserved", message }, - catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, - history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, - }, - }; -} - -/** A foreign service claim is an authority boundary, including explicit CLI restore. */ -function foreignOwnershipRestoreRefusal(message: string): CodexNativeRestoreResult { - return { - success: false, - message: `Codex native restore refused: ${message}`, - artifacts: { - config: { state: "skipped", changed: false, action: "failed", message }, - catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, - history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, - }, - }; -} - -function desiredEnabledRestoreSkip(): CodexNativeRestoreResult { - const message = "Codex integration was re-enabled; native restore was skipped."; - return skippedRestoreEnvelope(true, message); -} - -/** - * A schema-complete all-skipped envelope for outcomes decided before any - * restore machinery runs. Every `restore --json` path must stay shape-stable - * with `CodexNativeRestoreResult`; consumers never special-case early exits. - */ -export function skippedRestoreEnvelope(success: boolean, message: string): CodexNativeRestoreResult { - return { - success, - message, - artifacts: { - config: { state: "skipped", changed: false, action: "owned-fields-stripped", message }, - catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, - history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, - }, - }; -} - -/** Config was attempted and failed; downstream artifacts were never attempted. */ -function failedConfigRestoreEnvelope(config: CodexRestoreConfigResult): CodexNativeRestoreResult { - const result = skippedRestoreEnvelope(false, config.message); - result.artifacts.config = config; - return result; -} - -/** The config/profile half of a native restore, reported as one artifact. */ -function restoreCodexConfigInline(kind = "sync"): CodexRestoreConfigResult { - const preImages = captureCodexPreImages(); - const result = restoreCodexConfigInlineImpl(kind); - if (result.state === "failed") { - const compensated = restoreCodexPreImages(preImages); - if (!compensated.complete) throw new CodexPartialWriteError(compensated.unrestored); - } - return result; -} - -function restoreCodexConfigInlineImpl(kind: string): CodexRestoreConfigResult { - try { - beforeRestoreConfigForTests?.(kind); - const historyError = preflightCodexHistoryInjection(false, false); - if (historyError) return { state: "failed", changed: false, action: "failed", message: `Codex configuration and journal preserved: ${historyError}.` }; - const journal = restoreJournalState(); - if (journal.unverified) { - return { - state: "failed", changed: false, action: "failed", - message: "Codex journal recovery was not verified; current configuration files and the journal were preserved.", - }; - } - const restored = journal.configRestored - ? { success: true, message: "Codex config restored from opencodex journal." } - : removeCodexConfig({ preserveProfile: journal.profileRestored || journal.profileChanged }); - if (restored.success) { - // A successful journal/fallback write can race native history migration too. - // Refuse here while preimage compensation and the remove transaction can roll back. - const finalHistoryError = preflightCodexHistoryInjection(false, false); - if (finalHistoryError) return { state: "failed", changed: false, action: "failed", message: `Codex configuration and journal preserved: ${finalHistoryError}.` }; - } - return restored.success - ? { - state: "ok", - changed: journal.configRestored || journal.profileRestored || journal.profileChanged || restored.message.startsWith("Removed"), - action: journal.configRestored ? "journal-restored" : "owned-fields-stripped", - message: restored.message, - } - : { state: "failed", changed: false, action: "failed", message: restored.message }; - } catch (error) { - return { state: "failed", changed: false, action: "failed", message: error instanceof Error ? error.message : String(error) }; - } -} - -/** The catalog half, always inside its own K acquisition. */ -/** - * The catalog half, always inside its own K acquisition. - * - * `journaledCatalogPath` must be captured by the CALLER, before the config half runs: a - * successful journal restore deletes the journal, and a config restore can remove - * `model_catalog_json`. Reading it here would be too late in both cases (#1798). - */ -function restoreCodexCatalogArtifact( - revalidateDesiredState: boolean, - journaledCatalogPath: string | null, -): CodexRestoreCatalogResult { - const owningCodexHome = getCodexHome(); - try { - const restored = withCatalogWriteSerialization(owningCodexHome, permit => - revalidateDesiredState && shouldSyncCodexOnStart(loadConfig()) - ? null - : restoreCodexCatalogWithPermit(permit, owningCodexHome, journaledCatalogPath)); - return restored.kind === "completed" && restored.value !== null - ? { state: "ok", changed: restored.value.removed > 0, ...restored.value, message: "Codex catalog restored." } - : restored.kind === "completed" - ? { - state: "skipped", changed: false, removed: 0, kept: 0, path: null, - message: "Codex integration was re-enabled; native catalog restoration was skipped.", - } - : { - state: "failed", changed: false, removed: 0, kept: 0, path: DEFAULT_CATALOG_PATH, - message: `Codex catalog could not be restored: ${restored.reason}.`, - }; - } catch (error) { - return { - state: "failed", changed: false, removed: 0, kept: 0, path: DEFAULT_CATALOG_PATH, - message: error instanceof Error ? error.message : String(error), - }; - } -} - -/** - * Restore native Codex, running history in a Worker under H. - * - * On a coordinated home the config/profile restore happens INSIDE the Codex - * write lock, publishing a `remove` transition — the same serialization inject - * uses. Without it, an older restore could overwrite a config a concurrent - * enable had just written under the lock, and then honestly report success - * while desired intent said ON. The desired-state re-read under the lock turns - * that lost race into the discriminated `desired_enabled` skip. - */ -export async function restoreNativeCodexAsync( - options: { revalidateDesiredState?: boolean } = {}, -): Promise { - try { - return await restoreNativeCodexAsyncImpl(options); - } catch (error) { - if (!(error instanceof CodexRestoreRefusal)) throw error; - return failedConfigRestoreEnvelope(error.config); - } -} - -async function restoreNativeCodexAsyncImpl( - options: { revalidateDesiredState?: boolean }, -): Promise { - const activeProvider = currentExternalCodexModelProvider(); - if (activeProvider) { - // External-provider courtesy: only the stale journal is removed. The - // history worker must not launch — it would turn a read-mostly courtesy - // result into a history mutation on a home we do not own. - removeJournal(); - return externalProviderRestoreResult(activeProvider); - } - - // `restore` normally honours a human request even when an unrelated - // service-manager probe is unavailable. A recorded FOREIGN home is not an - // unrelated probe: it is positive evidence another installation owns these - // native artifacts, so do not create profile/claim locks before refusing. - if (options.revalidateDesiredState) { - const ownership = inspectNativeCodexOwnership(); - if (ownership.ownership === "foreign") return foreignOwnershipRestoreRefusal(ownership.reason); - if (shouldSyncCodexOnStart(loadConfig())) return desiredEnabledRestoreSkip(); - } - - const historyError = preflightCodexHistoryInjection(false, false); - if (historyError) return skippedRestoreEnvelope(false, `Native restore refused: ${historyError}. Config, catalog, history and provenance were preserved.`); - - const eligibility = codexWriteCoordinationEligibility({ - coordinatorPath: () => - resolveCodexCoordinatorDatabasePath(resolveEffectiveUserIdentity(), getCodexHome()), - residue: () => classifyNativeRoutedResidue(), - integrationRecord: () => readIntegrationRecord(), - }); - - // Captured before the config half: a successful journal restore DELETES the journal, and - // restoring the config can drop `model_catalog_json`. Either one would hide the routed - // catalog we actually wrote (#1798). - const journaledCatalogPath = journaledInjectedCatalogPath(); - let config: CodexRestoreConfigResult; - let transitionReceipt: { nativeGeneration: number; currentTxId: string } | undefined; - - if (eligibility.kind === "coordinated" || eligibility.kind === "adopt") { - // The restore has no candidate bytes to witness; freshness comes from the - // filesystem reads and the desired-state re-read performed under the lock. - const witness = { authoritySnapshotId: "codex-native-restore" }; - const coordinated = await withCodexWriteLock( - { - timeoutMs: DEFAULT_INJECT_LOCK_TIMEOUT_MS, - ...(eligibility.kind === "adopt" ? { adoption: { direction: "remove" as const } } : {}), - admitted: witness, - readAdmissionUnderLock: () => witness, - }, - (ctx) => { - if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { - throw new CodexWriteLockSkipped("desired_enabled"); - } - const published = ctx.coordinator.beginTransition( - { - nativeGeneration: ctx.expectation.nativeBefore, - currentTxId: ctx.currentTxId, - }, - { - txId: ctx.expectation.txId, - direction: "remove", - authoritySnapshotId: ctx.admission.authoritySnapshotId, - nextRetryAt: new Date().toISOString(), - }, - ); - if (published.kind !== "updated") { - throw new CodexWriteConflictError( - `The Codex transition could not be published: ${published.kind}.`, - ); - } - const preImages = captureCodexPreImages(); - let restored: CodexRestoreConfigResult; - try { - restored = restoreCodexConfigInline(eligibility.kind); - // Throw inside N so the published remove transition rolls back too. - if (restored.state === "failed") throw new CodexRestoreRefusal(restored); - } catch (error) { - const compensated = restoreCodexPreImages(preImages); - if (!compensated.complete) throw new CodexPartialWriteError(compensated.unrestored); - throw error; - } - return { - config: restored, - preImages, - receipt: { - nativeGeneration: ctx.expectation.nativeAfter, - currentTxId: ctx.expectation.txId, - }, - }; - }, - ); - if (coordinated.status === "skipped") return desiredEnabledRestoreSkip(); - if (coordinated.status !== "acquired") { - config = { - state: "failed", - changed: false, - action: "failed", - message: coordinated.status === "busy" - ? `Another process is writing Codex configuration right now (waited ${coordinated.waitedMs}ms). Retry shortly.` - : `Codex configuration was not restored: ${coordinated.message}`, - }; - } else { - recordCodexNativeTransactionProvenance( - coordinated.value.preImages, - coordinated.value.receipt.currentTxId, - ); - config = coordinated.value.config; - transitionReceipt = coordinated.value.receipt; - } - } else { - // Legacy-uncoordinated (or unresolvable) homes keep the unserialized path - // they have always had; restore is the escape hatch and must not strand - // them. The plain re-read still honors an intervening re-enable. - if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { - return desiredEnabledRestoreSkip(); - } - config = restoreCodexConfigInline(eligibility.kind); - } - - if (config.state === "failed") return failedConfigRestoreEnvelope(config); - const catalog = restoreCodexCatalogArtifact(options.revalidateDesiredState === true, journaledCatalogPath); - const outcome = await runCodexHistoryJob({ - ...resolveCodexHistoryJobTarget(), - ...(options.revalidateDesiredState ? { expectedDesiredEnabled: false } : {}), - operation: deriveCodexHistoryOperation({ direction: "restore", resumeHistory: true, legacyMode: false }), - }); - if (transitionReceipt) { - resolveCodexHistoryTransition(transitionReceipt, outcome); - } - const history: CodexRestoreHistoryResult = outcome.kind === "converged" - ? { - state: "ok", changed: outcome.rows > 0 || outcome.files > 0, rows: outcome.rows, files: outcome.files, ejectedRows: 0, - message: outcome.rows > 0 - ? `Resume history metadata restored from opencodex backup (${outcome.rows} thread(s)); original providers preserved.` - : "No backed-up resume-history metadata was pending; untracked routed history was left unchanged.", - } - : outcome.kind === "skipped" - ? { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message: "Codex resume history was skipped." } - : outcome.kind === "blocked" && (outcome.reason === "desired_disabled" || outcome.reason === "desired_enabled") - ? { - state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, - message: outcome.reason === "desired_disabled" - ? "Codex integration was disabled; history restoration was skipped." - : "Codex integration was enabled; history restoration was skipped.", - } - : outcome.kind === "blocked" || outcome.kind === "failed" - ? failedHistoryRestoreFromOutcome(outcome) - : failedHistoryRestore(); - const base = catalog.removed > 0 - ? `${config.message} Catalog restored to ${catalog.kept} native model(s) (dropped ${catalog.removed} proxy-routed).` - : config.message; - const success = catalog.state !== "failed" - && history.state !== "failed"; - return { - success, - message: `${base}${history.state === "failed" ? ` ⚠️ ${history.message}` : ""}`, - artifacts: { config, catalog, history }, - }; -} - -export function restoreNativeCodex(options: { skipHistory?: boolean; revalidateDesiredState?: boolean } = {}): CodexNativeRestoreResult { - const activeProvider = currentExternalCodexModelProvider(); - if (activeProvider) { - removeJournal(); - return externalProviderRestoreResult(activeProvider); - } - if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { - return desiredEnabledRestoreSkip(); - } - const historyError = preflightCodexHistoryInjection(false, false); - if (historyError) return skippedRestoreEnvelope(false, `Native restore refused: ${historyError}. Config, catalog, history and provenance were preserved.`); - // Captured before the config half: a successful journal restore DELETES the journal, and - // restoring the config can drop `model_catalog_json`. Either one would hide the routed - // catalog we actually wrote (#1798). - const journaledCatalogPath = journaledInjectedCatalogPath(); - const config = restoreCodexConfigInline(); - if (config.state === "failed") return failedConfigRestoreEnvelope(config); - const catalog = restoreCodexCatalogArtifact(options.revalidateDesiredState === true, journaledCatalogPath); - // Design B (loopback) steady state: threads are already tagged openai, so prove the - // no-op with a readonly probe instead of write-opening a DB the Codex app may hold - // (Windows: WAL writer lock -> seconds of stalling + a false warning on every stop). - // Legacy (non-loopback) installs keep the unconditional write-open restore. - let skipWhenProvablyNoop = false; - try { - skipWhenProvablyNoop = !shouldInjectApiAuthHeader(loadConfig()); - } catch { - /* unreadable config: keep the conservative write-open restore */ - } - // `skipHistory` is how the async wrapper takes this work for itself: the - // native files come down here, and history runs in the Worker under H. - const rawHistory = options.skipHistory - ? { rows: 0, files: 0 } - : syncCodexHistoryProvider("openai", undefined, undefined, { - skipWhenProvablyNoop, - }); - const history: CodexRestoreHistoryResult = options.skipHistory - ? { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message: "History restoration runs asynchronously." } - : rawHistory.failed - ? failedHistoryRestore(rawHistory.failureReason, undefined, rawHistory) - : { - state: "ok", - changed: rawHistory.rows > 0 || rawHistory.files > 0 || (rawHistory.ejectedRows ?? 0) > 0, - rows: rawHistory.rows, - files: rawHistory.files, - ejectedRows: rawHistory.ejectedRows ?? 0, - message: rawHistory.rows > 0 - ? `Resume history metadata restored from opencodex backup (${rawHistory.rows} thread(s)); original providers preserved.` - : "No backed-up resume-history metadata was pending; untracked routed history was left unchanged.", - }; - const message = catalog.removed > 0 - ? `${config.message} Catalog restored to ${catalog.kept} native model(s) (dropped ${catalog.removed} proxy-routed).` - : config.message; - return { - success: catalog.state !== "failed" && history.state !== "failed", - message, - artifacts: { config, catalog, history }, - }; -} - export function getCodexConfigPath(): string { return CODEX_CONFIG_PATH; } @@ -2340,3 +935,53 @@ export function formatApplyHistoryFailure(outcome: CodexHistoryJobOutcome, legac : "Codex resume history NOT changed"; return ` ⚠️ ${headline}: ${describeHistoryJobFailure(outcome, "apply", legacyMode)}\n`; } + +export { + providerBaseHost, + standaloneCodexRoutingTarget, +} from "./inject/routing-target"; +export type { CodexRoutingTarget } from "./inject/routing-target"; + +export { + applyEol, + buildOpenaiBaseUrlLine, + buildProfileFile, + buildProviderTableBlock, + buildRealtimeWsBaseUrlLine, + chooseCatalogPathForInjection, + currentExternalCodexModelProvider, + dominantEol, + externalCodexModelProvider, + setRootOpenaiBaseUrl, + setRootRealtimeWsBaseUrl, + stripInjectedOpenaiBaseUrl, + stripRootContextWindowOverrides, +} from "./inject/config-toml"; + +export { + classifyCodexRouting, + getCodexRoutingKind, + isCodexRoutingInjected, +} from "./inject/routing-classify"; +export type { CodexRoutingKind } from "./inject/routing-classify"; + +export { + removeCodexConfig, + stripOpencodexConfig, +} from "./inject/remove"; + +export type { + CodexNativeRestoreResult, + CodexRestoreArtifactState, + CodexRestoreCatalogResult, + CodexRestoreConfigResult, + CodexRestoreHistoryResult, +} from "./inject/restore"; +export { + failedHistoryRestoreFromOutcome, + restoreNativeCodex, + restoreNativeCodexAsync, + setBeforeRestoreConfigForTests, + skippedRestoreEnvelope, +} from "./inject/restore"; + diff --git a/src/codex/inject/config-toml.ts b/src/codex/inject/config-toml.ts new file mode 100644 index 0000000000..f8fd67b61f --- /dev/null +++ b/src/codex/inject/config-toml.ts @@ -0,0 +1,563 @@ +// Holds INV-TOML-01 from structure/overview.md; keep the id here if this file is split or renamed. +import { existsSync, readFileSync } from "node:fs"; +import { contextCompatibleBaseLine } from "../context-compat"; +import { resolveEffectiveProjectModelProvider } from "../project-config-warnings"; +import { + OCX_SECTION_MARKER, + REALTIME_WS_BASE_URL_KEY, + isRootOpenaiBaseUrlLine, + isRootRealtimeWsBaseUrlLine, + tomlStringPattern, +} from "../injected-marker"; +import { + CODEX_CONFIG_PATH, + DEFAULT_CATALOG_PATH, + parseTomlString, + resolveCodexConfigPath, + tomlString, +} from "../paths"; +import { + type CodexRoutingTarget, + providerBaseHost, + routingTargetOrigin, + usesProviderTable, + validateCodexRoutingTarget, +} from "./routing-target"; + +export function externalCodexModelProvider(content: string): string | null { + const provider = resolveEffectiveProjectModelProvider(content).provider; + return provider && provider !== "openai" && provider !== "opencodex" + ? provider + : null; +} + +export function currentExternalCodexModelProvider(): string | null { + if (!existsSync(CODEX_CONFIG_PATH)) return null; + return externalCodexModelProvider(readFileSync(CODEX_CONFIG_PATH, "utf8")); +} + +/** + * Detect the file's dominant line ending. Every transform in this module is LF-pure + * (split("\n") + hard "\n" joins), so CRLF configs (Windows-edited config.toml) are + * normalized to LF at the pipeline boundary and converted back on write — otherwise a + * single inject would leave a mixed-EOL file. + */ +export function dominantEol(content: string): "\r\n" | "\n" { + const crlf = (content.match(/\r\n/g) ?? []).length; + if (crlf === 0) return "\n"; + const bareLf = (content.match(/\n/g) ?? []).length - crlf; + return crlf >= bareLf ? "\r\n" : "\n"; +} + +/** Normalize all line endings to `eol` (CRLF first collapsed to LF, then expanded). */ +export function applyEol(content: string, eol: "\r\n" | "\n"): string { + const lf = content.replace(/\r\n/g, "\n"); + return eol === "\n" ? lf : lf.replace(/\n/g, "\r\n"); +} + +export function buildProviderTableBlock( + port: number, + supportsWebsockets?: boolean, + includeApiAuthHeader?: boolean, + hostname?: string, +): string; +export function buildProviderTableBlock( + target: CodexRoutingTarget, + supportsWebsockets?: boolean, +): string; +export function buildProviderTableBlock( + portOrTarget: number | CodexRoutingTarget, + supportsWebsockets = false, + includeApiAuthHeader = false, + hostname?: string, +): string { + const target = typeof portOrTarget === "number" + ? validateCodexRoutingTarget({ + baseUrl: `http://${providerBaseHost(hostname)}:${portOrTarget}/v1`, + requiresAdmissionToken: includeApiAuthHeader, + tokenEnv: "OPENCODEX_API_AUTH_TOKEN", + }) + : validateCodexRoutingTarget(portOrTarget); + return buildProviderTableBlockForTarget(target, supportsWebsockets); +} + +export function buildProviderTableBlockForTarget( + target: CodexRoutingTarget, + supportsWebsockets = false, +): string { + const lines = [ + "", + OCX_SECTION_MARKER, + "[model_providers.opencodex]", + 'name = "OpenCodex Proxy"', + `base_url = ${tomlString(target.baseUrl)}`, + 'wire_api = "responses"', + // false only in the authless Desktop opt-in (#1107); true keeps the App/TUI account gate. + `requires_openai_auth = ${target.desktopAuthless === true ? "false" : "true"}`, + ]; + if (target.requiresAdmissionToken) { + // codex-cli 0.146+ contract (#2073): env_key sends Authorization: Bearer $VAR and + // hard-errors on a missing/empty variable instead of silently omitting auth. It + // coexists with requires_openai_auth (env_key wins wire auth; the flag keeps the + // login/account UX), and the server substitutes stored main auth for our admission + // bearer (#1686), so the modern form is strictly better than the legacy + // env_http_headers table this line used to emit. + lines.push(`env_key = ${tomlString(target.tokenEnv)}`); + } + if (supportsWebsockets) lines.push("supports_websockets = true"); + return lines.join("\n") + "\n"; +} + +export function buildOpenaiBaseUrlLine( + port: number, + hostname?: string, +): string; +export function buildOpenaiBaseUrlLine(target: CodexRoutingTarget): string; +export function buildOpenaiBaseUrlLine( + portOrTarget: number | CodexRoutingTarget, + hostname?: string, +): string { + return typeof portOrTarget === "number" + ? `openai_base_url = "http://${providerBaseHost(hostname)}:${portOrTarget}/v1"` + : buildOpenaiBaseUrlLineForTarget(validateCodexRoutingTarget(portOrTarget)); +} + +function buildOpenaiBaseUrlLineForTarget(target: CodexRoutingTarget): string { + return `openai_base_url = ${tomlString(target.baseUrl)}`; +} + +/** + * Realtime sideband override (codex-rs `experimental_realtime_ws_base_url`), written with the + * SAME value as `openai_base_url`. Desktop voice creates its WebRTC call through the proxy + * (`POST /v1/live`, answered under the Pool account the proxy selects) but, since openai/codex + * 438c9e98d (#35830), joins the sideband at `wss://api.openai.com/v1/live/{callId}` with the + * app's own login unless this key redirects it. Two accounts, one call: the join 404s. Pointing + * the key at the proxy sends the join through `GET /v1/live/{callId}` (src/server/live.ts), + * where the same Pool account is reused. codex-rs turns `http` into `ws` and appends + * `/live/{callId}` itself; the value must stay the canonical `/v1` root. + */ +export function buildRealtimeWsBaseUrlLine(target: CodexRoutingTarget): string { + return `${REALTIME_WS_BASE_URL_KEY} = ${tomlString(target.baseUrl)}`; +} + +/** + * Design B root-key injection: place `OCX_SECTION_MARKER` + `openai_base_url` at the document + * ROOT (before the first table header). Idempotent: an existing marker-owned line is rewritten + * in place. A user's OWN root `openai_base_url` (no marker above it) is respected — we keep it + * and inject nothing, reporting `keptUserBaseUrl` so the caller can surface it. + */ +export function setRootOpenaiBaseUrl( + content: string, + port: number, + hostname?: string, +): { content: string; keptUserBaseUrl: boolean }; +export function setRootOpenaiBaseUrl( + content: string, + target: CodexRoutingTarget, +): { content: string; keptUserBaseUrl: boolean }; +export function setRootOpenaiBaseUrl( + content: string, + portOrTarget: number | CodexRoutingTarget, + hostname?: string, +): { content: string; keptUserBaseUrl: boolean } { + if (typeof portOrTarget !== "number") { + return setRootOpenaiBaseUrlForTarget(content, validateCodexRoutingTarget(portOrTarget)); + } + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const key = contextCompatibleBaseLine(content, buildOpenaiBaseUrlLine(portOrTarget, hostname)); + + for (let i = 0; i < rootEnd; i++) { + if (!isRootOpenaiBaseUrlLine(lines[i])) continue; + const markerOwned = i > 0 && lines[i - 1].includes(OCX_SECTION_MARKER); + if (!markerOwned) return { content, keptUserBaseUrl: true }; + lines[i] = key; + return { content: lines.join("\n"), keptUserBaseUrl: false }; + } + + if (firstTable === -1) { + return { + content: + content.replace(/\n+$/, "") + + "\n" + + OCX_SECTION_MARKER + + "\n" + + key + + "\n", + keptUserBaseUrl: false, + }; + } + let insertAt = firstTable; + while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; + lines.splice(insertAt, 0, OCX_SECTION_MARKER, key); + return { content: lines.join("\n"), keptUserBaseUrl: false }; +} + +export function setRootOpenaiBaseUrlForTarget( + content: string, + target: CodexRoutingTarget, +): { content: string; keptUserBaseUrl: boolean } { + const lines = content.split("\n"); + const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const key = contextCompatibleBaseLine(content, buildOpenaiBaseUrlLineForTarget(target)); + for (let index = 0; index < rootEnd; index += 1) { + if (!isRootOpenaiBaseUrlLine(lines[index])) continue; + const markerOwned = index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER); + if (!markerOwned) return { content, keptUserBaseUrl: true }; + lines[index] = key; + return { content: lines.join("\n"), keptUserBaseUrl: false }; + } + if (firstTable === -1) { + return { + content: `${content.replace(/\n+$/, "")}\n${OCX_SECTION_MARKER}\n${key}\n`, + keptUserBaseUrl: false, + }; + } + let insertAt = firstTable; + while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt -= 1; + lines.splice(insertAt, 0, OCX_SECTION_MARKER, key); + return { content: lines.join("\n"), keptUserBaseUrl: false }; +} + +/** + * Companion to `setRootOpenaiBaseUrlForTarget` for the realtime sideband override. Same + * ownership rule, applied per key: the line is ours only when the marker sits directly + * above it; a user's own line (no marker above it) is kept and nothing is injected. The + * key gets its OWN marker line rather than sharing the routing override's, so a user line + * that happens to sit right under our `openai_base_url` is never mistaken for ours. + * Placement: directly after the marker-owned `openai_base_url` pair. Only ever called on + * the Design B (loopback) path right after the routing override was written — the legacy + * provider-table form needs the admission-token header, which the sideband cannot carry. + */ +export function setRootRealtimeWsBaseUrl( + content: string, + target: CodexRoutingTarget, +): { content: string; keptUserRealtimeWsBaseUrl: boolean } { + const lines = content.split("\n"); + const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const key = buildRealtimeWsBaseUrlLine(validateCodexRoutingTarget(target)); + for (let index = 0; index < rootEnd; index += 1) { + if (!isRootRealtimeWsBaseUrlLine(lines[index])) continue; + const markerOwned = index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER); + if (!markerOwned) return { content, keptUserRealtimeWsBaseUrl: true }; + lines[index] = key; + return { content: lines.join("\n"), keptUserRealtimeWsBaseUrl: false }; + } + for (let index = 0; index < rootEnd; index += 1) { + if (!isRootOpenaiBaseUrlLine(lines[index])) continue; + if (!(index > 0 && lines[index - 1].includes(OCX_SECTION_MARKER))) continue; + lines.splice(index + 1, 0, OCX_SECTION_MARKER, key); + return { content: lines.join("\n"), keptUserRealtimeWsBaseUrl: false }; + } + // No marker-owned routing override to attach to: the override has no owner, so inject nothing. + return { content, keptUserRealtimeWsBaseUrl: false }; +} + +/** + * Remove the marker-owned root `openai_base_url` (marker line + the key line right after it). + * A user's own root override (no marker) survives; an orphaned marker with no key line after + * it is dropped too so repeated strip/inject cycles cannot accumulate marker comments. + * A marker-owned `experimental_realtime_ws_base_url` pair is removed by the same rule. + */ +export function stripInjectedOpenaiBaseUrl(content: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const drop = new Set(); + for (let i = 0; i < rootEnd; i++) { + if (!lines[i].includes(OCX_SECTION_MARKER)) continue; + if (i + 1 < rootEnd && (isRootOpenaiBaseUrlLine(lines[i + 1]) || isRootRealtimeWsBaseUrlLine(lines[i + 1]))) { + drop.add(i); + drop.add(i + 1); + } else if (i + 1 >= rootEnd || lines[i + 1].trim() === "") { + drop.add(i); // orphaned marker at root + } + } + if (drop.size === 0) return content; + return lines.filter((_, i) => !drop.has(i)).join("\n"); +} + +/** + * Strip every existing `model_provider` line that we must not duplicate: any line set to + * "opencodex" (wherever it sits — including a previously mis-nested one under a table), plus any + * ROOT-level model_provider (before the first table) of any value, since we override the global. + * A `model_provider` legitimately inside a user table/profile with a non-opencodex value is left + * untouched. + */ +export function stripExistingModelProvider(content: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + const out: string[] = []; + lines.forEach((line, i) => { + if (/^\s*model_provider\s*=/.test(line)) { + const isOurs = /^\s*model_provider\s*=\s*"opencodex"\s*$/.test(line); + const isRoot = firstTable === -1 || i < firstTable; + if (isOurs || isRoot) return; // drop it + } + out.push(line); + }); + return out.join("\n"); +} + +/** + * Drop ROOT-level `model_context_window` overrides (keys before the first table header). Codex + * treats this root key as a global override that wins over the per-model catalog values, so a stale + * `model_context_window = 1000000` makes every model (e.g. gpt-5.5) report a 1M window. User-owned + * compaction limits do not alter the advertised context window and must survive reinjection. + */ +export function stripRootContextWindowOverrides(content: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + return lines + .filter((line, i) => { + const isRoot = firstTable === -1 || i < firstTable; + return !isRoot || !/^\s*model_context_window\s*=/.test(line); + }) + .join("\n"); +} + +export function stripRootRoutedModel(content: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + return lines + .filter((line, i) => { + const isRoot = firstTable === -1 || i < firstTable; + if (!isRoot) return true; + const m = line.match(/^\s*model\s*=\s*("(?:\\.|[^"\\])*"|'[^']*')\s*$/); + if (!m) return true; + const model = parseTomlString(m[1]); + return !model?.includes("/"); + }) + .join("\n"); +} + +/** + * Insert `model_provider = "opencodex"` at the document ROOT — immediately before the first table + * header (TOML root keys must precede all tables). If there are no tables, append it to the root body. + */ +export function setRootModelProvider(content: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + const key = 'model_provider = "opencodex"'; + if (firstTable === -1) { + return content.replace(/\n+$/, "") + "\n" + key + "\n"; + } + let insertAt = firstTable; + while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; + lines.splice(insertAt, 0, key); + return lines.join("\n"); +} + +function readRootModelCatalogPath(content: string): string | null { + const lines = content.split("\n"); + const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); + let ownedCatalogPath: string | null = null; + for (let index = 0; index < rootEnd; index += 1) { + const match = modelCatalogAssignment.exec(lines[index]); + if (!match) continue; + const catalogPath = parseTomlString(match[1]); + if (!isOpencodexCatalogPath(catalogPath)) return catalogPath; + ownedCatalogPath ??= catalogPath; + } + return ownedCatalogPath; +} + +export function setRootModelCatalogPath(content: string, catalogPath: string): string { + const lines = content.split("\n"); + const firstTable = lines.findIndex((l) => /^\s*\[/.test(l)); + const key = `model_catalog_json = ${tomlString(catalogPath)}`; + const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + const ownedAssignments: number[] = []; + let hasUserAssignment = false; + for (let i = 0; i < rootEnd; i++) { + const m = modelCatalogAssignment.exec(lines[i]); + if (!m) continue; + const existing = parseTomlString(m[1]); + if (isOpencodexCatalogPath(existing)) { + ownedAssignments.push(i); + } else { + hasUserAssignment = true; + } + } + if (hasUserAssignment) { + const owned = new Set(ownedAssignments); + return lines.filter((_, index) => !owned.has(index)).join("\n"); + } + if (ownedAssignments.length > 0) { + lines[ownedAssignments[0]] = key; + const duplicates = new Set(ownedAssignments.slice(1)); + return lines.filter((_, index) => !duplicates.has(index)).join("\n"); + } + if (firstTable === -1) { + return content.replace(/\n+$/, "") + "\n" + key + "\n"; + } + let insertAt = firstTable; + while (insertAt > 0 && lines[insertAt - 1].trim() === "") insertAt--; + lines.splice(insertAt, 0, key); + return lines.join("\n"); +} + +export function removeProfileSection(content: string): string { + const lines = content.split("\n"); + const filtered: string[] = []; + let inProfile = false; + for (const line of lines) { + if (line.trim() === "[profiles.opencodex]") { + inProfile = true; + continue; + } + if (inProfile) { + if (/^\s*\[/.test(line) && line.trim() !== "[profiles.opencodex]") { + inProfile = false; + filtered.push(line); + } + continue; + } + filtered.push(line); + } + return ( + filtered + .join("\n") + .replace(/\n{3,}/g, "\n\n") + .trimEnd() + "\n" + ); +} + +export function normalizeServiceTier(content: string): string { + return content.replace( + /^(\s*service_tier\s*=\s*)["']priority["']\s*$/gm, + '$1"fast"', + ); +} + +export function ensureFastModeFeature(content: string, fastMode?: boolean): string { + // Tri-state fast mode (see OcxConfig.fastMode): true forces `fast_mode = true`, + // false forces `fast_mode = false`, and undefined leaves the user's config + // untouched (no [features] table is added and an existing fast_mode line is + // preserved as-is). Table and key matching accept the valid TOML spellings + // `[features] # comment`, `["features"]` / `['features']`, and quoted keys. + const lines = content.split("\n"); + const featuresHeader = /^\s*\[(["']?)\s*features\s*\1\]\s*(?:#.*)?$/; + const fastModeKey = /^\s*(?:"fast_mode"|'fast_mode'|fast_mode)\s*=/; + const featuresStart = lines.findIndex(line => featuresHeader.test(line)); + if (featuresStart === -1) { + if (fastMode === undefined) return content; + return content.trimEnd() + "\n\n[features]\nfast_mode = " + (fastMode ? "true" : "false") + "\n"; + } + + const nextTable = lines.findIndex( + (line, index) => index > featuresStart && /^\s*\[/.test(line), + ); + const featuresEnd = nextTable === -1 ? lines.length : nextTable; + for (let i = featuresStart + 1; i < featuresEnd; i++) { + if (fastModeKey.test(lines[i])) { + if (fastMode === undefined) return lines.join("\n"); + lines[i] = lines[i].replace(/^(\s*)(?:"fast_mode"|'fast_mode'|fast_mode)\s*=.*$/, `$1fast_mode = ${fastMode ? "true" : "false"}`); + return lines.join("\n"); + } + } + + if (fastMode === undefined) return lines.join("\n"); + let insertAt = featuresEnd; + while (insertAt > featuresStart + 1 && lines[insertAt - 1].trim() === "") insertAt--; + lines.splice(insertAt, 0, `fast_mode = ${fastMode ? "true" : "false"}`); + return lines.join("\n"); +} + +function isOpencodexCatalogPath(path: string): boolean { + return path.replace(/\\/g, "/").split("/").pop() === "opencodex-catalog.json"; +} + +export function stripOpencodexCatalogPath(content: string): string { + const modelCatalogAssignment = tomlStringPattern("model_catalog_json"); + const lines = content.split("\n"); + const firstTable = lines.findIndex((line) => /^\s*\[/.test(line)); + const rootEnd = firstTable === -1 ? lines.length : firstTable; + return lines + .filter((line, index) => { + if (index >= rootEnd) return true; + const m = modelCatalogAssignment.exec(line); + return !m || !isOpencodexCatalogPath(parseTomlString(m[1])); + }) + .join("\n"); +} + +export function buildProfileFile(port: number, catalogPath?: string | null, supportsWebsockets?: boolean, includeApiAuthHeader?: boolean, hostname?: string, fastMode?: boolean): string; +export function buildProfileFile(target: CodexRoutingTarget, catalogPath?: string | null, supportsWebsockets?: boolean, fastMode?: boolean): string; +export function buildProfileFile( + portOrTarget: number | CodexRoutingTarget, + catalogPath?: string | null, + supportsWebsockets = false, + includeApiAuthHeaderOrFastMode?: boolean, + hostname?: string, + fastMode?: boolean, +): string { + const target = typeof portOrTarget === "number" + ? validateCodexRoutingTarget({ + baseUrl: `http://${providerBaseHost(hostname)}:${portOrTarget}/v1`, + requiresAdmissionToken: includeApiAuthHeaderOrFastMode === true, + tokenEnv: "OPENCODEX_API_AUTH_TOKEN", + }) + : validateCodexRoutingTarget(portOrTarget); + return buildProfileFileForTarget( + target, + catalogPath, + supportsWebsockets, + typeof portOrTarget === "number" ? fastMode : includeApiAuthHeaderOrFastMode, + ); +} + +export function buildProfileFileForTarget( + target: CodexRoutingTarget, + catalogPath?: string | null, + supportsWebsockets = false, + fastMode?: boolean, +): string { + const origin = routingTargetOrigin(target); + const host = new URL(origin).host; + // Design B (loopback): the reference/fallback file documents the root override form. + // Non-loopback keeps the legacy provider-table shape (built-in provider cannot carry + // the x-opencodex-api-key env header); explicit Desktop policies share that shape. + if (!usesProviderTable(target)) { + const lines = [ + "# OpenCodex proxy fallback config (Design B)", + `# Root override that points Codex's built-in openai provider at the proxy on ${host}.`, + "# Merge these root keys into ~/.codex/config.toml manually if auto-injection was removed.", + buildOpenaiBaseUrlLineForTarget(target), + ]; + if (catalogPath) lines.push(`model_catalog_json = ${tomlString(catalogPath)}`); + if (fastMode !== undefined) lines.push("", "[features]", `fast_mode = ${fastMode ? "true" : "false"}`, ""); + return lines.join("\n"); + } + const lines = [ + "# OpenCodex proxy profile — use with: codex --profile opencodex", + `# Routes all model requests through the opencodex proxy at ${host}`, + 'model_provider = "opencodex"', + ]; + if (catalogPath) lines.push(`model_catalog_json = ${tomlString(catalogPath)}`); + if (fastMode !== undefined) lines.push("", "[features]", `fast_mode = ${fastMode ? "true" : "false"}`); + lines.push(buildProviderTableBlockForTarget(target, supportsWebsockets).trimEnd(), ""); + return lines.join("\n"); +} + +export function chooseCatalogPathForInjection( + content: string, + requested?: string | null, +): string | null { + if (requested !== undefined) return requested; + + const existing = readRootModelCatalogPath(content); + if (existing) { + const resolved = resolveCodexConfigPath(existing); + if (!isOpencodexCatalogPath(resolved) || existsSync(resolved)) + return existing; + } + + return existsSync(DEFAULT_CATALOG_PATH) ? DEFAULT_CATALOG_PATH : null; +} diff --git a/src/codex/inject/remove.ts b/src/codex/inject/remove.ts new file mode 100644 index 0000000000..fb56b71a44 --- /dev/null +++ b/src/codex/inject/remove.ts @@ -0,0 +1,192 @@ +import { existsSync, readFileSync, unlinkSync } from "node:fs"; +import { atomicWriteFile } from "../../config"; +import { + OCX_SECTION_MARKER, + REALTIME_WS_BASE_URL_KEY, + hasInjectedOpenaiBaseUrl, + rootTomlString, + stripJournaledOpenaiBaseUrl, +} from "../injected-marker"; +import { preflightCodexHistoryInjection } from "../history-provider"; +import { + journaledInjectedOpenaiBaseUrl, + journaledInjectedRealtimeWsBaseUrl, +} from "../journal"; +import { CODEX_CONFIG_PATH, CODEX_PROFILE_PATH, readRootTomlString } from "../paths"; +import { transformManagedSubagentDefaults } from "../subagent-defaults"; +import { + applyEol, + dominantEol, + removeProfileSection, + stripInjectedOpenaiBaseUrl, + stripOpencodexCatalogPath, + stripRootRoutedModel, +} from "./config-toml"; + +/** + * Sub-table headers like `[model_providers.opencodex.env_http_headers]` appear when a Codex app + * config rewrite re-serializes the provider's inline `env_http_headers` table. They define the + * same `model_providers.opencodex` provider, so cleanup must remove them too — otherwise the + * provider survives with no `name` and Codex rejects the whole config + * ("provider name must not be empty"). The dot terminator keeps a user's + * `[model_providers.opencodex_backup]`-style tables out of scope. + */ +function isOcxProviderHeaderLine(trimmedLine: string): boolean { + // Root form matched by regex, not equality: TOML v1.0 allows a trailing comment + // (`[model_providers.opencodex] # comment`), and an exact compare would miss that form. + // The sub-table prefix check already tolerates trailing comments by construction. + return ( + /^\[model_providers\.opencodex\]\s*(?:#.*)?$/.test(trimmedLine) || + trimmedLine.startsWith("[model_providers.opencodex.") + ); +} + +export function hasOcxProviderTable(content: string): boolean { + return content + .split("\n") + .some((line) => isOcxProviderHeaderLine(line.trim())); +} + +export function removeOcxSection(content: string): string { + const lines = content.split("\n"); + const filtered: string[] = []; + let inOcxSection = false; + for (const line of lines) { + if ( + line.includes(OCX_SECTION_MARKER) || + isOcxProviderHeaderLine(line.trim()) + ) { + inOcxSection = true; + continue; + } + if (inOcxSection) { + // End the injected section at the next table header that ISN'T our own. Exact match on the + // provider name (plus our own sub-tables) so a user's + // "[model_providers.opencodex_backup]" (or similar) is preserved, not swallowed. + if (/^\s*\[/.test(line) && !isOcxProviderHeaderLine(line.trim())) { + inOcxSection = false; + filtered.push(line); + } + continue; + } + filtered.push(line); + } + return ( + filtered + .join("\n") + .replace(/\n{3,}/g, "\n\n") + .trimEnd() + "\n" + ); +} + +interface StripOpencodexConfigResult { + content: string; + managedDefaultsError: string | null; +} + +/** + * Detailed form used by the on-disk restore path. A damaged ownership marker is + * ambiguous: keep the associated value, but return the transform error so the + * caller cannot report a complete restore. + */ +function stripOpencodexConfigResult( + content: string, + journaledBaseUrl: string | null = null, + journaledRealtimeWsBaseUrl: string | null = null, +): StripOpencodexConfigResult { + let out = content; + const hadRootOcxProvider = + readRootTomlString(out, "model_provider") === "opencodex"; + // #1798: marker adjacency is FORMATTING evidence, and a Codex app rewrite keeps values + // while dropping comments. Fall back to VALUE evidence -- the exact URL we recorded + // writing -- so an app-rewritten config is still recognized as ours. + const hadInjectedBaseUrl = hasInjectedOpenaiBaseUrl(out) + || (journaledBaseUrl !== null && rootTomlString(out, "openai_base_url") === journaledBaseUrl); + out = stripInjectedOpenaiBaseUrl(out); // before removeOcxSection — it keys on the marker line too + out = stripJournaledOpenaiBaseUrl(out, journaledBaseUrl, journaledRealtimeWsBaseUrl); + if (hasOcxProviderTable(out)) { + out = removeOcxSection(out); + } + out = removeProfileSection(out); + // Regex (not exact-string) removal so compact `model_provider="opencodex"` is stripped too — + // must match the detection regex above, or a detected line could survive un-removed. + out = out + .split("\n") + .filter((l) => !/^\s*model_provider\s*=\s*"opencodex"\s*$/.test(l)) + .join("\n"); + // Routed root model ids (`model = "provider/slug"`) only make sense while the proxy serves + // them — strip on both the legacy re-tag form and the Design B injected-base-url form. + if (hadRootOcxProvider || hadInjectedBaseUrl) out = stripRootRoutedModel(out); + const managedDefaults = transformManagedSubagentDefaults(out, null); + if (managedDefaults.ok) out = managedDefaults.content; + out = stripOpencodexCatalogPath(out); + return { + content: out.replace(/\n{3,}/g, "\n\n").trimEnd() + "\n", + managedDefaultsError: !managedDefaults.ok ? managedDefaults.error : null, + }; +} + +/** Pure transform: strip the opencodex provider block + `model_provider = "opencodex"` lines. */ +export function stripOpencodexConfig(content: string): string { + return stripOpencodexConfigResult(content).content; +} + +function hasOpencodexRouting(content: string): boolean { + return ( + hasOcxProviderTable(content) || + /^\s*model_provider\s*=\s*"opencodex"/m.test(content) || + hasInjectedOpenaiBaseUrl(content) + ); +} + +export function removeCodexConfig( + options: { preserveProfile?: boolean } = {}, +): { success: boolean; message: string } { + const historyError = preflightCodexHistoryInjection(false, false); + if (historyError) return { success: false, message: `Codex configuration preserved: ${historyError}. Native writer coordination is required.` }; + if (!existsSync(CODEX_CONFIG_PATH)) { + if (!options.preserveProfile && existsSync(CODEX_PROFILE_PATH)) + unlinkSync(CODEX_PROFILE_PATH); + return { + success: true, + message: `Codex config not found; no native restore was needed${options.preserveProfile ? "." : ", and the opencodex profile was removed if present."}`, + }; + } + const rawContent = readFileSync(CODEX_CONFIG_PATH, "utf-8"); + // Same EOL boundary as inject: strip in LF space, write back in the file's own ending. + // The unchanged fast path compares in LF space so an untouched file is never rewritten. + const eol = dominantEol(rawContent); + const content = applyEol(rawContent, "\n"); + // Read the recorded injection once: the strip below consumes it, and so does the + // ownership verdict, which must agree with what was actually removed. + const journaledBaseUrl = journaledInjectedOpenaiBaseUrl(); + const journaledRealtimeWsBaseUrl = journaledInjectedRealtimeWsBaseUrl(); + const had = hasOpencodexRouting(content) + || (journaledBaseUrl !== null && rootTomlString(content, "openai_base_url") === journaledBaseUrl) + || (journaledRealtimeWsBaseUrl !== null + && rootTomlString(content, REALTIME_WS_BASE_URL_KEY) === journaledRealtimeWsBaseUrl); + const stripped = stripOpencodexConfigResult(content, journaledBaseUrl, journaledRealtimeWsBaseUrl); + if (had || stripped.content !== content) { + atomicWriteFile(CODEX_CONFIG_PATH, applyEol(stripped.content, eol)); + } + if (!options.preserveProfile && existsSync(CODEX_PROFILE_PATH)) + unlinkSync(CODEX_PROFILE_PATH); + const removedMessage = had + ? `Removed opencodex routing from Codex config${options.preserveProfile ? "." : " + profile."}` + : "opencodex not present in Codex config."; + if (stripped.managedDefaultsError) { + const routingMessage = had + ? removedMessage + : "No opencodex routing was present in Codex config."; + return { + success: false, + message: + `${routingMessage} Native Codex sub-agent defaults could not be safely removed: ${stripped.managedDefaultsError}. ` + + "The ambiguous marker and adjacent value were preserved; inspect $CODEX_HOME/config.toml before using native Codex.", + }; + } + return { + success: true, + message: removedMessage, + }; +} diff --git a/src/codex/inject/restore.ts b/src/codex/inject/restore.ts new file mode 100644 index 0000000000..15282ea771 --- /dev/null +++ b/src/codex/inject/restore.ts @@ -0,0 +1,540 @@ +import { loadConfig } from "../../config"; +import { shouldSyncCodexOnStart } from "../desired-state"; +import { withCatalogWriteSerialization } from "../catalog-write-serialization"; +import { restoreCodexCatalogWithPermit } from "../catalog/sync"; +import { withCodexWriteLock, CodexWriteLockSkipped } from "../codex-write-lock"; +import { inspectNativeCodexOwnership } from "../../integrations/native/ownership-preflight"; +import { resolveCodexHistoryTransition } from "../history-transition"; +import { + captureCodexPreImages, + codexWriteCoordinationEligibility, + CodexPartialWriteError, + CodexWriteConflictError, + DEFAULT_INJECT_LOCK_TIMEOUT_MS, + recordCodexNativeTransactionProvenance, + restoreCodexPreImages, +} from "../inject-coordination"; +import { readIntegrationRecord } from "../integration-record"; +import { classifyNativeRoutedResidue } from "../native-residue"; +import { + resolveCodexCoordinatorDatabasePath, + resolveEffectiveUserIdentity, +} from "../user-identity"; +import { + journaledInjectedCatalogPath, + removeJournal, + restoreJournalState, +} from "../journal"; +import { + preflightCodexHistoryInjection, + syncCodexHistoryProvider, + type CodexHistoryFailureReason, +} from "../history-provider"; +import { + describeHistoryJobFailure, + deriveCodexHistoryOperation, + resolveCodexHistoryJobTarget, + runCodexHistoryJob, + type CodexHistoryJobOutcome, +} from "../history-job"; +import { + DEFAULT_CATALOG_PATH, + getCodexHome, + tomlString, +} from "../paths"; +import { shouldInjectApiAuthHeader } from "../loopback-target"; +import { currentExternalCodexModelProvider } from "./config-toml"; +import { removeCodexConfig } from "./remove"; + +class CodexRestoreRefusal extends Error { + constructor(readonly config: CodexRestoreConfigResult) { + super(config.message); + } +} + +let beforeRestoreConfigForTests: ((kind: string) => void) | undefined; +export function setBeforeRestoreConfigForTests(hook: typeof beforeRestoreConfigForTests): void { + beforeRestoreConfigForTests = hook; +} + +export type CodexRestoreArtifactState = "ok" | "skipped" | "failed"; + +export interface CodexRestoreConfigResult { + state: CodexRestoreArtifactState; + changed: boolean; + action: "journal-restored" | "owned-fields-stripped" | "external-provider-preserved" | "failed"; + message: string; +} + +export interface CodexRestoreCatalogResult { + state: CodexRestoreArtifactState; + changed: boolean; + removed: number; + kept: number; + path: string | null; + message: string; +} + +export interface CodexRestoreHistoryResult { + state: CodexRestoreArtifactState; + changed: boolean; + reason?: CodexHistoryFailureReason; + rows: number; + files: number; + ejectedRows: number; + message: string; +} + +export interface CodexNativeRestoreResult { + success: boolean; + message: string; + externalProvider?: string; + artifacts: { + config: CodexRestoreConfigResult; + catalog: CodexRestoreCatalogResult; + history: CodexRestoreHistoryResult; + }; +} + +function failedHistoryRestore( + reason?: CodexHistoryFailureReason, + detail?: string, + progress: { rows?: number; files?: number } = {}, +): CodexRestoreHistoryResult { + const rows = progress.rows ?? 0; + const files = progress.files ?? 0; + const changed = rows > 0 || files > 0; + return { + state: "failed", + changed, + ...(reason ? { reason } : {}), + rows, + files, + ejectedRows: 0, + message: reason === "permission" + ? changed + ? "Codex resume history changed but did NOT converge because permission was denied while finalizing the backup manifest; the manifest was retained for review and safe retry." + : "Codex resume history could NOT be restored because permission was denied." + : reason === "busy" + ? changed + ? "Codex resume history changed but did NOT converge because backup-manifest finalization remained busy; the manifest was retained for review and safe retry." + : detail ?? "Codex resume history could NOT be restored — the Codex app appears to be holding the history database." + : reason === "integrity" + ? changed + ? "Codex resume history changed but did NOT converge because the backup or target changed; the manifest was retained for review and safe retry." + : "Codex resume history could NOT be restored because the backup or restore target failed integrity checks; unverified provider metadata was left unchanged." + : detail + ? `Codex resume history could NOT be restored: ${detail}` + : "Codex resume history could NOT be restored; the reason was not recorded. Run 'ocx doctor'.", + }; +} + +/** + * Restore failure wording for a Worker outcome. + * + * Only a genuine busy result blames the Codex app. An unsafe-path refusal, an + * unavailable coordinator database, a permission denial, or a dead/timed-out + * worker is a different problem; the old collapse made every one of those read + * as "the Codex app is holding the database" (issue #1191). `busy` and + * `permission` keep the restore-specific sentence built by + * `failedHistoryRestore`; every other reason reuses the single formatter so + * the two modules cannot drift apart. + */ +export function failedHistoryRestoreFromOutcome( + outcome: Extract, +): CodexRestoreHistoryResult { + if (outcome.kind === "blocked" && outcome.reason === "busy") return failedHistoryRestore("busy"); + if (outcome.kind === "failed" && outcome.historyFailureReason === "busy") { + return failedHistoryRestore( + "busy", + describeHistoryJobFailure(outcome, "restore"), + { rows: outcome.rows, files: outcome.files }, + ); + } + if (outcome.kind === "failed" && outcome.historyFailureReason === "permission") { + return failedHistoryRestore("permission", undefined, { rows: outcome.rows, files: outcome.files }); + } + if (outcome.kind === "failed" && outcome.historyFailureReason === "integrity") { + return failedHistoryRestore("integrity", undefined, { rows: outcome.rows, files: outcome.files }); + } + return failedHistoryRestore(undefined, describeHistoryJobFailure(outcome, "restore")); +} + +function externalProviderRestoreResult(activeProvider: string): CodexNativeRestoreResult { + const message = `External Codex provider ${tomlString(activeProvider)} preserved; no native restore was needed.`; + return { + success: true, + message, + externalProvider: activeProvider, + artifacts: { + config: { state: "skipped", changed: false, action: "external-provider-preserved", message }, + catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, + history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, + }, + }; +} + +/** A foreign service claim is an authority boundary, including explicit CLI restore. */ +function foreignOwnershipRestoreRefusal(message: string): CodexNativeRestoreResult { + return { + success: false, + message: `Codex native restore refused: ${message}`, + artifacts: { + config: { state: "skipped", changed: false, action: "failed", message }, + catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, + history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, + }, + }; +} + +function desiredEnabledRestoreSkip(): CodexNativeRestoreResult { + const message = "Codex integration was re-enabled; native restore was skipped."; + return skippedRestoreEnvelope(true, message); +} + +/** + * A schema-complete all-skipped envelope for outcomes decided before any + * restore machinery runs. Every `restore --json` path must stay shape-stable + * with `CodexNativeRestoreResult`; consumers never special-case early exits. + */ +export function skippedRestoreEnvelope(success: boolean, message: string): CodexNativeRestoreResult { + return { + success, + message, + artifacts: { + config: { state: "skipped", changed: false, action: "owned-fields-stripped", message }, + catalog: { state: "skipped", changed: false, removed: 0, kept: 0, path: null, message }, + history: { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message }, + }, + }; +} + +/** Config was attempted and failed; downstream artifacts were never attempted. */ +function failedConfigRestoreEnvelope(config: CodexRestoreConfigResult): CodexNativeRestoreResult { + const result = skippedRestoreEnvelope(false, config.message); + result.artifacts.config = config; + return result; +} + +/** The config/profile half of a native restore, reported as one artifact. */ +function restoreCodexConfigInline(kind = "sync"): CodexRestoreConfigResult { + const preImages = captureCodexPreImages(); + const result = restoreCodexConfigInlineImpl(kind); + if (result.state === "failed") { + const compensated = restoreCodexPreImages(preImages); + if (!compensated.complete) throw new CodexPartialWriteError(compensated.unrestored); + } + return result; +} + +function restoreCodexConfigInlineImpl(kind: string): CodexRestoreConfigResult { + try { + beforeRestoreConfigForTests?.(kind); + const historyError = preflightCodexHistoryInjection(false, false); + if (historyError) return { state: "failed", changed: false, action: "failed", message: `Codex configuration and journal preserved: ${historyError}.` }; + const journal = restoreJournalState(); + if (journal.unverified) { + return { + state: "failed", changed: false, action: "failed", + message: "Codex journal recovery was not verified; current configuration files and the journal were preserved.", + }; + } + const restored = journal.configRestored + ? { success: true, message: "Codex config restored from opencodex journal." } + : removeCodexConfig({ preserveProfile: journal.profileRestored || journal.profileChanged }); + if (restored.success) { + // A successful journal/fallback write can race native history migration too. + // Refuse here while preimage compensation and the remove transaction can roll back. + const finalHistoryError = preflightCodexHistoryInjection(false, false); + if (finalHistoryError) return { state: "failed", changed: false, action: "failed", message: `Codex configuration and journal preserved: ${finalHistoryError}.` }; + } + return restored.success + ? { + state: "ok", + changed: journal.configRestored || journal.profileRestored || journal.profileChanged || restored.message.startsWith("Removed"), + action: journal.configRestored ? "journal-restored" : "owned-fields-stripped", + message: restored.message, + } + : { state: "failed", changed: false, action: "failed", message: restored.message }; + } catch (error) { + return { state: "failed", changed: false, action: "failed", message: error instanceof Error ? error.message : String(error) }; + } +} + +/** The catalog half, always inside its own K acquisition. */ +/** + * The catalog half, always inside its own K acquisition. + * + * `journaledCatalogPath` must be captured by the CALLER, before the config half runs: a + * successful journal restore deletes the journal, and a config restore can remove + * `model_catalog_json`. Reading it here would be too late in both cases (#1798). + */ +function restoreCodexCatalogArtifact( + revalidateDesiredState: boolean, + journaledCatalogPath: string | null, +): CodexRestoreCatalogResult { + const owningCodexHome = getCodexHome(); + try { + const restored = withCatalogWriteSerialization(owningCodexHome, permit => + revalidateDesiredState && shouldSyncCodexOnStart(loadConfig()) + ? null + : restoreCodexCatalogWithPermit(permit, owningCodexHome, journaledCatalogPath)); + return restored.kind === "completed" && restored.value !== null + ? { state: "ok", changed: restored.value.removed > 0, ...restored.value, message: "Codex catalog restored." } + : restored.kind === "completed" + ? { + state: "skipped", changed: false, removed: 0, kept: 0, path: null, + message: "Codex integration was re-enabled; native catalog restoration was skipped.", + } + : { + state: "failed", changed: false, removed: 0, kept: 0, path: DEFAULT_CATALOG_PATH, + message: `Codex catalog could not be restored: ${restored.reason}.`, + }; + } catch (error) { + return { + state: "failed", changed: false, removed: 0, kept: 0, path: DEFAULT_CATALOG_PATH, + message: error instanceof Error ? error.message : String(error), + }; + } +} + +/** + * Restore native Codex, running history in a Worker under H. + * + * On a coordinated home the config/profile restore happens INSIDE the Codex + * write lock, publishing a `remove` transition — the same serialization inject + * uses. Without it, an older restore could overwrite a config a concurrent + * enable had just written under the lock, and then honestly report success + * while desired intent said ON. The desired-state re-read under the lock turns + * that lost race into the discriminated `desired_enabled` skip. + */ +export async function restoreNativeCodexAsync( + options: { revalidateDesiredState?: boolean } = {}, +): Promise { + try { + return await restoreNativeCodexAsyncImpl(options); + } catch (error) { + if (!(error instanceof CodexRestoreRefusal)) throw error; + return failedConfigRestoreEnvelope(error.config); + } +} + +async function restoreNativeCodexAsyncImpl( + options: { revalidateDesiredState?: boolean }, +): Promise { + const activeProvider = currentExternalCodexModelProvider(); + if (activeProvider) { + // External-provider courtesy: only the stale journal is removed. The + // history worker must not launch — it would turn a read-mostly courtesy + // result into a history mutation on a home we do not own. + removeJournal(); + return externalProviderRestoreResult(activeProvider); + } + + // `restore` normally honours a human request even when an unrelated + // service-manager probe is unavailable. A recorded FOREIGN home is not an + // unrelated probe: it is positive evidence another installation owns these + // native artifacts, so do not create profile/claim locks before refusing. + if (options.revalidateDesiredState) { + const ownership = inspectNativeCodexOwnership(); + if (ownership.ownership === "foreign") return foreignOwnershipRestoreRefusal(ownership.reason); + if (shouldSyncCodexOnStart(loadConfig())) return desiredEnabledRestoreSkip(); + } + + const historyError = preflightCodexHistoryInjection(false, false); + if (historyError) return skippedRestoreEnvelope(false, `Native restore refused: ${historyError}. Config, catalog, history and provenance were preserved.`); + + const eligibility = codexWriteCoordinationEligibility({ + coordinatorPath: () => + resolveCodexCoordinatorDatabasePath(resolveEffectiveUserIdentity(), getCodexHome()), + residue: () => classifyNativeRoutedResidue(), + integrationRecord: () => readIntegrationRecord(), + }); + + // Captured before the config half: a successful journal restore DELETES the journal, and + // restoring the config can drop `model_catalog_json`. Either one would hide the routed + // catalog we actually wrote (#1798). + const journaledCatalogPath = journaledInjectedCatalogPath(); + let config: CodexRestoreConfigResult; + let transitionReceipt: { nativeGeneration: number; currentTxId: string } | undefined; + + if (eligibility.kind === "coordinated" || eligibility.kind === "adopt") { + // The restore has no candidate bytes to witness; freshness comes from the + // filesystem reads and the desired-state re-read performed under the lock. + const witness = { authoritySnapshotId: "codex-native-restore" }; + const coordinated = await withCodexWriteLock( + { + timeoutMs: DEFAULT_INJECT_LOCK_TIMEOUT_MS, + ...(eligibility.kind === "adopt" ? { adoption: { direction: "remove" as const } } : {}), + admitted: witness, + readAdmissionUnderLock: () => witness, + }, + (ctx) => { + if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { + throw new CodexWriteLockSkipped("desired_enabled"); + } + const published = ctx.coordinator.beginTransition( + { + nativeGeneration: ctx.expectation.nativeBefore, + currentTxId: ctx.currentTxId, + }, + { + txId: ctx.expectation.txId, + direction: "remove", + authoritySnapshotId: ctx.admission.authoritySnapshotId, + nextRetryAt: new Date().toISOString(), + }, + ); + if (published.kind !== "updated") { + throw new CodexWriteConflictError( + `The Codex transition could not be published: ${published.kind}.`, + ); + } + const preImages = captureCodexPreImages(); + let restored: CodexRestoreConfigResult; + try { + restored = restoreCodexConfigInline(eligibility.kind); + // Throw inside N so the published remove transition rolls back too. + if (restored.state === "failed") throw new CodexRestoreRefusal(restored); + } catch (error) { + const compensated = restoreCodexPreImages(preImages); + if (!compensated.complete) throw new CodexPartialWriteError(compensated.unrestored); + throw error; + } + return { + config: restored, + preImages, + receipt: { + nativeGeneration: ctx.expectation.nativeAfter, + currentTxId: ctx.expectation.txId, + }, + }; + }, + ); + if (coordinated.status === "skipped") return desiredEnabledRestoreSkip(); + if (coordinated.status !== "acquired") { + config = { + state: "failed", + changed: false, + action: "failed", + message: coordinated.status === "busy" + ? `Another process is writing Codex configuration right now (waited ${coordinated.waitedMs}ms). Retry shortly.` + : `Codex configuration was not restored: ${coordinated.message}`, + }; + } else { + recordCodexNativeTransactionProvenance( + coordinated.value.preImages, + coordinated.value.receipt.currentTxId, + ); + config = coordinated.value.config; + transitionReceipt = coordinated.value.receipt; + } + } else { + // Legacy-uncoordinated (or unresolvable) homes keep the unserialized path + // they have always had; restore is the escape hatch and must not strand + // them. The plain re-read still honors an intervening re-enable. + if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { + return desiredEnabledRestoreSkip(); + } + config = restoreCodexConfigInline(eligibility.kind); + } + + if (config.state === "failed") return failedConfigRestoreEnvelope(config); + const catalog = restoreCodexCatalogArtifact(options.revalidateDesiredState === true, journaledCatalogPath); + const outcome = await runCodexHistoryJob({ + ...resolveCodexHistoryJobTarget(), + ...(options.revalidateDesiredState ? { expectedDesiredEnabled: false } : {}), + operation: deriveCodexHistoryOperation({ direction: "restore", resumeHistory: true, legacyMode: false }), + }); + if (transitionReceipt) { + resolveCodexHistoryTransition(transitionReceipt, outcome); + } + const history: CodexRestoreHistoryResult = outcome.kind === "converged" + ? { + state: "ok", changed: outcome.rows > 0 || outcome.files > 0, rows: outcome.rows, files: outcome.files, ejectedRows: 0, + message: outcome.rows > 0 + ? `Resume history metadata restored from opencodex backup (${outcome.rows} thread(s)); original providers preserved.` + : "No backed-up resume-history metadata was pending; untracked routed history was left unchanged.", + } + : outcome.kind === "skipped" + ? { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message: "Codex resume history was skipped." } + : outcome.kind === "blocked" && (outcome.reason === "desired_disabled" || outcome.reason === "desired_enabled") + ? { + state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, + message: outcome.reason === "desired_disabled" + ? "Codex integration was disabled; history restoration was skipped." + : "Codex integration was enabled; history restoration was skipped.", + } + : outcome.kind === "blocked" || outcome.kind === "failed" + ? failedHistoryRestoreFromOutcome(outcome) + : failedHistoryRestore(); + const base = catalog.removed > 0 + ? `${config.message} Catalog restored to ${catalog.kept} native model(s) (dropped ${catalog.removed} proxy-routed).` + : config.message; + const success = catalog.state !== "failed" + && history.state !== "failed"; + return { + success, + message: `${base}${history.state === "failed" ? ` ⚠️ ${history.message}` : ""}`, + artifacts: { config, catalog, history }, + }; +} + +export function restoreNativeCodex(options: { skipHistory?: boolean; revalidateDesiredState?: boolean } = {}): CodexNativeRestoreResult { + const activeProvider = currentExternalCodexModelProvider(); + if (activeProvider) { + removeJournal(); + return externalProviderRestoreResult(activeProvider); + } + if (options.revalidateDesiredState && shouldSyncCodexOnStart(loadConfig())) { + return desiredEnabledRestoreSkip(); + } + const historyError = preflightCodexHistoryInjection(false, false); + if (historyError) return skippedRestoreEnvelope(false, `Native restore refused: ${historyError}. Config, catalog, history and provenance were preserved.`); + // Captured before the config half: a successful journal restore DELETES the journal, and + // restoring the config can drop `model_catalog_json`. Either one would hide the routed + // catalog we actually wrote (#1798). + const journaledCatalogPath = journaledInjectedCatalogPath(); + const config = restoreCodexConfigInline(); + if (config.state === "failed") return failedConfigRestoreEnvelope(config); + const catalog = restoreCodexCatalogArtifact(options.revalidateDesiredState === true, journaledCatalogPath); + // Design B (loopback) steady state: threads are already tagged openai, so prove the + // no-op with a readonly probe instead of write-opening a DB the Codex app may hold + // (Windows: WAL writer lock -> seconds of stalling + a false warning on every stop). + // Legacy (non-loopback) installs keep the unconditional write-open restore. + let skipWhenProvablyNoop = false; + try { + skipWhenProvablyNoop = !shouldInjectApiAuthHeader(loadConfig()); + } catch { + /* unreadable config: keep the conservative write-open restore */ + } + // `skipHistory` is how the async wrapper takes this work for itself: the + // native files come down here, and history runs in the Worker under H. + const rawHistory = options.skipHistory + ? { rows: 0, files: 0 } + : syncCodexHistoryProvider("openai", undefined, undefined, { + skipWhenProvablyNoop, + }); + const history: CodexRestoreHistoryResult = options.skipHistory + ? { state: "skipped", changed: false, rows: 0, files: 0, ejectedRows: 0, message: "History restoration runs asynchronously." } + : rawHistory.failed + ? failedHistoryRestore(rawHistory.failureReason, undefined, rawHistory) + : { + state: "ok", + changed: rawHistory.rows > 0 || rawHistory.files > 0 || (rawHistory.ejectedRows ?? 0) > 0, + rows: rawHistory.rows, + files: rawHistory.files, + ejectedRows: rawHistory.ejectedRows ?? 0, + message: rawHistory.rows > 0 + ? `Resume history metadata restored from opencodex backup (${rawHistory.rows} thread(s)); original providers preserved.` + : "No backed-up resume-history metadata was pending; untracked routed history was left unchanged.", + }; + const message = catalog.removed > 0 + ? `${config.message} Catalog restored to ${catalog.kept} native model(s) (dropped ${catalog.removed} proxy-routed).` + : config.message; + return { + success: catalog.state !== "failed" && history.state !== "failed", + message, + artifacts: { config, catalog, history }, + }; +} diff --git a/src/codex/inject/routing-classify.ts b/src/codex/inject/routing-classify.ts new file mode 100644 index 0000000000..abb7a92d52 --- /dev/null +++ b/src/codex/inject/routing-classify.ts @@ -0,0 +1,109 @@ +import { existsSync, readFileSync } from "node:fs"; +import { + hasInjectedCodexRouting, + hasInjectedOpenaiBaseUrl, + providerTableStart, + providerTableString, + rootTomlString, +} from "../injected-marker"; +import { CODEX_CONFIG_PATH } from "../paths"; + +export type CodexRoutingKind = + "native" | "opencodex-local" | "custom-local" | "custom-remote" | "unknown"; + +type RoutingEndpointKind = "local" | "remote" | "unknown"; + +function ipv4Octets(hostname: string): number[] | null { + const dotted = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(hostname); + if (dotted) { + const octets = dotted.slice(1).map(Number); + return octets.some((octet) => octet > 255) ? null : octets; + } + const mapped = /^::ffff:([0-9a-f]{1,4}):([0-9a-f]{1,4})$/i.exec(hostname); + if (!mapped) return null; + const high = Number.parseInt(mapped[1], 16); + const low = Number.parseInt(mapped[2], 16); + return [high >>> 8, high & 0xff, low >>> 8, low & 0xff]; +} + +function classifyRoutingEndpoint(value: string): RoutingEndpointKind { + try { + const url = new URL(value); + if (url.protocol !== "http:" && url.protocol !== "https:") return "unknown"; + const hostname = url.hostname + .toLowerCase() + .replace(/^\[|\]$/g, "") + .replace(/\.$/, ""); + if (!hostname) return "unknown"; + if (hostname === "localhost" || hostname.endsWith(".localhost")) + return "local"; + if (hostname === "::" || hostname === "::1" || hostname === "0.0.0.0") + return "local"; + const octets = ipv4Octets(hostname); + if (octets) { + if (octets.every((octet) => octet === 0)) return "local"; + if (octets[0] === 127) return "local"; + return "remote"; + } + if (/^::ffff:/i.test(hostname)) return "unknown"; + return "remote"; + } catch { + return "unknown"; + } +} + +/** Classify actual routing dependency separately from opencodex ownership. */ +export function classifyCodexRouting(content: string): CodexRoutingKind { + const rootBaseUrl = rootTomlString(content, "openai_base_url"); + if (rootBaseUrl) { + const endpoint = classifyRoutingEndpoint(rootBaseUrl); + if (endpoint === "unknown") return "unknown"; + if (hasInjectedOpenaiBaseUrl(content)) return "opencodex-local"; + return endpoint === "local" ? "custom-local" : "custom-remote"; + } + const rootProvider = rootTomlString(content, "model_provider"); + if (rootProvider) { + const providerTableExists = + providerTableStart(content.split("\n"), rootProvider) !== -1; + const providerBaseUrl = providerTableString( + content, + rootProvider, + "base_url", + ); + if (providerBaseUrl) { + const endpoint = classifyRoutingEndpoint(providerBaseUrl); + if (endpoint === "unknown") return "unknown"; + if (rootProvider === "opencodex") return "opencodex-local"; + return endpoint === "local" ? "custom-local" : "custom-remote"; + } + if ( + rootProvider === "opencodex" || + providerTableExists || + rootProvider !== "openai" + ) + return "unknown"; + } + return "native"; +} + +/** Read-only probe used by status, doctor, and the dashboard. */ +export function isCodexRoutingInjected(): boolean { + const path = CODEX_CONFIG_PATH; + if (!existsSync(path)) return false; + try { + return hasInjectedCodexRouting(readFileSync(path, "utf8")); + } catch { + return false; + } +} + +export function getCodexRoutingKind(): CodexRoutingKind { + const path = CODEX_CONFIG_PATH; + if (!existsSync(path)) return "native"; + try { + return classifyCodexRouting(readFileSync(path, "utf8")); + } catch { + return "unknown"; + } +} + diff --git a/src/codex/inject/routing-target.ts b/src/codex/inject/routing-target.ts new file mode 100644 index 0000000000..67a4bf6322 --- /dev/null +++ b/src/codex/inject/routing-target.ts @@ -0,0 +1,125 @@ +import { subagentDefaultSyncEffective } from "../../config"; +import { + effectiveLoopbackListenerPort, + isLoopbackHostname, + shouldInjectApiAuthHeader, +} from "../loopback-target"; +import type { ManagedSubagentDefaults } from "../subagent-defaults"; +import type { OcxConfig } from "../../types"; + +export interface CodexRoutingTarget { + baseUrl: string; + requiresAdmissionToken: boolean; + tokenEnv: "OPENCODEX_API_AUTH_TOKEN"; + /** + * Opt-in authless Codex Desktop mode (#1107): inject the dedicated provider table with + * `requires_openai_auth = false` so Desktop skips the ChatGPT login gate. Only ever true for + * loopback targets that need no admission token; non-loopback admission is a separate layer + * and is never weakened by this flag. + */ + desktopAuthless?: boolean; + /** Select the dedicated provider identity so Codex owns compaction locally. */ + clientCompaction?: boolean; +} + +export function validateCodexRoutingTarget(target: CodexRoutingTarget): CodexRoutingTarget { + let parsed: URL; + try { + parsed = new URL(target.baseUrl); + } catch { + throw new TypeError("Codex routing target must be an absolute HTTP(S) /v1 URL"); + } + if ( + (parsed.protocol !== "http:" && parsed.protocol !== "https:") + || parsed.username + || parsed.password + || parsed.pathname !== "/v1" + || parsed.search + || parsed.hash + || target.tokenEnv !== "OPENCODEX_API_AUTH_TOKEN" + ) { + throw new TypeError("Codex routing target must be a canonical HTTP(S) /v1 URL without credentials, query, or fragment"); + } + return { ...target, baseUrl: `${parsed.origin}/v1` }; +} + +/** Provider-table form is used when auth, admission, or compaction policy needs a dedicated provider. */ +export function usesProviderTable(target: CodexRoutingTarget): boolean { + return target.requiresAdmissionToken + || target.desktopAuthless === true + || target.clientCompaction === true; +} + +export function standaloneCodexRoutingTarget( + port: number, + config?: Pick< + OcxConfig, + "hostname" | "unauthenticatedLoopbackListener" | "codexDesktopAuthless" | "codexClientCompaction" + >, +): CodexRoutingTarget { + // An enabled listener with no `port` is the companion form: it answers on `port` itself, + // bound to 127.0.0.1 (#4236). Resolving it through the shared helper is what makes the + // one-port hub work without every writer repeating `?? port`. + const loopback = config?.unauthenticatedLoopbackListener; + const effectivePort = effectiveLoopbackListenerPort(config, port) ?? port; + const hostname = loopback?.enabled ? undefined : config?.hostname; + const requiresAdmissionToken = loopback?.enabled ? false : shouldInjectApiAuthHeader(config); + return { + baseUrl: `http://${providerBaseHost(hostname)}:${effectivePort}/v1`, + requiresAdmissionToken, + tokenEnv: "OPENCODEX_API_AUTH_TOKEN", + ...(config?.codexDesktopAuthless === true && !requiresAdmissionToken + ? { desktopAuthless: true } + : {}), + ...(config?.codexClientCompaction === true && !requiresAdmissionToken + ? { clientCompaction: true } + : {}), + }; +} + +export function routingTargetOrigin(target: CodexRoutingTarget): string { + return target.baseUrl.slice(0, -3); +} + +export function configuredManagedSubagentDefaults( + config: + | Pick< + OcxConfig, + "injectionModel" | "injectionEffort" | "syncCodexSubagentDefaults" + > + | undefined, +): ManagedSubagentDefaults | null { + if (!subagentDefaultSyncEffective(config ?? {})) return null; + return { + model: config!.injectionModel!.trim(), + ...(config!.injectionEffort?.trim() + ? { reasoningEffort: config!.injectionEffort.trim() } + : {}), + }; +} + +/** + * The `[model_providers.opencodex]` TABLE only. A table is position-independent in TOML, so it is + * safe to append at EOF. The bare root key `model_provider = "opencodex"` is NOT included here — + * it must live at the document root (before any table header) and is set separately by + * setRootModelProvider(). Appending the bare key at EOF was the original bug: it nested under + * whatever `[table]` happened to be open last (e.g. `[plugins."chrome@openai-bundled"]`), so Codex + * never saw a global model_provider and silently fell back to the `openai` (ChatGPT) provider. + */ +export function providerBaseHost(hostname: string | undefined): string { + const trimmed = (hostname ?? "127.0.0.1").trim(); + const lower = trimmed.toLowerCase(); + // Match what the server actually binds. Writing "localhost" while binding IPv4-only + // 127.0.0.1 breaks on Windows, where localhost commonly resolves to ::1 first. + if (lower === "::1" || lower === "[::1]") return "[::1]"; + if ( + isLoopbackHostname(trimmed) || + trimmed === "0.0.0.0" || + trimmed === "::" || + trimmed === "[::]" + ) + return "127.0.0.1"; + if (trimmed.startsWith("[") && trimmed.endsWith("]")) return trimmed; + return trimmed.includes(":") ? `[${trimmed}]` : trimmed; +} + diff --git a/src/codex/routing.ts b/src/codex/routing.ts index 433089cd21..0a9a0b9c3c 100644 --- a/src/codex/routing.ts +++ b/src/codex/routing.ts @@ -1,428 +1,198 @@ -import { randomUUID } from "node:crypto"; import { saveConfigPreservingClaudeCode } from "../config"; -import { isCodexAccountGenerationLive, readCodexAccountRecord, type CodexRefreshProvenance } from "./account-store"; +import { isCodexAccountGenerationLive } from "./account-store"; import { codexAccountLogLabel } from "./account-label"; -import { NATIVE_RESERVE_MODEL } from "./catalog/native-models"; import { isCodexAccountPaused } from "./account-pause"; -import { clearCodexAccountPin, codexAccountPriorityLookup, pinnedCodexAccountId } from "./account-priority"; +import { clearCodexAccountPin, pinnedCodexAccountId } from "./account-priority"; import { isCodexAccountUsable, type CodexAccountUsabilityOptions } from "./account-usability"; -import { clearAccountNeedsReauth, isAccountNeedsReauth, markAccountNeedsReauth } from "./account-runtime-state"; -import { - POOL_KEY_CODEX, - normalizeAccountPoolStickyLimit, - normalizeCodexAccountPoolStrategy, - notePoolRotationFailure, - notePoolRotationSuccess, - peekRoundRobinAccount, - pickRoundRobinAccount, - seedPoolRotationAccount, - selectPriorityTier, -} from "./pool-rotation"; -import { - CODEX_EXHAUSTED_USAGE_PERCENT, - CODEX_UNKNOWN_USAGE_SCORE, - getAccountQuota, - isRetiredCodexSparkModel, - resetAtToMs, -} from "./quota"; -import { codexPlanKey, isThirtyDayOnlyCodexPlan } from "./plan"; -import { - MAIN_CODEX_ACCOUNT_ID, - getMainAccountPlan, - hasMainAccountRefreshGrant, -} from "./main-account"; +import { isAccountNeedsReauth, markAccountNeedsReauth } from "./account-runtime-state"; +import { POOL_KEY_CODEX, notePoolRotationFailure } from "./pool-rotation"; +import { getAccountQuota, isRetiredCodexSparkModel } from "./quota"; +import { MAIN_CODEX_ACCOUNT_ID } from "./main-account"; import { isSelectableCodexPoolAccount } from "./account-id"; import type { OcxConfig } from "../types"; import { captureConfigGeneration, type GenerationContext } from "../lib/state-store-sweeper"; -import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers"; -import { retainedUtf8Bytes } from "../lib/admission"; import { recordUpstreamHostFailure } from "./upstream-host-health"; import type { CodexThreadLineage } from "./lineage"; -import { clearAllCodexPoolRefreshFailures, isCodexPoolRefreshCooling } from "./pool-refresh-backoff"; - -type ThreadAffinityEntry = { - accountId: string; - generation: number; - createdAt: number; - lastUsedAt: number; - // Last time the bound account's quota threshold was re-evaluated for this - // thread (interval-gated to avoid per-request flapping). See REEVAL_INTERVAL_MS. - lastReevalAt: number; - // When a transient failure streak first forced this thread onto another account - // while the binding was HELD (#4546). Cleared the moment the bound account serves - // again; once it ages past CODEX_TRANSIENT_AFFINITY_HOLD_MS the binding is - // released through the ordinary path instead of detouring forever. - transientHoldSince?: number; - // Which account is serving this thread while its own is held under a transient hold. - // Remembered rather than re-picked per request: under round-robin a fresh pick each turn - // would walk the ring and start cold on every hop, which is the behaviour the hold exists - // to prevent. Cleared with transientHoldSince when the bound account serves again. - transientDetourAccountId?: string; -}; - -export type CodexThreadResolution = - | { status: "selected"; accountId: string; affinity?: CodexAffinityDecision } - | { status: "none"; affinity?: CodexAffinityDecision } - | { status: "expired"; accountId: string; affinity?: CodexAffinityDecision }; - -/** What happened to this thread's binding on this request (#4546). */ -export type CodexAffinityMove = - /** Served by its own bound account, which was healthy. */ - | "reused" - /** Served by its own bound account while something transient was wrong with it. */ - | "held" - /** Served by another account while the binding stayed put. */ - | "detour" - /** The binding was released and a different account took the thread. */ - | "rebound" - /** There was no live binding; this request established one. */ - | "new_bind" - /** The binding was released without a replacement on this request. */ - | "cleared"; - -/** - * Why. A move is the expensive event -- it discards the prompt-cache prefix warmed on the old - * account -- so the operator should not have to infer it from account labels across log lines, - * which is how #4546 had to be diagnosed. - */ -export type CodexAffinityReason = - | "healthy" - | "quota_headroom" - | "quota_refusal" - | "transient" - | "transient_hold_expired" - | "unusable" - | "paused" - | "plan_excluded" - | "cooldown" - | "quota_avoided" - | "generation" - | "expired" - | "model_lane" - /** First placement followed the parent's CURRENT serving account (#4546, wp8). */ - | "lineage_parent" - /** First placement followed a compatible sibling's current serving account. */ - | "lineage_sibling"; - -export interface CodexAffinityDecision { - move: CodexAffinityMove; - reason: CodexAffinityReason; -} - -/** The decision to report once a binding has been released and selection starts over. */ -function affinityAfterRelease( - threadId: string | null, - releaseReason: CodexAffinityReason | undefined, -): CodexAffinityDecision { - // Reported now, so it must not be reported again by the next request. - clearPendingReleaseReason(threadId); - return releaseReason === undefined - ? { move: "new_bind", reason: "healthy" } - : { move: "rebound", reason: releaseReason }; -} - -/** - * What to report when selection produced no account at all. The binding is gone and nothing took - * it, which is a `cleared`, and the pending reason is deliberately NOT consumed: a no-account - * result reaches no auth context and therefore no usage entry, so the next resolve that does - * produce one is the first place this release can actually be seen. - */ -function affinityOnNoAccount( - threadId: string | null, - releaseReason: CodexAffinityReason | undefined, -): CodexAffinityDecision | undefined { - if (releaseReason === undefined) return undefined; - // Hand it forward as well as reporting it. A reason derived from the entry this request just - // released lives only in a local, so without this the next resolve finds no entry and no - // pending reason and calls the rebind a fresh healthy bind. - notePendingReleaseReason(threadId, releaseReason); - return { move: "cleared", reason: releaseReason }; -} - -/** - * Process-local cursor for automatic RR/fill-first (and quota-429 when not - * sync-writing) picks. Keeps unrelated `saveConfig` from persisting transient - * rotation as the operator's `activeCodexAccountId`. Manual selection clears it - * so disk/`config.activeCodexAccountId` remains authoritative. - */ -let runtimeActiveCodexAccountId: string | undefined; - -type CodexUpstreamHealth = { - consecutiveFailures: number; - /** Consecutive healthy terminals observed while recovering from escalation level 2+. */ - consecutiveSuccesses?: number; - lastFailureStatus?: number; - lastFailureAt?: number; - /** Hard cooldown (quota 429). Survives a later 2xx; blocks auth + selection. */ - cooldownUntil?: number; - /** - * How long a quota refusal keeps selection away from this account (or this native quota - * group), as opposed to how long it is hard-blocked. - * - * The two are deliberately different lengths. {@link CODEX_MAX_RESET_DERIVED_COOLDOWN_MS} - * caps the hard cooldown at 15 minutes because a reset announcement is advisory and plan - * quota usually frees up before it — an account must stay reachable so the pool can find - * that out (#433). The window the refusal announced is not 15 minutes, though, so once the - * cooldown lapses the account is selectable again while its burst window is still spent, - * and the strategy picks it straight back: this proxy reads a weekly bar a burst limit never - * touches, so a refused account still scores as the coolest in the pool. Every request then - * earns the same 429 until the process restarts, which is the only thing that drops this map. - * - * So the announcement governs avoidance and the cap still governs blocking. Avoidance is soft - * in the {@link softAvoidUntil} sense: it reorders the pool and releases a bound thread, and - * the last-resort paths still reach the account when nothing else can serve, so one pessimistic - * announcement cannot stall routing. - */ - quotaAvoidUntil?: number; - /** When the current cooldown was recorded; origin of the probe interval clock. */ - cooldownSince?: number; - /** - * What produced the cooldown. An explicit Retry-After is a literal retry - * directive and is never probed; a quota resetAt only announces a window - * refresh, so it may be probed early (#433). - */ - cooldownSource?: CodexCooldownSource; - /** - * Bumped on every cooldown write. A probe lease records the generation it was - * issued for so a lease cannot clear a cooldown that a later 429 replaced. - */ - cooldownGeneration?: number; - /** - * Identity of the in-flight probe. A cooled-down account sends no traffic, so - * no organic 2xx can prove recovery; only the outcome carrying this id may - * clear the cooldown. - */ - probeLeaseId?: string; - /** Cooldown generation at the moment the lease was granted. */ - probeLeaseGeneration?: number; - /** Last probe grant or conclusion; paces the probe interval. */ - lastProbeAt?: number; - /** - * Soft avoid after connect_error / timeout / transient 5xx. Cleared on 2xx. - * Blocks pool selection + thread affinity reuse so a sticky session can leave a - * flaky account without throwing CodexAccountCooldownError (hard-only). - */ - softAvoidUntil?: number; - /** - * Credential generation a 401/403 quarantine was derived from (#2892 gap 4). - * - * Provenance lives ON the entry rather than in a side map keyed by account id. A side map spends - * "whatever health is current when the old credential is found dead", which deletes a later - * unrelated entry: a G1 401, then a G2 save, then a genuine G2 503 would lose the 503. Only the - * entry that carries this field can be spent, and any later write simply replaces it. - */ - credentialFailureGeneration?: number; -}; - -const CODEX_DEFAULT_QUOTA_COOLDOWN_MS = 60_000; -const CODEX_MAX_QUOTA_COOLDOWN_MS = 24 * 60 * 60_000; -/** - * A weekly/monthly quota `resetAt` announces when the window refreshes; it is not - * a "come back after this" directive like Retry-After. Plan quota routinely frees - * up long before the advertised reset, so cap reset-derived cooldowns far below - * the Retry-After ceiling (#433). - */ -const CODEX_MAX_RESET_DERIVED_COOLDOWN_MS = 15 * 60_000; -/** - * Ceiling on quota-refusal avoidance. Generous enough to cover a full five-hour burst window, - * tight enough that a weekly or monthly reset four days out cannot take an account out of - * rotation for the {@link CODEX_MAX_QUOTA_COOLDOWN_MS} day the Retry-After ceiling allows. - */ -const CODEX_MAX_QUOTA_AVOID_MS = 6 * 60 * 60_000; -/** Minimum gap between probe leases for one cooled-down account. */ -export const CODEX_QUOTA_PROBE_INTERVAL_MS = 5 * 60_000; -export const CODEX_FAILURE_WINDOW_MS = 5 * 60_000; -/** - * How recently a 100% burst reading must have been OBSERVED to exclude an account when it - * carries no reset timestamp (#3425). Deliberately far tighter than the 6h disk-hydration - * horizon in `quota.ts`: shorter than any plausible five-hour burst window, so a persisted - * reading can never strand a recovered account, and long enough that a snapshot taken at - * admission is still fresh when selection reads it. - */ -export const TERMINAL_SHORT_WINDOW_FRESHNESS_MS = 5 * 60_000; -/** How long a transient failure keeps the account out of pool selection. */ -export const CODEX_TRANSIENT_SOFT_AVOID_MS = 30_000; -const CODEX_TRANSIENT_SOFT_AVOID_ESCALATION_MS = [ +import { isCodexPoolRefreshCooling } from "./pool-refresh-backoff"; +import { + classifyCodexUpstreamOutcome, + computeCodexUsageScore, + computeQuotaCooldown, + quotaAvoidUntilFor, + CODEX_FAILURE_WINDOW_MS, + CODEX_TRANSIENT_SOFT_AVOID_ESCALATION_MS, + type CodexUpstreamOutcome, + type CodexUpstreamOutcomeMeta, +} from "./routing/cooldown-math"; +import { + codexPoolKeyForScope, + codexQuotaScopeForModel, + deleteAccountHealth, + deleteAllScopedHealth, + deleteScopedHealth, + dropSpentCredentialFailure, + getAccountHealth, + getCodexAccountCooldownUntil, + getCodexAccountSoftAvoidUntil, + getCodexQuotaHealthSnapshot, + isCodexAccountSoftAvoided, + isCodexQuotaAvoided, + isHealthAccountAdmissible, + isHealthGenerationReconciled, + isIndependentCodexQuotaScope, + preservedCooldownFields, + pruneHealthAccountsForContext, + commitHealthReconcile, + clearUpstreamHealthState, + resetHealthReconcileState, + deleteAllHealthForAccount, + scopedHealthFor, + setAccountHealth, + setScopedHealth, + type CodexQuotaScope, + type CodexUpstreamHealth, +} from "./routing/health-store"; +import { ownsProbeLease, probeMayClearCooldown, withProbeLeaseReleased } from "./routing/probe-lease"; +import { + adoptLegacyLineageAffinity, + affinityAfterRelease, + affinityOnNoAccount, + bindModelDetourAffinity, + bindThreadAffinity, + clearThreadAccountMapForAccount, + deleteModelDetourAffinity, + deleteThreadAffinity, + deleteThreadAffinitiesForAccount, + getThreadAffinity, + getThreadAffinityScopes, + getModelDetourAffinity, + isThreadAffinityExpired, + isThreadAffinityGenerationLive, + peekPendingReleaseReason, + CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS, + CODEX_TRANSIENT_AFFINITY_HOLD_MS, + type CodexAffinityReason, + type CodexThreadResolution, + type ThreadAffinityEntry, +} from "./routing/thread-affinity"; +import { + accountPoolStrategyForScope, + applyFailureFailover, + applyQuotaAutoSwitch, + codexAccountBlockReason, + getEligiblePoolAccounts, + getPoolAccountPlanForSelection, + hasCodexQuotaHeadroom, + isCodexAccountPlanExcluded, + isCacheAffinityEnabled, + isCodexAccountSelectable, + isHealthySharedCodexSelection, + isUnknownUsage, + pickAlternateCodexAccount, + pickLowerUsageAccount, + pickLowestUsageAmong, + pickLowestUsageCodexAccount, + pickPriorityPreemption, + pickResetFirstCodexAccount, + pickUnboundStrategyAccount, + sharedStateSelectionOptions, + strategySelectionOptionsForModelDetour, + shouldFailover, + peekAlternateCodexAccount, +} from "./routing/selection"; +import { + clearAllManualPreferences, + consumeManualPreference, + forgetManualPreference, + forgetRoutingPreferencesOutside, + forgetRuntimeActiveCodexAccount, + getEffectiveActiveCodexAccountId, + manualPreferenceBlocks, + promoteActiveCodexAccount, + rememberActiveCodexAccount, + setActiveCodexAccount, +} from "./routing/active-account"; + +export { + CODEX_QUOTA_PROBE_INTERVAL_MS, + CODEX_FAILURE_WINDOW_MS, + TERMINAL_SHORT_WINDOW_FRESHNESS_MS, CODEX_TRANSIENT_SOFT_AVOID_MS, - 2 * 60_000, - 10 * 60_000, - 30 * 60_000, -] as const; -export const CODEX_THREAD_AFFINITY_IDLE_TTL_MS = 24 * 60 * 60_000; -export const CODEX_THREAD_AFFINITY_MAX_ENTRIES = 2048; -const MAX_AFFINITY_COMPONENT_BYTES = 512; -// Min interval between quota threshold re-evaluations for a single bound thread. -// Well under the 5h/weekly quota windows, but enough to stop per-request flapping. -export const CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS = 60_000; - -/** - * How long a live binding outlives a TRANSIENT failure streak on its own account (#4546). - * - * Being unable to send right now is not the same as losing ownership of the conversation. - * A 5xx streak is frequently provider-wide rather than account-specific, and deleting the - * binding for it discards a prompt-cache prefix that the next turn then pays for again -- - * the same cost the quota threshold used to impose, arriving through a different door. - * So the request detours to another account while the binding is held here. - * - * Bounded, because an unbounded hold is its own defect: an account that never recovers - * would keep a thread detouring indefinitely while the conversation's real warm prefix - * accumulates somewhere else. Ten minutes is longer than the whole soft-avoid escalation - * ladder up to its final step, so an ordinary outage resolves inside the hold and a - * genuine one converts to a real rebind instead of a permanent detour. - */ -export const CODEX_TRANSIENT_AFFINITY_HOLD_MS = 10 * 60_000; - -const upstreamHealth = new Map(); -/** - * Reset-derived 429s can describe a quota owned by one native model family, - * rather than the whole ChatGPT account. Keep those advisory cooldowns apart - * from account-wide Retry-After/default throttles and transient health. - */ -const quotaScopedHealth = new Map>(); -/** - * Spend a credential-failure health entry whose credential no longer exists (#2892 gap 4). - * - * A 401/403 describes one CREDENTIAL, not an account, and a replacement can land at any point after - * the outcome is recorded — so re-reading the store inside `recordCodexUpstreamOutcome` narrows the - * window without closing it. The reader decides instead, and it may only spend an entry that - * actually carries credential provenance: a later transient or quota write replaces the entry and - * with it the tag, so this can never delete evidence that belongs to a different failure. - */ -function dropSpentCredentialFailure(accountId: string): void { - const health = upstreamHealth.get(accountId); - const generation = health?.credentialFailureGeneration; - if (health === undefined || generation === undefined) return; - if (isCodexAccountGenerationLive(accountId, generation)) return; - upstreamHealth.delete(accountId); -} -let lastReconciledGeneration = 0; -let liveHealthAccountIds = new Set(); - -export type CodexUpstreamOutcome = number | "connect_error" | "timeout" | "connect_neutral"; -export type CodexUpstreamOutcomeClass = "success" | "credential" - | "workspace" | "quota" | "transient" | "caller" | "neutral" | "unknown"; -export type CodexCooldownSource = "retry-after" | "reset-derived" | "default"; -/** - * Native Codex quota groups known to be independent upstream. Keep the mapping - * deliberately conservative: unlisted models share the normal native group. - * Add a new explicit group here only when its independent upstream quota is - * confirmed, so shared limits never receive cross-model bypasses. - */ -export type CodexQuotaScope = "shared" | "reserve"; - -export type CodexQuotaRecoveryProbeClaim = { - accountId: string; - scope?: CodexQuotaScope; - leaseId: string; - cooldownGeneration: number; - credentialGeneration: number; - /** Claim-time `replacedAt`; unchanged after a probe-owned refresh, stamped on external replacement. */ - credentialReplacedAt?: number; -}; - -export type CodexQuotaRecoveryProbeProof = { - credentialGeneration?: number; -}; - -/** - * Requests without a resolved native model retain the historic one-account-per- - * thread behavior. Requests with a known quota scope get an independent - * affinity so a Reserve failover cannot displace the same thread's Terra/Luna - * account (and vice versa). - */ -type BaseThreadAffinityScope = CodexQuotaScope | "legacy"; -type ModelDetourAffinityScope = `model-detour:${BaseThreadAffinityScope}:${string}`; -type ThreadAffinityScope = BaseThreadAffinityScope | ModelDetourAffinityScope; -const LEGACY_THREAD_AFFINITY_SCOPE = "legacy" as const; -const threadAccountMap = new Map>(); -let threadAffinityEntryTotal = 0; - -/** - * Which pool account minted the conversation's carried OpenAI state - * (`previous_response_id`, encrypted reasoning, provider conversation/file ids). - * Keyed by the same affinity key as {@link threadAccountMap}, bounded the same - * way, and process-local — raw account ids never reach a log. - */ -type ConversationStateIssuerEntry = { - accountId: string; - lastUsedAt: number; -}; -const conversationStateIssuerMap = new Map(); - -function isModelDetourAffinityScope(scope: ThreadAffinityScope): scope is ModelDetourAffinityScope { - return scope.startsWith("model-detour:"); -} - -const NATIVE_MODEL_QUOTA_SCOPES: Readonly> = { - [NATIVE_RESERVE_MODEL]: "reserve", -}; - -export function codexQuotaScopeForModel(modelId: string | undefined): CodexQuotaScope | undefined { - if (!modelId?.trim()) return undefined; - return NATIVE_MODEL_QUOTA_SCOPES[modelId.trim().toLowerCase()] ?? "shared"; -} - -/** Independent quota groups must not mutate the shared active-account cursor. */ -function isIndependentCodexQuotaScope(quotaScope?: CodexQuotaScope): boolean { - return quotaScope !== undefined && quotaScope !== "shared"; -} - -function codexPoolKeyForScope(quotaScope?: CodexQuotaScope): string { - return isIndependentCodexQuotaScope(quotaScope) ? `${POOL_KEY_CODEX}:${quotaScope}` : POOL_KEY_CODEX; -} - -export type CodexUpstreamOutcomeMeta = { - retryAfter?: string | null; - resetAt?: unknown | unknown[]; - now?: number; - /** (provider, host) ledger key for account-neutral reachability failures (#914). */ - hostKey?: string; - /** - * Upstream denial evidence for a 403. A workspace/entitlement denial means the CREDENTIAL - * is fine and the account simply cannot reach this workspace, so it must not be quarantined - * for reauthentication (#1789). Absent evidence keeps the historical credential handling. - */ - denial?: "workspace" | "entitlement"; - /** Stable transport code recorded alongside a neutral host failure. */ - lastFailureCode?: string; - /** Native model selected for this request; used only for confirmed scoped quotas. */ - modelId?: string; - /** When set, clears affinity for this thread immediately on transient failure. */ - threadId?: string | null; - /** - * Suppress Pool rotation and quota/transient affinity mutations for an account-qualified - * request. Credential failures still sweep stale affinities because reauthentication is - * account-wide. - */ - fixedAccount?: boolean; - /** - * Probe lease held by this request, when it was admitted through an active - * quota cooldown. Only the outcome carrying the current lease may clear the - * cooldown (#433). - */ - probeLeaseId?: string; - /** Scope of `probeLeaseId` when it was granted against a model-scoped cooldown. */ - probeQuotaScope?: CodexQuotaScope; - /** - * Already-chosen alternate for same-request 429 retry. When set, promotion - * reuses this account instead of calling {@link pickAlternateCodexAccount} - * again (which would advance a round-robin ring twice). - */ - promoteAccountId?: string; - /** Generation captured when this routed account was selected. */ - writerGeneration?: number; - /** - * Credential generation this request's bearer was read at. Distinct from - * `writerGeneration`, which tracks the config store. - * - * A 401 that arrives after the credential was already replaced is evidence about a - * token nobody is using any more, so it must not quarantine the replacement. Absent - * means the caller cannot supply lineage and the historical unfenced handling stands. - */ - credentialGeneration?: number; -}; - + classifyCodexUpstreamOutcome, + computeCodexUsageScore, + computeQuotaCooldown, + computeQuotaCooldownUntil, + parseRetryAfterMs, + parseResetCooldownMs, +} from "./routing/cooldown-math"; +export type { + CodexUpstreamOutcome, + CodexUpstreamOutcomeClass, + CodexCooldownSource, + CodexUpstreamOutcomeMeta, +} from "./routing/cooldown-math"; +export { + codexQuotaScopeForModel, + listLiveCodexAccountIds, + getCodexUpstreamHealth, + getCodexAccountCooldownUntil, + getCodexAccountHealthSnapshot, + getCodexQuotaHealthSnapshot, + isCodexAccountInCooldown, + clearCodexAccountCooldown, + getCodexAccountSoftAvoidUntil, + isCodexAccountSoftAvoided, +} from "./routing/health-store"; +export type { CodexQuotaScope } from "./routing/health-store"; +export { + tryAcquireCodexQuotaProbeLease, + canAcquireCodexQuotaProbeLease, + claimDueCodexQuotaRecoveryProbes, + claimManualResetCooldowns, + settleManualResetCooldown, + settleCodexQuotaRecoveryProbe, + tryAcquireCodexQuotaScopeProbeLease, + canAcquireCodexQuotaScopeProbeLease, + releaseCodexQuotaProbeLease, + releaseCodexQuotaScopeProbeLease, +} from "./routing/probe-lease"; +export type { + CodexQuotaRecoveryProbeClaim, + CodexQuotaRecoveryProbeProof, + ManualResetCooldownClaim, + ManualResetRefreshLineage, +} from "./routing/probe-lease"; +export { + CODEX_THREAD_AFFINITY_IDLE_TTL_MS, + CODEX_THREAD_AFFINITY_MAX_ENTRIES, + CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS, + CODEX_TRANSIENT_AFFINITY_HOLD_MS, + clearConversationStateIssuerMap, + clearThreadAccountMap, + clearThreadAccountMapForAccount, + debugCodexAffinityGenerations, + handOffThreadAffinityGeneration, + peekConversationStateIssuer, + rememberConversationStateIssuer, +} from "./routing/thread-affinity"; +export type { + CodexThreadResolution, + CodexAffinityMove, + CodexAffinityReason, + CodexAffinityDecision, +} from "./routing/thread-affinity"; +export { + isCodexAccountPlanExcluded, + getPoolAccountPlan, + pickLowestUsageCodexAccount, + pickAlternateCodexAccount, +} from "./routing/selection"; +export { + resetCodexRoutingForManualSelection, + getEffectiveActiveCodexAccountId, + isEffectiveCodexAccountPinned, +} from "./routing/active-account"; function hasConfiguredPoolAccount( config: OcxConfig, accountId: string, @@ -435,1345 +205,41 @@ function hasConfiguredPoolAccount( .some(account => isSelectableCodexPoolAccount(account) && account.id === accountId); } -export function listLiveCodexAccountIds(config: OcxConfig): ReadonlySet { - const ids = new Set((config.codexAccounts ?? []).map(account => account.id)); - const openai = config.providers.openai; - if (openai && openai.disabled !== true && isCanonicalOpenAiForwardProvider(openai)) { - ids.add(MAIN_CODEX_ACCOUNT_ID); - } - return ids; -} - -export function clearThreadAccountMap(): void { - threadAccountMap.clear(); - threadAffinityEntryTotal = 0; - // A refresh cooldown is per-account runtime state learned alongside these bindings. Leaving it - // behind here keeps an account out of selection after the roster it belonged to is gone. - clearAllCodexPoolRefreshFailures(); - conversationStateIssuerMap.clear(); -} - -export function clearConversationStateIssuerMap(): void { - conversationStateIssuerMap.clear(); -} - -export function clearThreadAccountMapForAccount( - accountId: string, - reason: CodexAffinityReason = "unusable", -): void { - for (const [threadId, affinities] of threadAccountMap) { - for (const [scope, entry] of affinities) { - if (entry.accountId === accountId && affinities.delete(scope)) { - threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); - notePendingReleaseReason(threadId, reason); - } - } - if (affinities.size === 0) threadAccountMap.delete(threadId); - } -} - -function pruneConversationStateIssuers(now: number): void { - for (const [key, entry] of conversationStateIssuerMap) { - if (now - entry.lastUsedAt > CODEX_THREAD_AFFINITY_IDLE_TTL_MS) { - conversationStateIssuerMap.delete(key); - } - } - while (conversationStateIssuerMap.size > CODEX_THREAD_AFFINITY_MAX_ENTRIES) { - let oldestKey: string | null = null; - let oldestAt = Number.POSITIVE_INFINITY; - for (const [key, entry] of conversationStateIssuerMap) { - if (entry.lastUsedAt < oldestAt) { - oldestAt = entry.lastUsedAt; - oldestKey = key; - } - } - if (!oldestKey) break; - conversationStateIssuerMap.delete(oldestKey); - } -} - -/** - * Record the pool account that just issued carried conversation state for this - * binding key. In-memory only; the id is never written to a request log. - */ -export function rememberConversationStateIssuer( - bindingKey: string, - accountId: string, - now = Date.now(), -): void { - if (!bindingKey.trim() || !accountId.trim()) return; - if (!admissibleAffinityComponent(bindingKey) || !admissibleAffinityComponent(accountId)) return; - pruneConversationStateIssuers(now); - conversationStateIssuerMap.set(bindingKey, { accountId, lastUsedAt: now }); - pruneConversationStateIssuers(now); -} - -/** Last account that minted carried state for this binding, if still in the TTL window. */ -export function peekConversationStateIssuer( - bindingKey: string, - now = Date.now(), -): string | undefined { - if (!bindingKey.trim() || !admissibleAffinityComponent(bindingKey)) return undefined; - pruneConversationStateIssuers(now); - const entry = conversationStateIssuerMap.get(bindingKey); - if (!entry) return undefined; - entry.lastUsedAt = now; - return entry.accountId; -} - -/** - * Why a binding was released, held until that thread's next resolve can report it (#4546). - * - * A release and the request that pays for it are two different moments: a 429 clears the pin - * inside the outcome recorder, and the next request arrives with nothing left to explain why it - * is starting cold. Bounded, because it is a diagnostic and must not become a leak. - */ -const pendingReleaseReasons = new Map(); -const MAX_PENDING_RELEASE_REASONS = 4096; - -function notePendingReleaseReason(threadId: string | null, reason: CodexAffinityReason): void { - if (threadId === null) return; - if (!pendingReleaseReasons.has(threadId) && pendingReleaseReasons.size >= MAX_PENDING_RELEASE_REASONS) { - const oldest = pendingReleaseReasons.keys().next(); - if (!oldest.done) pendingReleaseReasons.delete(oldest.value); - } - pendingReleaseReasons.set(threadId, reason); -} - -function peekPendingReleaseReason(threadId: string | null): CodexAffinityReason | undefined { - if (threadId === null) return undefined; - return pendingReleaseReasons.get(threadId); -} - -/** - * Forget a release only once it has actually been reported. - * - * Consuming it at derivation time lost it whenever selection then failed to produce an account: - * a no-account return carries no payload, so the release went unrecorded and the next successful - * resolve claimed a fresh healthy bind (#4598). A release survives until some resolve reports it. - */ -function clearPendingReleaseReason(threadId: string | null): void { - if (threadId !== null) pendingReleaseReasons.delete(threadId); -} - export function clearCodexUpstreamHealth(): void { // Operator preferences are routing state, not health, but they live and die with the same // reset points. Leaving them behind lets a selection from one context suppress the // automatic cursor in the next one. - manualPreference.clear(); - upstreamHealth.clear(); - quotaScopedHealth.clear(); - runtimeActiveCodexAccountId = undefined; + clearAllManualPreferences(); + clearUpstreamHealthState(); + forgetRuntimeActiveCodexAccount(); // The reconcile watermark is part of this state, not something that outlives it. Keeping // it across a full reset is incoherent: there is no health left to protect, yet // recordCodexUpstreamOutcome would still drop a writer whose generation predates the // watermark for any account missing from the equally stale live set. Left behind, it also // leaks between test files, which is how it was found. - lastReconciledGeneration = 0; - liveHealthAccountIds = new Set(); + resetHealthReconcileState(); } export function clearCodexUpstreamHealthForAccount(accountId: string): void { - upstreamHealth.delete(accountId); - quotaScopedHealth.delete(accountId); + deleteAllHealthForAccount(accountId); // Deletion is the third operator exit, next to pause and exclusion, and it is the one // with no reconcile path behind it: once the account is gone nothing can succeed on it, // so an unspent preference naming it would suppress the automatic cursor for every other // account until the process restarts. - forgetManualPreference(accountId); -} - -export function reconcileCodexRoutingHealth(context: GenerationContext): number { - if (context.generation <= lastReconciledGeneration) return 0; - let removed = 0; - for (const accountId of upstreamHealth.keys()) { - if (context.codexAccountIds.has(accountId)) continue; - upstreamHealth.delete(accountId); - removed += 1; - } - for (const accountId of quotaScopedHealth.keys()) { - if (context.codexAccountIds.has(accountId)) continue; - quotaScopedHealth.delete(accountId); - removed += 1; - } - // Sweep preferences the same way, for the account set this generation actually has. The - // delete path above is the direct route; this is the one that catches an account removed - // by an edit the runtime never saw. Deliberately not counted in `removed`, which reports - // health rows. - for (const [poolKey, preferred] of manualPreference) { - if (context.codexAccountIds.has(preferred)) continue; - manualPreference.delete(poolKey); - } - liveHealthAccountIds = new Set(context.codexAccountIds); - lastReconciledGeneration = context.generation; - return removed; -} - -export function getCodexUpstreamHealth( - accountId: string, -): CodexUpstreamHealth | null { - dropSpentCredentialFailure(accountId); - return upstreamHealth.get(accountId) ?? null; -} - -function scopedHealthFor(accountId: string, scope: CodexQuotaScope): CodexUpstreamHealth | undefined { - return quotaScopedHealth.get(accountId)?.get(scope); -} - -function setScopedHealth(accountId: string, scope: CodexQuotaScope, health: CodexUpstreamHealth): void { - let scopes = quotaScopedHealth.get(accountId); - if (!scopes) { - scopes = new Map(); - quotaScopedHealth.set(accountId, scopes); - } - scopes.set(scope, health); -} - -function deleteScopedHealth(accountId: string, scope: CodexQuotaScope): void { - const scopes = quotaScopedHealth.get(accountId); - if (!scopes) return; - scopes.delete(scope); - if (scopes.size === 0) quotaScopedHealth.delete(accountId); -} - -export function computeCodexUsageScore(quota: { - weeklyPercent?: number; - monthlyPercent?: number; - shortPercent?: number; - shortResetAt?: number; - shortObservedAt?: number; -} | null, plan?: unknown, now: number = Date.now()): number { - if (!quota) return CODEX_UNKNOWN_USAGE_SCORE; - const finite = (value: unknown): value is number => typeof value === "number" && Number.isFinite(value); - const longWindows = isThirtyDayOnlyCodexPlan(plan) - ? [quota.monthlyPercent] - : [quota.weeklyPercent, quota.monthlyPercent]; - const knownLong = longWindows.filter(finite); - // The short burst window only REFINES a known long-window position; it cannot stand in for - // one. A snapshot carrying just `shortPercent: 0` would otherwise score a flat 0 and make an - // account whose weekly/monthly usage is entirely unverified look like the emptiest in the - // pool, so `pickLowestUsageAmong` would send every request to it. Unknown has to stay - // unknown until a governing window is actually observed. - // - // A FULL burst window is the exception (#3029). It is not an optimistic guess about an - // unobserved window — it is a direct observation that the account cannot serve a request - // right now, whatever its monthly position turns out to be. Unknown-means-selectable is - // correct for uncertainty and wrong for a measured refusal: the account stays selected, - // `applyQuotaAutoSwitch` never fires, and the pool wedges on an exhausted credential. - if (knownLong.length === 0) { - return isTerminalShortWindow(quota, now) ? CODEX_EXHAUSTED_USAGE_PERCENT : CODEX_UNKNOWN_USAGE_SCORE; - } - const values = finite(quota.shortPercent) ? [...knownLong, quota.shortPercent] : knownLong; - return Math.max(...values); -} - -/** - * A short-only reading that proves the account is blocked NOW. - * - * Freshness is not optional. `getAccountQuota` performs no expiry check, partial updates - * carry a still-open short tuple forward, and disk hydration accepts a persisted reading for - * hours — so scoring 100 from `shortPercent` alone would keep excluding an account whose - * five-hour window has since reset. Merge no longer carries an elapsed shortResetAt, but an - * explicit incoming elapsed tuple is still stored, and a missing reset cannot be aged there. - * That is #3029 pointed the other way: the issue is that - * an exhausted account stays selected, and "a recovered account stays excluded" trades one - * unusable pool for another. - * - * A reading with no `shortResetAt` cannot be aged, so it stays unknown. The conservative - * direction here is the one that keeps an account selectable: a wrongly-selected account - * fails one request, while a wrongly-excluded one is invisible until someone reads the pool - * by hand. - * - * A missing reset can instead be aged by shortObservedAt (#3425). General updatedAt is not - * sufficient: credit-only updates preserve the old short tuple but advance that timestamp. - * Old disk snapshots without short-window provenance remain unknown. - */ -function isTerminalShortWindow( - quota: { shortPercent?: number; shortResetAt?: number; shortObservedAt?: number }, - now: number, -): boolean { - if (typeof quota.shortPercent !== "number" || !Number.isFinite(quota.shortPercent)) return false; - if (quota.shortPercent < CODEX_EXHAUSTED_USAGE_PERCENT) return false; - const resetAt = quota.shortResetAt; - if (typeof resetAt !== "number" || !Number.isFinite(resetAt) || resetAt <= 0) { - const observedAt = quota.shortObservedAt; - if (typeof observedAt !== "number" || !Number.isFinite(observedAt)) return false; - const age = now - observedAt; - return age >= 0 && age <= TERMINAL_SHORT_WINDOW_FRESHNESS_MS; - } - // Seconds and milliseconds both reach storage, so the split lives in one place next to the - // merge that also ages a stored reset instant (`resetAtToMs`, src/codex/quota.ts). - return resetAtToMs(resetAt) > now; -} - -export function classifyCodexUpstreamOutcome( - outcome: CodexUpstreamOutcome, - denial?: "workspace" | "entitlement", -): CodexUpstreamOutcomeClass { - if (outcome === "connect_neutral") return "neutral"; - if (outcome === "connect_error" || outcome === "timeout") return "transient"; - if (!Number.isFinite(outcome)) return "unknown"; - if (outcome >= 200 && outcome < 300) return "success"; - // Explicit 3xx policy (#914): a redirect response is relayed as-is and is - // never account or host health evidence — it proves the host is reachable - // and says nothing about the credential. Relayed as the neutral class so a - // stray 3xx cannot increment an account's transient streak. - if (outcome >= 300 && outcome < 400) return "neutral"; - // 401 is always a credential problem. A 403 is only a credential problem when nothing - // tells us otherwise: a workspace/entitlement denial (#1789) means the credential is valid - // and the account simply lacks access here, so quarantining it for reauth is wrong advice. - // Absent denial evidence the historical mapping stands, so the change fails safe. - if (outcome === 403 && denial !== undefined) return "workspace"; - if (outcome === 401 || outcome === 403) return "credential"; - // 402 Payment Required is treated as quota exhaustion for pool cooldown/failover - // (same-request alternate retry records this outcome for the depleted account). - if (outcome === 429 || outcome === 402) return "quota"; - if (outcome >= 400 && outcome < 500) return "caller"; - if (outcome >= 500 && outcome < 600) return "transient"; - return "unknown"; -} - -function clampCooldownMs(ms: number): number { - return Math.min(Math.max(ms, 1), CODEX_MAX_QUOTA_COOLDOWN_MS); -} - -export function parseRetryAfterMs(value: string | null | undefined, now = Date.now()): number | undefined { - const text = value?.trim(); - if (!text) return undefined; - if (/^\d+(?:\.\d+)?$/.test(text)) { - const seconds = Number(text); - if (Number.isFinite(seconds) && seconds > 0) return clampCooldownMs(Math.ceil(seconds * 1000)); - } - const timestamp = Date.parse(text); - if (!Number.isFinite(timestamp)) return undefined; - const delay = timestamp - now; - return delay > 0 ? clampCooldownMs(delay) : undefined; -} - -function resetTimestampMs(value: unknown): number | undefined { - const numeric = typeof value === "number" - ? value - : typeof value === "string" && value.trim() !== "" - ? Number(value) - : undefined; - if (typeof numeric !== "number" || !Number.isFinite(numeric) || numeric <= 0) return undefined; - return numeric < 1_000_000_000_000 ? numeric * 1000 : numeric; -} - -export function parseResetCooldownMs(resetAt: unknown | unknown[] | undefined, now = Date.now()): number | undefined { - const values = Array.isArray(resetAt) ? resetAt : [resetAt]; - let best: number | undefined; - for (const value of values) { - const timestamp = resetTimestampMs(value); - if (timestamp === undefined) continue; - const delay = timestamp - now; - if (delay <= 0) continue; - // A far-future reset must not pin the account for the full Retry-After - // ceiling: quota usually frees up well before the advertised window (#433). - const clamped = Math.min(clampCooldownMs(delay), CODEX_MAX_RESET_DERIVED_COOLDOWN_MS); - if (best === undefined || clamped < best) best = clamped; - } - return best; -} - -export function computeQuotaCooldown(meta: CodexUpstreamOutcomeMeta = {}): { - until: number; - source: CodexCooldownSource; -} { - const now = meta.now ?? Date.now(); - const retryAfterMs = parseRetryAfterMs(meta.retryAfter, now); - if (retryAfterMs !== undefined) return { until: now + retryAfterMs, source: "retry-after" }; - const resetCooldownMs = parseResetCooldownMs(meta.resetAt, now); - if (resetCooldownMs !== undefined) return { until: now + resetCooldownMs, source: "reset-derived" }; - return { until: now + CODEX_DEFAULT_QUOTA_COOLDOWN_MS, source: "default" }; -} - -/** - * When the pool should stop preferring an account after it refused on quota. - * - * The earliest window the refusal actually announced, bounded by {@link CODEX_MAX_QUOTA_AVOID_MS}, - * and never shorter than the cooldown the same refusal produced — a Retry-After directive that - * outlasts every announcement still governs. - */ -function quotaAvoidUntilFor(meta: CodexUpstreamOutcomeMeta, now: number, cooldownUntil: number): number { - const values = Array.isArray(meta.resetAt) ? meta.resetAt : [meta.resetAt]; - let announced: number | undefined; - for (const value of values) { - const timestamp = resetTimestampMs(value); - if (timestamp === undefined) continue; - const delay = timestamp - now; - if (delay <= 0) continue; - const until = now + Math.min(delay, CODEX_MAX_QUOTA_AVOID_MS); - if (announced === undefined || until < announced) announced = until; - } - return Math.max(cooldownUntil, announced ?? 0); -} - -/** Live quota-refusal avoidance for an account, including the lane the request belongs to. */ -function codexQuotaAvoidUntil( - accountId: string, - quotaScope: CodexQuotaScope | undefined, - now: number, -): number | null { - const live = (value: number | undefined): number | null => - typeof value === "number" && Number.isFinite(value) && value > now ? value : null; - const account = live(upstreamHealth.get(accountId)?.quotaAvoidUntil); - const scoped = quotaScope === undefined - ? null - : live(scopedHealthFor(accountId, quotaScope)?.quotaAvoidUntil); - if (account === null) return scoped; - return scoped === null ? account : Math.max(account, scoped); -} - -function isCodexQuotaAvoided( - accountId: string, - quotaScope: CodexQuotaScope | undefined, - now: number, -): boolean { - return codexQuotaAvoidUntil(accountId, quotaScope, now) !== null; -} - -export function computeQuotaCooldownUntil(meta: CodexUpstreamOutcomeMeta = {}): number { - return computeQuotaCooldown(meta).until; -} - -/** - * Grant at most one probe lease per interval for a cooled-down account. - * - * A cooled-down account is short-circuited locally, so it never sends traffic and - * no organic 2xx can prove that upstream quota recovered — the cooldown can only - * end by expiry or a proxy restart (#433). Releasing a single probe breaks that - * deadlock. Explicit Retry-After cooldowns are excluded: those are literal retry - * directives, not window announcements. - * - * Returns the lease id, or null when no probe may go out right now. - */ -export function tryAcquireCodexQuotaProbeLease(accountId: string, now = Date.now()): string | null { - if (!canAcquireCodexQuotaProbeLease(accountId, now)) return null; - const health = upstreamHealth.get(accountId)!; - const probeLeaseId = randomUUID(); - upstreamHealth.set(accountId, { - ...health, - probeLeaseId, - probeLeaseGeneration: health.cooldownGeneration ?? 0, - lastProbeAt: now, - }); - return probeLeaseId; -} - -/** Side-effect-free check mirroring {@link tryAcquireCodexQuotaProbeLease} eligibility. */ -export function canAcquireCodexQuotaProbeLease(accountId: string, now = Date.now()): boolean { - return canAcquireQuotaProbeLease(upstreamHealth.get(accountId), now); -} - -function canAcquireQuotaProbeLease(health: CodexUpstreamHealth | undefined, now: number): boolean { - if (!health) return false; - const cooldownUntil = health.cooldownUntil; - if (typeof cooldownUntil !== "number" || !Number.isFinite(cooldownUntil) || cooldownUntil <= now) return false; - if (health.cooldownSource === "retry-after") return false; - if (health.probeLeaseId !== undefined) return false; - const origin = health.lastProbeAt ?? health.cooldownSince ?? cooldownUntil; - return now - origin >= CODEX_QUOTA_PROBE_INTERVAL_MS; -} - -/** - * Claim due reset-derived cooldown probes without consulting account selection. - * Added Pool credentials only; owned main usage recovery is handled separately. - */ -export function claimDueCodexQuotaRecoveryProbes( - config: OcxConfig, - limit: number, - now = Date.now(), -): CodexQuotaRecoveryProbeClaim[] { - const boundedLimit = Math.max(0, Math.floor(limit)); - if (boundedLimit === 0) return []; - const candidates: Array<{ - accountId: string; - scope?: CodexQuotaScope; - health: CodexUpstreamHealth; - credentialGeneration: number; - credentialReplacedAt?: number; - order: number; - }> = []; - for (const [order, account] of (config.codexAccounts ?? []).entries()) { - if (!isSelectableCodexPoolAccount(account) - || isCodexAccountPaused(config, account.id) - || isAccountNeedsReauth(account.id)) continue; - const record = readCodexAccountRecord(account.id); - if (!record?.credential || record.deletedAt != null) continue; - const due = [ - { scope: undefined, health: upstreamHealth.get(account.id) }, - ...[...(quotaScopedHealth.get(account.id) ?? [])].map(([scope, health]) => ({ scope, health })), - ].filter((entry): entry is { scope?: CodexQuotaScope; health: CodexUpstreamHealth } => - // Generic WHAM evidence can recover only ordinary quota, never Reserve. - // Do not spend this account's one claim per pass on an independent scope and - // delay the shared scope that the response can actually recover. - (entry.scope === undefined || entry.scope === "shared") - && entry.health?.cooldownSource === "reset-derived" - && canAcquireQuotaProbeLease(entry.health, now)) - .sort((a, b) => - (a.health.lastProbeAt ?? a.health.cooldownSince ?? 0) - - (b.health.lastProbeAt ?? b.health.cooldownSince ?? 0)); - const candidate = due[0]; - if (candidate) candidates.push({ - accountId: account.id, - ...(candidate.scope ? { scope: candidate.scope } : {}), - health: candidate.health, - credentialGeneration: record.generation, - ...(record.replacedAt !== undefined ? { credentialReplacedAt: record.replacedAt } : {}), - order, - }); - } - candidates.sort((a, b) => { - const age = (a.health.lastProbeAt ?? a.health.cooldownSince ?? 0) - - (b.health.lastProbeAt ?? b.health.cooldownSince ?? 0); - return age || a.order - b.order; - }); - return candidates.slice(0, boundedLimit).map(candidate => { - const leaseId = randomUUID(); - const next = { - ...candidate.health, - probeLeaseId: leaseId, - probeLeaseGeneration: candidate.health.cooldownGeneration ?? 0, - lastProbeAt: now, - }; - if (candidate.scope) setScopedHealth(candidate.accountId, candidate.scope, next); - else upstreamHealth.set(candidate.accountId, next); - return { - accountId: candidate.accountId, - ...(candidate.scope ? { scope: candidate.scope } : {}), - leaseId, - cooldownGeneration: candidate.health.cooldownGeneration ?? 0, - credentialGeneration: candidate.credentialGeneration, - ...(candidate.credentialReplacedAt !== undefined - ? { credentialReplacedAt: candidate.credentialReplacedAt } - : {}), - }; - }); -} - -type CooldownRecoveryLease = Pick; - -export type ManualResetCooldownClaim = - | { kind: "pool"; probe: CodexQuotaRecoveryProbeClaim } - | { kind: "main"; probe: CooldownRecoveryLease }; - -function manualResetAccountEligible(config: OcxConfig, accountId: string): boolean { - return !isCodexAccountPaused(config, accountId) && !isAccountNeedsReauth(accountId) - && (accountId === MAIN_CODEX_ACCOUNT_ID - || (config.codexAccounts ?? []).some(account => account.id === accountId && isSelectableCodexPoolAccount(account))); -} - -/** Explicit reset bypasses probe pacing, never another owner's lease or quota scope. */ -export function claimManualResetCooldowns( - config: OcxConfig, - accountId: string, - now = Date.now(), - expectedPoolGeneration?: number, -): ManualResetCooldownClaim[] { - if (!manualResetAccountEligible(config, accountId)) return []; - const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); - if (accountId !== MAIN_CODEX_ACCOUNT_ID && (!record?.credential || record.deletedAt != null)) return []; - if (record && expectedPoolGeneration !== undefined && record.generation !== expectedPoolGeneration) return []; - const claims: ManualResetCooldownClaim[] = []; - for (const scope of [undefined, "shared"] as const) { - const health = scope ? scopedHealthFor(accountId, scope) : upstreamHealth.get(accountId); - if (!health || health.cooldownSource !== "reset-derived" || health.probeLeaseId !== undefined - || !Number.isFinite(health.cooldownUntil) || !(health.cooldownUntil! > now)) continue; - const leaseId = randomUUID(); - const cooldownGeneration = health.cooldownGeneration ?? 0; - const next = { ...health, probeLeaseId: leaseId, probeLeaseGeneration: cooldownGeneration, lastProbeAt: now }; - if (scope) setScopedHealth(accountId, scope, next); - else upstreamHealth.set(accountId, next); - const probe = { accountId, scope, leaseId, cooldownGeneration }; - claims.push(record ? { kind: "pool", probe: { - ...probe, credentialGeneration: record.generation, credentialReplacedAt: record.replacedAt, - } } : { kind: "main", probe }); - } - return claims; -} - -export type ManualResetRefreshLineage = Readonly<{ - fromGeneration: number; - toGeneration: number; - provenance: CodexRefreshProvenance; -}>; - -type ManualResetQuotaProof = CodexQuotaRecoveryProbeProof & { - refreshLineage?: ManualResetRefreshLineage; -}; - -/** Main proof is checked by the already-owned auth operation, never by a Pool record. */ -export function settleManualResetCooldown( - config: OcxConfig, - claim: ManualResetCooldownClaim, - recovered: boolean, - proof: ManualResetQuotaProof = {}, - now = Date.now(), -): boolean { - if (!recovered) return settleCooldownRecoveryLease(claim.probe, false, now); - const eligible = manualResetAccountEligible(config, claim.probe.accountId); - if (claim.kind === "main") return settleCooldownRecoveryLease(claim.probe, eligible, now); - const lineage = proof.refreshLineage; - // Equal wall-clock replacement stamps do not establish ancestry. Manual +1 - // recovery additionally needs the actual forced-refresh result for this edge. - const ownedGeneration = proof.credentialGeneration === claim.probe.credentialGeneration - || (proof.credentialGeneration === claim.probe.credentialGeneration + 1 - && lineage?.fromGeneration === claim.probe.credentialGeneration - && lineage.toGeneration === proof.credentialGeneration - && (lineage.provenance === "self-refresh" || lineage.provenance === "joined-lineage")); - return settleCodexQuotaRecoveryProbe(claim.probe, eligible && ownedGeneration, proof, now); -} - -/** Settle one background recovery claim without mutating account-wide outcome state. */ -export function settleCodexQuotaRecoveryProbe( - claim: CodexQuotaRecoveryProbeClaim, - recovered: boolean, - proof: CodexQuotaRecoveryProbeProof, - now = Date.now(), -): boolean { - const health = claim.scope - ? scopedHealthFor(claim.accountId, claim.scope) - : upstreamHealth.get(claim.accountId); - if (!health || health.probeLeaseId !== claim.leaseId) return false; - const currentRecord = readCodexAccountRecord(claim.accountId); - const proofGeneration = proof.credentialGeneration; - // A probe-owned token refresh (getValidCodexToken) advances the credential generation by - // exactly one while preserving `replacedAt`; an external credential replacement bumps the - // generation too but stamps a fresh `replacedAt`. Accept the +1 transition only when the - // claim-time lineage is intact AND the generation the fresh quota was proven under is live. - const generationFenced = proofGeneration !== undefined - && (proofGeneration === claim.credentialGeneration - ? isCodexAccountGenerationLive(claim.accountId, proofGeneration) - : proofGeneration === claim.credentialGeneration + 1 - && currentRecord?.replacedAt === claim.credentialReplacedAt - && isCodexAccountGenerationLive(claim.accountId, proofGeneration)); - return settleCooldownRecoveryLease(claim, recovered && generationFenced, now); -} - -function settleCooldownRecoveryLease(claim: CooldownRecoveryLease, recovered: boolean, now: number): boolean { - const health = claim.scope ? scopedHealthFor(claim.accountId, claim.scope) : upstreamHealth.get(claim.accountId); - if (!health || health.probeLeaseId !== claim.leaseId) return false; - const fenced = (claim.scope === undefined || claim.scope === "shared") - && health.cooldownSource === "reset-derived" - && (health.cooldownGeneration ?? 0) === claim.cooldownGeneration - && (health.probeLeaseGeneration ?? 0) === claim.cooldownGeneration; - if (!recovered || !fenced) { - const released = withProbeLeaseReleased(health, now); - if (claim.scope) setScopedHealth(claim.accountId, claim.scope, released); - else upstreamHealth.set(claim.accountId, released); - return false; - } - if (claim.scope) { - deleteScopedHealth(claim.accountId, claim.scope); - } else { - const { - cooldownUntil: _until, - cooldownSince: _since, - cooldownSource: _source, - probeLeaseId: _leaseId, - probeLeaseGeneration: _leaseGeneration, - // "The quota window moved" is a statement about the whole refusal, so the avoidance it - // announced goes with the block it produced. Leaving it would make this escape hatch stop - // escaping: the account would still be passed over by every selection it is meant to win. - quotaAvoidUntil: _avoid, - ...rest - } = health; - upstreamHealth.set(claim.accountId, { - ...rest, - cooldownGeneration: claim.cooldownGeneration + 1, - lastProbeAt: now, - }); - } - return true; -} - -/** Acquire the recovery probe for one confirmed model-specific quota group. */ -export function tryAcquireCodexQuotaScopeProbeLease( - accountId: string, - scope: CodexQuotaScope, - now = Date.now(), -): string | null { - const health = scopedHealthFor(accountId, scope); - if (!canAcquireQuotaProbeLease(health, now)) return null; - const probeLeaseId = randomUUID(); - setScopedHealth(accountId, scope, { - ...health!, - probeLeaseId, - probeLeaseGeneration: health!.cooldownGeneration ?? 0, - lastProbeAt: now, - }); - return probeLeaseId; -} - -/** Side-effect-free check for a confirmed model-specific quota probe. */ -export function canAcquireCodexQuotaScopeProbeLease( - accountId: string, - scope: CodexQuotaScope, - now = Date.now(), -): boolean { - return canAcquireQuotaProbeLease(scopedHealthFor(accountId, scope), now); -} - -/** - * Hand a probe lease back without recording an upstream outcome. Used by paths - * that take a lease and then fail before any request reaches upstream. - */ -export function releaseCodexQuotaProbeLease(accountId: string, leaseId: string, now = Date.now()): void { - const health = upstreamHealth.get(accountId); - if (!health || health.probeLeaseId !== leaseId) return; - upstreamHealth.set(accountId, withProbeLeaseReleased(health, now)); -} - -/** Release a model-specific quota probe when the request never reaches upstream. */ -export function releaseCodexQuotaScopeProbeLease( - accountId: string, - scope: CodexQuotaScope, - leaseId: string, - now = Date.now(), -): void { - const health = scopedHealthFor(accountId, scope); - if (!health || health.probeLeaseId !== leaseId) return; - setScopedHealth(accountId, scope, withProbeLeaseReleased(health, now)); -} - -/** - * True when this outcome belongs to the account's in-flight probe. The - * undefined-id guard matters: without it an outcome carrying no lease would match - * an account holding no lease and be mistaken for the probe owner. - */ -function ownsProbeLease(health: CodexUpstreamHealth | undefined, meta: CodexUpstreamOutcomeMeta): boolean { - return meta.probeLeaseId !== undefined && meta.probeLeaseId === health?.probeLeaseId; -} - -/** - * True when the owning probe may still clear the cooldown. A later 429 bumps the - * generation, so a probe that started under an older cooldown must not erase the - * newer restriction (which may carry an explicit Retry-After). - */ -function probeMayClearCooldown(health: CodexUpstreamHealth | undefined, meta: CodexUpstreamOutcomeMeta): boolean { - return ownsProbeLease(health, meta) - && (health!.probeLeaseGeneration ?? 0) === (health!.cooldownGeneration ?? 0); -} - -/** Strip the in-flight lease while preserving every hard-cooldown field. */ -function withProbeLeaseReleased(health: CodexUpstreamHealth, now: number): CodexUpstreamHealth { - const { probeLeaseId: _id, probeLeaseGeneration: _gen, ...rest } = health; - return { ...rest, lastProbeAt: now }; -} - -/** - * Hard-cooldown bookkeeping that ordinary success/transient transitions rebuild - * their health object from. Dropping these would let one late unrelated response - * erase a Retry-After source, a cooldown generation, or someone else's live probe. - */ -function preservedCooldownFields(health: CodexUpstreamHealth | undefined): Partial { - if (!health) return {}; - // `credentialFailureGeneration` is provenance for ONE credential failure, so it must not survive - // into a later transient or quota entry — otherwise that entry inherits the tag and gets spent - // when the old credential dies, deleting evidence that was never about it (#2892 gap 4 review). - const { - consecutiveFailures: _f, consecutiveSuccesses: _s, lastFailureStatus: _st, lastFailureAt: _at, - softAvoidUntil: _sa, credentialFailureGeneration: _cg, ...cooldownFields - } = health; - return cooldownFields; -} - -/** Manual selection resets transient routing evidence without bypassing a real 429 cooldown. */ -export function resetCodexRoutingForManualSelection(accountId: string): void { - clearThreadAccountMap(); - // Manual selection is the operator source of truth — drop any automatic runtime cursor. - runtimeActiveCodexAccountId = undefined; - // Record the pick as an unspent one-shot on the SHARED scope only. An independent scope - // gets no entry on purpose: every write site the guard protects is already skipped for - // independent scopes, so an entry there would be state nothing reads — and state nothing - // reads is what the next reader mistakes for a rule. - // - // Seeding happens ONLY here. A pool-driven promote must never create or move a preference, - // or the pool would manufacture an operator intent nobody expressed. - manualPreference.set(POOL_KEY_CODEX, accountId); - // Seed the RR ring so the next unbound new session honors the manually selected account - // under round-robin (affinity-cleared threads / null threadId). Fill-first already follows - // config.activeCodexAccountId, which the caller persists before invoking this. - seedPoolRotationAccount(POOL_KEY_CODEX, accountId); - for (const scope of new Set(Object.values(NATIVE_MODEL_QUOTA_SCOPES))) { - if (isIndependentCodexQuotaScope(scope)) { - seedPoolRotationAccount(codexPoolKeyForScope(scope), accountId); - } - } - // Quota avoidance is a preference, like the soft avoid dropped above, and an operator naming - // this account has overruled it. The hard cooldown is the part that survives. - const overrule = (health: CodexUpstreamHealth) => { - const { quotaAvoidUntil: _avoid, ...retained } = preservedCooldownFields(health); - return retained; - }; - const current = upstreamHealth.get(accountId); - if (current) { - const retained = overrule(current); - if (Object.keys(retained).length === 0) upstreamHealth.delete(accountId); - else upstreamHealth.set(accountId, { consecutiveFailures: 0, ...retained }); - } - // A reset-derived refusal records its avoidance on the SCOPED map and returns before the - // account-wide entry is written, so naming the account has to reach that map too. Stopping - // at `upstreamHealth` — and returning early when it holds nothing — overruled nothing in - // the case that produces the avoidance this function exists to overrule. - for (const [scope, health] of [...(quotaScopedHealth.get(accountId) ?? [])]) { - const retained = overrule(health); - if (Object.keys(retained).length === 0) deleteScopedHealth(accountId, scope); - else setScopedHealth(accountId, scope, { consecutiveFailures: 0, ...retained }); - } -} - -export function getCodexAccountCooldownUntil(accountId: string, now = Date.now()): number | null { - const cooldownUntil = upstreamHealth.get(accountId)?.cooldownUntil; - return typeof cooldownUntil === "number" && Number.isFinite(cooldownUntil) && cooldownUntil > now ? cooldownUntil : null; -} - -/** Read-only cooldown snapshot for shared OAuth health projection (no write side effects). */ -export function getCodexAccountHealthSnapshot(accountId: string, now = Date.now()): { - cooldownUntil?: number; - cooldownSource?: CodexCooldownSource; -} | null { - const cooldownUntil = getCodexAccountCooldownUntil(accountId, now); - if (cooldownUntil === null) return null; - const source = upstreamHealth.get(accountId)?.cooldownSource; - return { - cooldownUntil, - ...(source ? { cooldownSource: source } : {}), - }; -} - -/** - * Read the cooldown relevant to a routed native model. Account-wide cooldowns - * (Retry-After/default) always win; reset-derived scoped state applies only to - * its confirmed quota group. - */ -export function getCodexQuotaHealthSnapshot( - accountId: string, - quotaScope: CodexQuotaScope | undefined, - now = Date.now(), -): { - cooldownUntil?: number; - cooldownSource?: CodexCooldownSource; - quotaScope?: CodexQuotaScope; -} | null { - const account = getCodexAccountHealthSnapshot(accountId, now); - if (account) return account; - if (!quotaScope) return null; - const scoped = scopedHealthFor(accountId, quotaScope); - const cooldownUntil = scoped?.cooldownUntil; - if (typeof cooldownUntil !== "number" || !Number.isFinite(cooldownUntil) || cooldownUntil <= now) return null; - return { - cooldownUntil, - ...(scoped?.cooldownSource ? { cooldownSource: scoped.cooldownSource } : {}), - quotaScope, - }; -} - -export function isCodexAccountInCooldown(accountId: string, now = Date.now()): boolean { - return getCodexAccountCooldownUntil(accountId, now) !== null; -} - -/** - * Manually lift a hard quota cooldown without touching failure history. - * - * Injected Codex routing makes this proxy the ONLY model path for Codex Desktop, so a - * cooldown that outlives the real upstream limit reads to the user as "the whole app is - * broken" with no escape but editing config.toml. This is that escape hatch. - * - * Deliberately narrow: - * - Failure counters and softAvoid survive. Clearing a cooldown says "the quota window - * moved", not "this account is healthy"; failover must keep its knowledge. - * - Dropping `probeLeaseId` is what stops a stale in-flight probe from later "proving" - * recovery against a NEWER cooldown: {@link ownsProbeLease} needs the id to match. - * `cooldownGeneration` is preserved and bumped as redundancy only — a fresh 429 already - * bumps it in {@link recordCodexUpstreamOutcome}, so the bump here is not load-bearing - * today and is kept so the invariant survives a future change that retains the lease. - * - * Returns false when the account carried neither a live cooldown nor a live avoidance window. - * The window outlives the cooldown by design — the cooldown caps at fifteen minutes and the - * window runs up to six hours — so the moment an operator actually reaches for this escape - * hatch is usually after the cooldown lapsed and only the window is still keeping the account - * out of rotation. Refusing to look at the window then would leave the hatch shut in the one - * case it exists for. - */ -export function clearCodexAccountCooldown(accountId: string, now = Date.now()): boolean { - const clear = (health: CodexUpstreamHealth): CodexUpstreamHealth | null => { - const cooldownUntil = health.cooldownUntil; - const liveCooldown = typeof cooldownUntil === "number" && Number.isFinite(cooldownUntil) && cooldownUntil > now; - const avoidUntil = health.quotaAvoidUntil; - const liveAvoidance = typeof avoidUntil === "number" && Number.isFinite(avoidUntil) && avoidUntil > now; - if (!liveCooldown && !liveAvoidance) return null; - const { - cooldownUntil: _until, - cooldownSince: _since, - cooldownSource: _source, - probeLeaseId: _leaseId, - probeLeaseGeneration: _leaseGeneration, - // Same reasoning as the probe recovery above: "the quota window moved" is a statement - // about the whole refusal, so the avoidance it announced goes with the block it - // produced. Keeping it would leave this escape hatch not escaping, because selection - // would still pass over the account for as long as the announced window runs. - quotaAvoidUntil: _avoid, - ...rest - } = health; - return { - ...rest, - cooldownGeneration: (health.cooldownGeneration ?? 0) + 1, - lastProbeAt: now, - }; - }; - - let cleared = false; - const accountHealth = upstreamHealth.get(accountId); - if (accountHealth) { - const next = clear(accountHealth); - if (next) { - upstreamHealth.set(accountId, next); - cleared = true; - } - } - for (const [scope, health] of quotaScopedHealth.get(accountId) ?? []) { - const next = clear(health); - if (next) { - setScopedHealth(accountId, scope, next); - cleared = true; - } - } - return cleared; -} - -export function getCodexAccountSoftAvoidUntil(accountId: string, now = Date.now()): number | null { - const softAvoidUntil = upstreamHealth.get(accountId)?.softAvoidUntil; - return typeof softAvoidUntil === "number" && Number.isFinite(softAvoidUntil) && softAvoidUntil > now - ? softAvoidUntil - : null; -} - -export function isCodexAccountSoftAvoided(accountId: string, now = Date.now()): boolean { - return getCodexAccountSoftAvoidUntil(accountId, now) !== null; -} - -/** - * Plan keys the operator excluded from automatic rotation. Absent or empty means no policy, so an - * existing install rotates exactly as before. Compared with `codexPlanKey` because the stored plan - * is an unrestricted provider string whose casing this repository does not control. - */ -function excludedCodexPoolPlanKeys(config: OcxConfig): ReadonlySet | undefined { - const configured = config.codexPool?.excludedPlans; - if (!configured?.length) return undefined; - const keys = configured - .map(plan => codexPlanKey(plan)) - .filter((key): key is string => key !== undefined); - return keys.length > 0 ? new Set(keys) : undefined; -} - -/** - * Whether the operator's plan policy removes this account from automatic selection. - * - * Modelled on pause rather than usability: an excluded account keeps its credential, quota history, - * and affinity, stays visible on the account surface, and is still reachable by explicit account - * selection. Only automatic rotation skips it, which is the distinction #4211 asked for. - * - * It is checked in the same two places pause is checked, and that is not redundancy. The eligible - * list is consulted only when routing picks a NEW account; an already-active or already-affined - * account is served straight from {@link isCodexAccountSelectable}. A lapsed subscription leaves - * behind exactly that account, so a policy that filtered only the eligible list would miss the case - * it exists for. - * - * `__main__` is exempt. {@link getPoolAccountPlanForSelection} withholds the main plan during a - * selection-only drain so routing never reads the fenced native credential for it, so a rule that - * covered main would disagree with itself between drain and ordinary routing. - */ -export function isCodexAccountPlanExcluded( - config: OcxConfig, - accountId: string, - precomputed?: ReadonlySet, -): boolean { - if (accountId === MAIN_CODEX_ACCOUNT_ID) return false; - // Callers that test a whole list pass the set once rather than rebuilding it per row. - const excluded = precomputed ?? excludedCodexPoolPlanKeys(config); - if (!excluded) return false; - const plan = codexPlanKey(getPoolAccountPlan(config, accountId)); - return plan !== undefined && excluded.has(plan); -} - -function isCodexAccountSelectable( - config: OcxConfig, - accountId: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): boolean { - return !isCodexAccountPaused(config, accountId) - && !isCodexAccountPlanExcluded(config, accountId) - && getCodexQuotaHealthSnapshot(accountId, quotaScope, now) === null - && !isCodexQuotaAvoided(accountId, quotaScope, now) - && !isCodexAccountSoftAvoided(accountId, now) - && !isCodexPoolRefreshCooling(accountId, now) - && isCodexAccountUsable(config, accountId, selectionOptions); -} - -/** - * Which guard in {@link isCodexAccountSelectable} refused this account, if any. - * - * Deliberately the same predicates in the same order as that function, because the point is to - * REPORT the guard that actually fired rather than to re-derive a plausible-looking cause. An - * earlier version of the release reason checked only a subset and let a paused, plan-excluded, - * cooled-down or quota-avoided release fall through to a quota fallback, which named something - * routing never used -- a diagnostic that is confidently wrong in exactly the cases an operator - * would consult it for (#4598). - */ -function codexAccountBlockReason( - config: OcxConfig, - accountId: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): CodexAffinityReason | undefined { - if (isCodexAccountPaused(config, accountId)) return "paused"; - if (isCodexAccountPlanExcluded(config, accountId)) return "plan_excluded"; - if (getCodexQuotaHealthSnapshot(accountId, quotaScope, now) !== null) return "cooldown"; - if (isCodexQuotaAvoided(accountId, quotaScope, now)) return "quota_avoided"; - if (isCodexAccountSoftAvoided(accountId, now)) return "transient"; - if (isCodexPoolRefreshCooling(accountId, now)) return "transient"; - if (!isCodexAccountUsable(config, accountId, selectionOptions)) return "unusable"; - return undefined; -} - -function threadAffinityScope(quotaScope?: CodexQuotaScope): BaseThreadAffinityScope { - return quotaScope ?? LEGACY_THREAD_AFFINITY_SCOPE; -} - -function admissibleAffinityComponent(value: string): boolean { - return retainedUtf8Bytes(value) <= MAX_AFFINITY_COMPONENT_BYTES; -} - -function modelDetourAffinityScope( - modelId: string | undefined, - quotaScope?: CodexQuotaScope, -): ModelDetourAffinityScope | undefined { - const canonicalModelId = modelId?.trim().toLowerCase(); - if (!canonicalModelId || !admissibleAffinityComponent(canonicalModelId)) return undefined; - return `model-detour:${threadAffinityScope(quotaScope)}:${canonicalModelId}`; -} - -function getThreadAffinityForScope( - threadId: string, - scope: ThreadAffinityScope, -): ThreadAffinityEntry | undefined { - if (!admissibleAffinityComponent(threadId)) return undefined; - return threadAccountMap.get(threadId)?.get(scope); -} - -function getThreadAffinity(threadId: string, quotaScope?: CodexQuotaScope): ThreadAffinityEntry | undefined { - return getThreadAffinityForScope(threadId, threadAffinityScope(quotaScope)); -} - -function getModelDetourAffinity( - threadId: string, - modelId: string | undefined, - quotaScope?: CodexQuotaScope, -): ThreadAffinityEntry | undefined { - const scope = modelDetourAffinityScope(modelId, quotaScope); - return scope ? getThreadAffinityForScope(threadId, scope) : undefined; -} - -function deleteThreadAffinityForScope(threadId: string, scope: ThreadAffinityScope): void { - if (!admissibleAffinityComponent(threadId)) return; - const affinities = threadAccountMap.get(threadId); - if (!affinities) return; - if (affinities.delete(scope)) { - threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); - } - if (affinities.size === 0) threadAccountMap.delete(threadId); -} - -function deleteThreadAffinity(threadId: string, quotaScope?: CodexQuotaScope): void { - deleteThreadAffinityForScope(threadId, threadAffinityScope(quotaScope)); -} - -function deleteModelDetourAffinity( - threadId: string, - modelId: string | undefined, - quotaScope?: CodexQuotaScope, -): void { - const scope = modelDetourAffinityScope(modelId, quotaScope); - if (scope) deleteThreadAffinityForScope(threadId, scope); -} - -/** Remove only the matching failed account's affinities for one thread. */ -function deleteThreadAffinitiesForAccount(threadId: string, accountId: string): void { - if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; - const affinities = threadAccountMap.get(threadId); - if (!affinities) return; - for (const [scope, entry] of affinities) { - if (entry.accountId === accountId && affinities.delete(scope)) { - threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); - } - } - if (affinities.size === 0) threadAccountMap.delete(threadId); -} - -function threadAffinityEntryCount(): number { - return threadAffinityEntryTotal; -} - -function isThreadAffinityExpired(entry: ThreadAffinityEntry, now: number): boolean { - return now - entry.lastUsedAt > CODEX_THREAD_AFFINITY_IDLE_TTL_MS; -} - -function isThreadAffinityGenerationLive(entry: ThreadAffinityEntry): boolean { - if (entry.accountId === MAIN_CODEX_ACCOUNT_ID) return entry.generation === 0; - return isCodexAccountGenerationLive(entry.accountId, entry.generation); -} - -/** Generations this account's affinity entries are bound at. Test observability only. */ -export function debugCodexAffinityGenerations(accountId: string): number[] { - const generations: number[] = []; - for (const affinities of threadAccountMap.values()) { - for (const entry of affinities.values()) { - if (entry.accountId === accountId) generations.push(entry.generation); - } - } - return generations; -} - -/** - * Advance this account's affinity entries from the generation a rejected credential - * was bound under to the generation its own refresh produced. - * - * A 401 refresh-and-replay keeps the request on the same account, but the CAS write - * moves the credential from G to G+1, and {@link isThreadAffinityGenerationLive} - * demands exact equality — so without this the entry the replay just preserved is - * dead on the next request. Not quarantining an account is not the same as keeping - * its affinity. - * - * Lineage is proven by the CALLER, which must pass only a generation its own refresh - * produced. Re-deriving it here from `replacedAt` cannot work: the caller reads that - * field after the refresh and this function would re-read the same record, so the - * comparison is tautological and an external replacement passes it. An external - * replacement must retire the affinity, because that credential may belong to a - * different upstream identity. - */ -export function handOffThreadAffinityGeneration( - accountId: string, - fromGeneration: number, - toGeneration: number, -): boolean { - if (accountId === MAIN_CODEX_ACCOUNT_ID) return false; - if (toGeneration !== fromGeneration + 1) return false; - const record = readCodexAccountRecord(accountId); - if (!record?.credential || record.deletedAt != null) return false; - if (record.generation !== toGeneration) return false; - let handedOff = false; - for (const affinities of threadAccountMap.values()) { - for (const entry of affinities.values()) { - if (entry.accountId !== accountId || entry.generation !== fromGeneration) continue; - entry.generation = toGeneration; - handedOff = true; - } - } - return handedOff; -} - -function pruneExpiredThreadAffinities(now: number): void { - for (const [threadId, affinities] of threadAccountMap) { - for (const [scope, entry] of affinities) { - if (isThreadAffinityExpired(entry, now) && affinities.delete(scope)) { - threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); - } - } - if (affinities.size === 0) threadAccountMap.delete(threadId); - } -} - -function pruneLruThreadAffinities(): void { - if (threadAffinityEntryCount() <= CODEX_THREAD_AFFINITY_MAX_ENTRIES) return; - while (threadAffinityEntryCount() > CODEX_THREAD_AFFINITY_MAX_ENTRIES) { - let oldestThreadId: string | null = null; - let oldestScope: ThreadAffinityScope | null = null; - let oldestLastUsedAt = Number.POSITIVE_INFINITY; - let oldestIsDetour = false; - for (const [threadId, affinities] of threadAccountMap) { - for (const [scope, entry] of affinities) { - const candidateIsDetour = isModelDetourAffinityScope(scope); - if ( - (candidateIsDetour && !oldestIsDetour) - || (candidateIsDetour === oldestIsDetour && entry.lastUsedAt < oldestLastUsedAt) - ) { - oldestThreadId = threadId; - oldestScope = scope; - oldestLastUsedAt = entry.lastUsedAt; - oldestIsDetour = candidateIsDetour; - } - } - } - if (!oldestThreadId || !oldestScope) return; - deleteThreadAffinityForScope(oldestThreadId, oldestScope); - } -} - -function bindThreadAffinityForScope( - threadId: string, - accountId: string, - now: number, - scope: ThreadAffinityScope, -): void { - if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; - const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); - if (accountId !== MAIN_CODEX_ACCOUNT_ID && (!record?.credential || record.deletedAt != null)) return; - pruneExpiredThreadAffinities(now); - const affinities = threadAccountMap.get(threadId) ?? new Map(); - const previous = affinities.get(scope); - affinities.set(scope, { - accountId, - generation: accountId === MAIN_CODEX_ACCOUNT_ID ? 0 : record!.generation, - createdAt: previous?.createdAt ?? now, - lastUsedAt: now, - lastReevalAt: now, - }); - if (!previous) threadAffinityEntryTotal += 1; - threadAccountMap.set(threadId, affinities); - pruneLruThreadAffinities(); -} - -function bindThreadAffinity( - threadId: string, - accountId: string, - now: number, - quotaScope?: CodexQuotaScope, -): void { - bindThreadAffinityForScope(threadId, accountId, now, threadAffinityScope(quotaScope)); -} - -function bindModelDetourAffinity( - threadId: string, - accountId: string, - now: number, - modelId: string | undefined, - quotaScope?: CodexQuotaScope, -): void { - const scope = modelDetourAffinityScope(modelId, quotaScope); - if (scope) bindThreadAffinityForScope(threadId, accountId, now, scope); -} - -function getEligiblePoolAccounts( - config: OcxConfig, - excludeId?: string, - now = Date.now(), - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, - skipFailoverReadyCandidates = false, -): readonly string[] { - const excludedPlans = excludedCodexPoolPlanKeys(config); - const ids = (config.codexAccounts ?? []) - .filter(account => isSelectableCodexPoolAccount(account) - && account.id !== excludeId - && !isCodexAccountPaused(config, account.id) - && !isCodexAccountPlanExcluded(config, account.id, excludedPlans) - && !isAccountNeedsReauth(account.id) - && (!skipFailoverReadyCandidates || !shouldFailover(config, account.id, now))) - .filter(account => getCodexQuotaHealthSnapshot(account.id, quotaScope, now) === null) - .filter(account => !isCodexAccountSoftAvoided(account.id, now)) - .filter(account => !isCodexQuotaAvoided(account.id, quotaScope, now)) - .filter(account => !isCodexPoolRefreshCooling(account.id, now)) - .filter(account => isCodexAccountUsable(config, account.id, selectionOptions)) - .map(account => account.id); - // The main Codex account is not stored in config.codexAccounts; include it as a - // first-class rotation candidate when its read-only token is usable (Option A). - if ( - excludeId !== MAIN_CODEX_ACCOUNT_ID - && !isCodexAccountPaused(config, MAIN_CODEX_ACCOUNT_ID) - && (!isAccountNeedsReauth(MAIN_CODEX_ACCOUNT_ID) || hasMainAccountRefreshGrant()) - && getCodexQuotaHealthSnapshot(MAIN_CODEX_ACCOUNT_ID, quotaScope, now) === null - && !isCodexAccountSoftAvoided(MAIN_CODEX_ACCOUNT_ID, now) - // The main login is not in `config.codexAccounts`, so it never passes through the - // filters above and this is the only place an avoidance window can exclude it. Without - // this the window a refusal announced applies to the pool but not to the account that - // earned it: the cooldown caps at fifteen minutes, the window runs up to six hours, and - // in between the main account returns as a first-class candidate. - && !isCodexQuotaAvoided(MAIN_CODEX_ACCOUNT_ID, quotaScope, now) - && !isCodexPoolRefreshCooling(MAIN_CODEX_ACCOUNT_ID, now) - && (!skipFailoverReadyCandidates || !shouldFailover(config, MAIN_CODEX_ACCOUNT_ID, now)) - && isCodexAccountUsable(config, MAIN_CODEX_ACCOUNT_ID, selectionOptions) - ) { - ids.unshift(MAIN_CODEX_ACCOUNT_ID); - } - // Single choke point for selection order: every strategy, failover, and preview - // reaches the pool through here, so tiering applies once rather than per picker. - // Eligibility above is unchanged — this only narrows an already-eligible list. - return selectPriorityTier( - ids, - codexAccountPriorityLookup(config), - id => hasCodexQuotaHeadroom(config, id, selectionOptions, now), - pinnedCodexAccountId(config), - ); -} - -function listEligibleCodexAccountIds( - config: OcxConfig, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): readonly string[] { - return getEligiblePoolAccounts(config, undefined, now, quotaScope, selectionOptions); -} - -/** Shared reset timestamps are not evidence for independent model-quota groups. */ -function accountPoolStrategyForScope(config: OcxConfig, quotaScope?: CodexQuotaScope) { - const strategy = normalizeCodexAccountPoolStrategy(config.accountPoolStrategy); - return strategy === "reset-first" && isIndependentCodexQuotaScope(quotaScope) ? "quota" : strategy; -} - -function stickyLimitForConfig(config: OcxConfig): number { - return normalizeAccountPoolStickyLimit(config.accountPoolStickyLimit); -} - -/** - * Whether an account still has quota to give under the auto-switch threshold. - * - * Fill-first and the priority tier filter share this predicate, and share both of - * its escape hatches. A disabled threshold means only health, pause, and reauth - * may drain an account; unknown usage is a guess, so it must neither force - * fill-first off the active account nor drain a tier that was simply never - * primed. A genuinely exhausted account 429s into cooldown and leaves - * eligibility on its own. - */ -function hasCodexQuotaHeadroom( - config: OcxConfig, - accountId: string, - selectionOptions?: CodexAccountUsabilityOptions, - now: number = Date.now(), -): boolean { - const threshold = config.autoSwitchThreshold ?? 80; - if (threshold <= 0) return true; - const usage = computeCodexUsageScore( - getAccountQuota(accountId), - getPoolAccountPlanForSelection(config, accountId, selectionOptions), - now, - ); - if (isUnknownUsage(usage)) return true; - return usage < threshold; -} - -/** - * Is a live binding held for its prompt cache? - * - * Unset means yes. Cache affinity shipped as an opt-in flag (#4292) and then #4546 measured - * what the default costs: a pool whose accounts all sit in the 80-99% band hands a bound - * conversation from account to account, and because provider prompt caches are account-isolated - * every hop re-sends the entire prefix. An install that has never heard of this flag is exactly - * the install that gets hurt by it, so the protection cannot be something you have to find. - * - * `false` restores capacity-first routing byte-for-byte. It is a real choice -- a pinned thread - * on a busy account pays latency -- and it stays available; it is just no longer the default. - */ -function isCacheAffinityEnabled(config: OcxConfig): boolean { - return config.pool?.cacheAffinity !== false; + forgetManualPreference(accountId); } +export function reconcileCodexRoutingHealth(context: GenerationContext): number { + if (isHealthGenerationReconciled(context.generation)) return 0; + const removed = pruneHealthAccountsForContext(context.codexAccountIds); + // Sweep preferences the same way, for the account set this generation actually has. The + // delete path above is the direct route; this is the one that catches an account removed + // by an edit the runtime never saw. Deliberately not counted in `removed`, which reports + // health rows. + forgetRoutingPreferencesOutside(context.codexAccountIds); + commitHealthReconcile(context.generation, context.codexAccountIds); + return removed; +} /** * Is a transient failure streak the ONLY thing standing between this thread and its account? * @@ -1824,7 +290,7 @@ function isTransientHoldExpired(entry: ThreadAffinityEntry, now: number): boolea * that chance away. */ function isTransientHoldSpentForAccount(threadId: string, accountId: string, now: number): boolean { - const affinities = threadAccountMap.get(threadId); + const affinities = getThreadAffinityScopes(threadId); if (!affinities) return false; let matched = false; for (const entry of affinities.values()) { @@ -1938,475 +404,6 @@ function pickLineageServingAccount( return null; } -/** - * Move one scope's binding from the pre-#4546 RAW parent key onto the key this thread uses now. - * - * Bindings and the key that derives them are process-local, so an ordinary restart already - * discards every binding and there is nothing to migrate. The case this exists for is the - * narrow one: a code swap under a live conversation, where the map still holds entries made by - * the old rule. Rebinding those cold is precisely the defect the lineage work exists to prevent, - * so the conversation keeps its account and the legacy entry is retired in the same step. - * - * One way, once. The legacy entry is deleted even when it was dead on arrival, because nothing - * can reach it again under the new rule and an orphan only spends an LRU slot a live - * conversation needs. Only the account moves: a transient hold describes a failure happening - * right now, and the ordinary path re-derives it on this very request. - */ -function adoptLegacyAffinityForScope( - threadId: string, - legacyKey: string, - now: number, - scope: ThreadAffinityScope, -): void { - if (getThreadAffinityForScope(threadId, scope) !== undefined) return; - const legacy = getThreadAffinityForScope(legacyKey, scope); - if (legacy === undefined) return; - if (!isThreadAffinityExpired(legacy, now) && isThreadAffinityGenerationLive(legacy)) { - bindThreadAffinityForScope(threadId, legacy.accountId, now, scope); - } - deleteThreadAffinityForScope(legacyKey, scope); -} - -/** Both lanes of the legacy migration: the ordinary binding and this request's model detour. */ -function adoptLegacyLineageAffinity( - threadId: string, - lineage: CodexThreadLineage | undefined, - now: number, - quotaScope?: CodexQuotaScope, - modelId?: string, -): void { - const legacyKey = lineage?.legacyConversationKey; - if (legacyKey === undefined || legacyKey === threadId) return; - adoptLegacyAffinityForScope(threadId, legacyKey, now, threadAffinityScope(quotaScope)); - const detourScope = modelDetourAffinityScope(modelId, quotaScope); - if (detourScope) adoptLegacyAffinityForScope(threadId, legacyKey, now, detourScope); -} - -/** Earliest future shared short/weekly reset; missing evidence and ties use usage order. */ -function pickResetFirstCodexAccount( - config: OcxConfig, - ids: readonly string[], - now: number, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - const available = ids.filter(id => hasCodexQuotaHeadroom(config, id, selectionOptions, now)); - if (available.length === 0) return pickLowestUsageAmong(config, ids, selectionOptions, now); - let earliest = Number.POSITIVE_INFINITY; - let candidates: string[] = []; - for (const id of available) { - const quota = getAccountQuota(id); - const resets = [quota?.shortResetAt, quota?.weeklyResetAt] - .filter((reset): reset is number => typeof reset === "number" && Number.isFinite(reset)) - .map(resetAtToMs) - .filter(reset => reset > now); - const next = Math.min(...resets); - if (next < earliest) { - earliest = next; - candidates = [id]; - } else if (next === earliest) candidates.push(id); - } - return pickLowestUsageAmong(config, candidates, selectionOptions, now); -} - -/** - * Fill-first: keep selectable active under threshold; otherwise advance to the next - * eligible id in stable sorted order after the current active (wrapping). - */ -function pickFillFirstCodexAccount( - config: OcxConfig, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - const eligible = listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions); - if (eligible.length === 0) return null; - - const active = getEffectiveActiveCodexAccountId(config); - if (active && eligible.includes(active) && hasCodexQuotaHeadroom(config, active, selectionOptions, now)) { - return active; - } - - return pickNextFillFirstCodexAccount(config, active ?? null, eligible, now, selectionOptions); -} - -/** Next eligible account in stable order after `afterId` (wrapping). */ -function pickNextFillFirstCodexAccount( - config: OcxConfig, - afterId: string | null, - eligible: readonly string[] = listEligibleCodexAccountIds(config, Date.now()), - now = Date.now(), - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - if (eligible.length === 0) return null; - const ordered = [...eligible].sort((a, b) => a.localeCompare(b)); - if (!afterId) { - // Prefer an under-threshold account when starting with no active cursor. - for (const id of ordered) { - if (hasCodexQuotaHeadroom(config, id, selectionOptions, now)) return id; - } - return ordered[0] ?? null; - } - - const allConfigured = [ - ...(isCodexAccountUsable(config, MAIN_CODEX_ACCOUNT_ID, selectionOptions) || afterId === MAIN_CODEX_ACCOUNT_ID - ? [MAIN_CODEX_ACCOUNT_ID] - : []), - ...(config.codexAccounts ?? []).filter(account => !account.isMain).map(account => account.id), - ]; - const stableAll = [...new Set(allConfigured)].sort((a, b) => a.localeCompare(b)); - const startIdx = stableAll.indexOf(afterId); - if (startIdx < 0) { - for (const id of ordered) { - if (hasCodexQuotaHeadroom(config, id, selectionOptions, now)) return id; - } - return ordered[0] ?? null; - } - - // Skip successors that are also at/above threshold (known drained usage). - let fallback: string | null = null; - for (let step = 1; step <= stableAll.length; step++) { - const candidate = stableAll[(startIdx + step) % stableAll.length]!; - if (!eligible.includes(candidate)) continue; - if (!fallback) fallback = candidate; - if (hasCodexQuotaHeadroom(config, candidate, selectionOptions, now)) return candidate; - } - return fallback ?? ordered[0] ?? null; -} - -/** - * Unbound new-session pick for round-robin / fill-first. Returns null to fall through - * to the legacy quota path (or when the strategy is quota). - * - * When `commit` is true (resolve path), advances RR state. `commitSharedActive` - * and `commitAffinity` independently control the two cross-request side effects: - * model-scoped entitlement selection can bind a new task without replacing an - * existing task binding or global active choice. Preview remains a dry-run peek. - * - * Automatic strategy picks never sync-write config; only manual selection persists active. - * - * Known limitation (follow-up): when a subagent preview peeks an RR account and the request - * then falls back to a non-Codex provider, the ring is not reserved/committed. Prefer seeding - * the peeked account if that path becomes load-bearing. - */ -function pickUnboundStrategyAccount( - config: OcxConfig, - threadId: string | null, - now: number, - commit: boolean, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, - commitSharedActive = commit, - commitAffinity = commit, -): string | null { - const strategy = accountPoolStrategyForScope(config, quotaScope); - if (strategy === "quota") return null; - const poolKey = codexPoolKeyForScope(quotaScope); - - let picked: string | null = null; - if (strategy === "round-robin") { - const eligible = listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions); - const limit = stickyLimitForConfig(config); - if (!commit) { - return peekRoundRobinAccount(poolKey, eligible, limit); - } - picked = pickRoundRobinAccount(poolKey, eligible, limit); - if (!picked) return null; - if (commitSharedActive) { - if (!isIndependentCodexQuotaScope(quotaScope) - && !manualPreferenceBlocks(codexPoolKeyForScope(quotaScope), picked)) { - rememberActiveCodexAccount(config, picked); - } - } - if (commitAffinity && threadId) bindThreadAffinity(threadId, picked, now, quotaScope); - notePoolRotationSuccess(poolKey, picked, limit); - return picked; - } - - if (strategy === "fill-first" || strategy === "reset-first") { - picked = strategy === "reset-first" - ? pickResetFirstCodexAccount(config, listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions), now, selectionOptions) - : pickFillFirstCodexAccount(config, now, quotaScope, selectionOptions); - if (!picked) return null; - if (commitSharedActive) { - if (!isIndependentCodexQuotaScope(quotaScope) - && !manualPreferenceBlocks(codexPoolKeyForScope(quotaScope), picked)) { - rememberActiveCodexAccount(config, picked); - } - } - if (commitAffinity && threadId) bindThreadAffinity(threadId, picked, now, quotaScope); - return picked; - } - - return null; -} - -export function getPoolAccountPlan(config: OcxConfig, accountId: string): string | undefined { - if (accountId === MAIN_CODEX_ACCOUNT_ID) return getMainAccountPlan(); - return (config.codexAccounts ?? []) - .find(account => isSelectableCodexPoolAccount(account) && account.id === accountId)?.plan; -} - -/** Selection-only main routing must not lazily read the fenced native credential for its plan. */ -function getPoolAccountPlanForSelection( - config: OcxConfig, - accountId: string, - selectionOptions?: CodexAccountUsabilityOptions, -): string | undefined { - if (accountId === MAIN_CODEX_ACCOUNT_ID && selectionOptions?.nativeMainSelectionOnly === true) { - return undefined; - } - return getPoolAccountPlan(config, accountId); -} - -/** Shared routing state must ignore a request-scoped entitlement roster. */ -function sharedStateSelectionOptions( - selectionOptions?: CodexAccountUsabilityOptions, -): Pick< - CodexAccountUsabilityOptions, - "nativeMainSelectionOnly" | "isMainAccountTokenLive" -> | undefined { - if (!selectionOptions) return undefined; - return { - ...(selectionOptions.nativeMainSelectionOnly !== undefined - ? { nativeMainSelectionOnly: selectionOptions.nativeMainSelectionOnly } - : {}), - ...(selectionOptions.isMainAccountTokenLive - ? { isMainAccountTokenLive: selectionOptions.isMainAccountTokenLive } - : {}), - }; -} - -function pickLowerUsageAccount( - config: OcxConfig, - active: string, - activeUsage: number, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, - skipFailoverReadyCandidates = false, -): string { - let best = active; - let bestUsage = activeUsage; - for (const id of getEligiblePoolAccounts( - config, - active, - now, - quotaScope, - selectionOptions, - skipFailoverReadyCandidates, - )) { - const usage = computeCodexUsageScore( - getAccountQuota(id), - getPoolAccountPlanForSelection(config, id, selectionOptions), - now, - ); - if (usage < bestUsage) { - best = id; - bestUsage = usage; - } - } - return best; -} - -/** Coolest account in an already-selected candidate list; first index wins ties. */ -function pickLowestUsageAmong( - config: OcxConfig, - ids: readonly string[], - selectionOptions?: CodexAccountUsabilityOptions, - now: number = Date.now(), -): string | null { - let best: string | null = null; - let bestUsage = Number.POSITIVE_INFINITY; - for (const id of ids) { - const usage = computeCodexUsageScore( - getAccountQuota(id), - getPoolAccountPlanForSelection(config, id, selectionOptions), - now, - ); - if (usage < bestUsage) { - best = id; - bestUsage = usage; - } - } - return best; -} - -export function pickLowestUsageCodexAccount( - config: OcxConfig, - excludeId?: string, - now = Date.now(), - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - return pickLowestUsageAmong( - config, - getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions), - selectionOptions, - now, - ); -} - -/** - * Strategy-aware alternate after a cooled/excluded account (same-request 429 retry - * and active promotion). Quota keeps lowest-usage; fill-first advances stable order; - * round-robin takes the next ring pick (caller should have noted the failure). - */ -export function pickAlternateCodexAccount( - config: OcxConfig, - excludeId: string, - now = Date.now(), - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - const strategy = accountPoolStrategyForScope(config, quotaScope); - // The exclusion is passed into eligibility rather than post-filtered off its - // result: when the excluded account is the only healthy member of the top - // tier, the tier walk must be free to descend instead of selecting that tier - // and then handing back an empty list. - if (strategy === "round-robin") { - const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); - return pickRoundRobinAccount(codexPoolKeyForScope(quotaScope), eligible, stickyLimitForConfig(config)); - } - if (strategy === "fill-first") { - const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); - return pickNextFillFirstCodexAccount(config, excludeId, eligible, now, selectionOptions); - } - if (strategy === "reset-first") { - return pickResetFirstCodexAccount(config, getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions), now, selectionOptions); - } - return pickLowestUsageCodexAccount(config, excludeId, now, quotaScope, selectionOptions); -} - -/** - * The account {@link pickAlternateCodexAccount} WOULD return, without returning it. - * - * Only the round-robin branch has a side effect -- `pickRoundRobinAccount` commits the pick and - * advances the ring -- so every other strategy delegates rather than growing a second copy of - * the selection rule that could drift from it. - * - * This exists because preview and resolve have to agree on the FIRST transient detour, not just - * on later ones. Preview feeds subagent model-availability scoring, so a preview that reported - * the bound account while resolve was about to serve from a cool sibling could retire a model - * over usage the request would never have touched. - */ -function peekAlternateCodexAccount( - config: OcxConfig, - excludeId: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - if (accountPoolStrategyForScope(config, quotaScope) === "round-robin") { - const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); - return peekRoundRobinAccount(codexPoolKeyForScope(quotaScope), eligible, stickyLimitForConfig(config)); - } - return pickAlternateCodexAccount(config, excludeId, now, quotaScope, selectionOptions); -} - -/** Effective active: automatic runtime cursor, else operator/persisted selection. */ -/** - * Unspent operator selections, keyed by pool scope. - * - * Codex has no account-side equivalent of the Anthropic `selectionRevision`, so staleness - * cannot be detected by comparing values: a pool-driven promote legitimately moves the - * persisted active account, and reading that as staleness would silently spend the - * operator's one-shot. Invalidation is keyed to the OPERATOR path instead — another manual - * selection, the account leaving the pool, or a successful dispatch on it. - */ -const manualPreference = new Map(); - -/** - * Spend the one-shot for a pool scope once a dispatch on that account actually succeeded. - * This is the Codex analogue of `commitAnthropicSelectionRouting`, which Codex lacks. - * - * Wiring this BEFORE the guard below is not a style choice. Measured: with the guard in - * place and no consume site, the first manual selection freezes the automatic cursor - * permanently and 15 of 69 rotation tests fail. - */ -function consumeManualPreference(accountId: string, poolKey: string): void { - if (manualPreference.get(poolKey) === accountId) manualPreference.delete(poolKey); -} - -/** - * Drop an account's preference in every scope. Pause and exclusion do not route through - * `resetCodexRoutingForManualSelection`, so without this a preference could outlive the - * account it names and keep suppressing the automatic cursor. - */ -function forgetManualPreference(accountId: string): void { - for (const [poolKey, preferred] of manualPreference) { - if (preferred === accountId) manualPreference.delete(poolKey); - } -} - -/** - * True while an unspent operator selection for this scope names a DIFFERENT account than - * the automatic pick about to be recorded. - * - * Callers pass their own scope: an independent quota scope keeps its own entry and must - * never read the shared one. The failover promote does NOT consult this — see its call - * site for why. - */ -function manualPreferenceBlocks(poolKey: string, accountId: string): boolean { - const preferred = manualPreference.get(poolKey); - return preferred !== undefined && preferred !== accountId; -} - -export function getEffectiveActiveCodexAccountId(config: OcxConfig): string | undefined { - return runtimeActiveCodexAccountId ?? config.activeCodexAccountId; -} - -/** - * Whether the account routing is currently on is there because an operator asked - * for it, rather than because a strategy landed on it. Surfaces read this instead - * of comparing the stored pin themselves, which would report a pin that a later - * automatic pick has already moved past. - */ -export function isEffectiveCodexAccountPinned(config: OcxConfig): boolean { - const pinned = pinnedCodexAccountId(config); - return pinned !== undefined && pinned === getEffectiveActiveCodexAccountId(config); -} - -/** - * Automatic strategy / failover cursor only — never mutates `config.activeCodexAccountId` - * so an unrelated `saveConfig` cannot persist transient rotation as operator selection. - */ -function rememberActiveCodexAccount(_config: OcxConfig, accountId: string): void { - runtimeActiveCodexAccountId = accountId; -} - -/** - * End the manual pin when routing moves to a different account. Returns whether - * the pin changed so the caller can fold it into a write it was already making. - */ -function releaseCodexAccountPinFor(config: OcxConfig, accountId: string): boolean { - const pinned = pinnedCodexAccountId(config); - if (pinned === undefined || pinned === accountId) return false; - clearCodexAccountPin(config); - return true; -} - -/** Persist operator (or quota-strategy) active selection to config + disk. */ -function setActiveCodexAccount(config: OcxConfig, accountId: string): void { - runtimeActiveCodexAccountId = undefined; - const releasedPin = releaseCodexAccountPinFor(config, accountId); - if (config.activeCodexAccountId === accountId && !releasedPin) return; - config.activeCodexAccountId = accountId; - saveConfigPreservingClaudeCode(config); -} - -/** Quota strategy persists; RR/fill-first keep a process-local cursor only. */ -function promoteActiveCodexAccount(config: OcxConfig, accountId: string): void { - if (normalizeCodexAccountPoolStrategy(config.accountPoolStrategy) === "quota") { - setActiveCodexAccount(config, accountId); - return; - } - // Runtime-only, like the cursor itself: a caller that persists (pause, delete) - // saves this release with its own write; a transient failover does not, so the - // pin survives a restart that also clears the failure history behind it. - releaseCodexAccountPinFor(config, accountId); - rememberActiveCodexAccount(config, accountId); -} - /** * Reconcile the effective active account after an administrative exclusion such as pause. * The operator's persisted selection is cleared when it names the excluded account; quota @@ -2432,54 +429,12 @@ export function reconcileCodexActiveAfterExclusion( clearCodexAccountPin(config, excludedAccountId); if (!wasEffective) return getEffectiveActiveCodexAccountId(config) ?? null; - runtimeActiveCodexAccountId = undefined; + forgetRuntimeActiveCodexAccount(); const fallback = pickAlternateCodexAccount(config, excludedAccountId, now); if (fallback) promoteActiveCodexAccount(config, fallback); return fallback; } -function isUnknownUsage(usage: number): boolean { - return usage >= CODEX_UNKNOWN_USAGE_SCORE; -} - -/** - * Move an unbound request back up when a higher tier regains headroom — the - * weekly-reset case. Returns null when nothing should change. - * - * Downward moves are deliberately left to {@link applyQuotaAutoSwitch}: this only - * fires when the tier filter has already excluded `active`, and only toward a - * tier that strictly outranks it. Threads bound by affinity never reach here. - */ -function pickPriorityPreemption( - config: OcxConfig, - active: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, -): string | null { - const eligible = getEligiblePoolAccounts(config, undefined, now, quotaScope, selectionOptions); - if (eligible.length === 0 || eligible.includes(active)) return null; - const pinned = pinnedCodexAccountId(config); - // A live pin already lowered the tier ceiling; never preempt past an explicit - // operator choice. Same liveness test the tier filter applies, so preview and - // resolve agree even before the pin is garbage-collected. - if ( - pinned !== undefined - && eligible.includes(pinned) - && hasCodexQuotaHeadroom(config, pinned, selectionOptions, now) - ) return null; - const priorityOf = codexAccountPriorityLookup(config); - if (priorityOf(eligible[0]!) <= priorityOf(active)) return null; - // Members without headroom are in the tier only because a sibling has some; - // picking one would hand the request straight back to a drained account. - return pickLowestUsageAmong( - config, - eligible.filter(id => hasCodexQuotaHeadroom(config, id, selectionOptions, now)), - selectionOptions, - now, - ); -} - /** * Release a pin whose account is durably drained. "Use this account now" ends * when the account crosses the auto-switch threshold or stops being selectable @@ -2514,107 +469,6 @@ function releaseDrainedCodexAccountPin( saveConfigPreservingClaudeCode(config); } -function applyQuotaAutoSwitch( - config: OcxConfig, - active: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, - commitSharedSelection = true, -): string { - const threshold = config.autoSwitchThreshold ?? 80; - if (threshold <= 0) return active; - const quota = getAccountQuota(active); - const activeUsage = computeCodexUsageScore( - quota, - getPoolAccountPlanForSelection(config, active, selectionOptions), - now, - ); - // Unknown usage is not evidence that a user's explicit selection crossed the - // threshold. Wait for quota priming instead of rotating among guesses. - if (isUnknownUsage(activeUsage)) return active; - if (activeUsage < threshold) return active; - const best = pickLowerUsageAccount(config, active, activeUsage, now, quotaScope, selectionOptions); - if (best !== active) { - if (commitSharedSelection && !isIndependentCodexQuotaScope(quotaScope)) { - setActiveCodexAccount(config, best); - } - return best; - } - - return active; -} - -function shouldFailover(config: OcxConfig, accountId: string, now: number): boolean { - const threshold = config.upstreamFailoverThreshold ?? 3; - if (threshold <= 0) return false; - dropSpentCredentialFailure(accountId); - const health = upstreamHealth.get(accountId); - if (health?.lastFailureAt && now - health.lastFailureAt > CODEX_FAILURE_WINDOW_MS) return false; - return !!health && health.consecutiveFailures >= threshold; -} - -function isHealthySharedCodexSelection( - config: OcxConfig, - accountId: string, - now: number, - quotaScope: CodexQuotaScope | undefined, - selectionOptions: CodexAccountUsabilityOptions | undefined, -): boolean { - return isCodexAccountSelectable(config, accountId, now, quotaScope, selectionOptions) - && hasCodexQuotaHeadroom(config, accountId, selectionOptions, now) - && !shouldFailover(config, accountId, now); -} - -function strategySelectionOptionsForModelDetour( - config: OcxConfig, - now: number, - quotaScope: CodexQuotaScope | undefined, - selectionOptions: CodexAccountUsabilityOptions | undefined, -): CodexAccountUsabilityOptions | undefined { - if (selectionOptions?.modelEligibleAccountIds === undefined) return selectionOptions; - const sharedSelectionOptions = sharedStateSelectionOptions(selectionOptions) ?? {}; - return { - ...selectionOptions, - modelEligibleAccountIds: new Set( - [...selectionOptions.modelEligibleAccountIds].filter(accountId => - isHealthySharedCodexSelection( - config, - accountId, - now, - quotaScope, - sharedSelectionOptions, - ) - ), - ), - }; -} - -function applyFailureFailover( - config: OcxConfig, - active: string, - now: number, - quotaScope?: CodexQuotaScope, - selectionOptions?: CodexAccountUsabilityOptions, - commitSharedSelection = true, -): string { - if (!shouldFailover(config, active, now)) return active; - const best = pickAlternateCodexAccount(config, active, now, quotaScope, selectionOptions); - if (best) { - // The scope still routes away from the failing account — that is this request's - // own decision — but an independent one must not persist a new shared active - // account. recordCodexUpstreamOutcome only suppresses the promotion it makes at - // the moment of the failure; the streak outlives the soft avoid, so a later - // scoped resolve reaches here with the streak still tripped and would otherwise - // move the shared cursor after all. - if (commitSharedSelection && !isIndependentCodexQuotaScope(quotaScope)) { - promoteActiveCodexAccount(config, best); - } - return best; - } - return active; -} - export function resolveCodexAccountForThread( threadId: string | null, config: OcxConfig, @@ -2661,7 +515,7 @@ function carriesQuotaRefusal(health: CodexUpstreamHealth | undefined): boolean { * quota group, so a spent Spark window still cannot displace the same thread's Terra binding. */ function hasUnrecoveredCodexQuotaRefusal(accountId: string, quotaScope?: CodexQuotaScope): boolean { - if (carriesQuotaRefusal(upstreamHealth.get(accountId))) return true; + if (carriesQuotaRefusal(getAccountHealth(accountId))) return true; return quotaScope !== undefined && carriesQuotaRefusal(scopedHealthFor(accountId, quotaScope)); } @@ -3408,6 +1262,7 @@ export function resolveCodexAccountForThreadDetailed( return { status: "selected", accountId: active, affinity: affinityAfterRelease(threadId, releaseReason) }; } + export function recordCodexUpstreamOutcome( config: OcxConfig, accountId: string | null, @@ -3423,7 +1278,7 @@ export function recordCodexUpstreamOutcome( } if (!accountId) return; const writerGeneration = meta.writerGeneration ?? captureConfigGeneration(); - if (writerGeneration < lastReconciledGeneration && !liveHealthAccountIds.has(accountId)) return; + if (!isHealthAccountAdmissible(accountId, writerGeneration)) return; const now = meta.now ?? Date.now(); const outcomeClass = classifyCodexUpstreamOutcome(outcome, meta.denial); // Reject retired quota evidence before stale-credential cleanup or any shared mutation. @@ -3468,12 +1323,12 @@ export function recordCodexUpstreamOutcome( if (Object.keys(retained).length > 1) setScopedHealth(accountId, quotaScope, retained); else deleteScopedHealth(accountId, quotaScope); } - const current = upstreamHealth.get(accountId); + const current = getAccountHealth(accountId); const cooldownUntil = getCodexAccountCooldownUntil(accountId, now); // A leased probe that is still on its own cooldown generation proves the // account recovered: clear the hard cooldown outright (#433). if (cooldownUntil && probeMayClearCooldown(current, meta)) { - upstreamHealth.delete(accountId); + deleteAccountHealth(accountId); return; } // Owning probe on a stale generation: the lease is done, but a newer 429 @@ -3485,7 +1340,7 @@ export function recordCodexUpstreamOutcome( if (failoverEnabled && current && current.consecutiveFailures >= 2) { const consecutiveSuccesses = (current.consecutiveSuccesses ?? 0) + 1; if (consecutiveSuccesses < 2) { - upstreamHealth.set(accountId, { + setAccountHealth(accountId, { ...base!, ...preserved, consecutiveSuccesses, @@ -3495,14 +1350,14 @@ export function recordCodexUpstreamOutcome( } // Level 1 clears immediately; escalated accounts need two consecutive healthy terminals. // Hard quota cooldown intentionally survives either recovery path. - if (cooldownUntil) upstreamHealth.set(accountId, { consecutiveFailures: 0, ...preserved }); - else upstreamHealth.delete(accountId); + if (cooldownUntil) setAccountHealth(accountId, { consecutiveFailures: 0, ...preserved }); + else deleteAccountHealth(accountId); return; } if (outcomeClass === "caller") { // A 4xx does not change account health, but it does conclude an in-flight // probe — otherwise the lease would never be handed back. - const current = upstreamHealth.get(accountId); + const current = getAccountHealth(accountId); const scopedProbe = meta.probeQuotaScope ? scopedHealthFor(accountId, meta.probeQuotaScope) : undefined; @@ -3510,7 +1365,7 @@ export function recordCodexUpstreamOutcome( setScopedHealth(accountId, meta.probeQuotaScope, withProbeLeaseReleased(scopedProbe, now)); } if (ownsProbeLease(current, meta)) { - upstreamHealth.set(accountId, withProbeLeaseReleased(current!, now)); + setAccountHealth(accountId, withProbeLeaseReleased(current!, now)); } return; } @@ -3521,7 +1376,7 @@ export function recordCodexUpstreamOutcome( // it and must not happen (#914). Conclude any owned probe lease, record the // failure under the (provider, host) ledger when one is named, and leave // account health, thread affinity, and the active account untouched. - const current = upstreamHealth.get(accountId); + const current = getAccountHealth(accountId); const scopedProbe = meta.probeQuotaScope ? scopedHealthFor(accountId, meta.probeQuotaScope) : undefined; @@ -3529,7 +1384,7 @@ export function recordCodexUpstreamOutcome( setScopedHealth(accountId, meta.probeQuotaScope, withProbeLeaseReleased(scopedProbe, now)); } if (ownsProbeLease(current, meta)) { - upstreamHealth.set(accountId, withProbeLeaseReleased(current!, now)); + setAccountHealth(accountId, withProbeLeaseReleased(current!, now)); } return; } @@ -3540,8 +1395,8 @@ export function recordCodexUpstreamOutcome( // Record the failure so routing stops preferring it, but do not mark it for // reauthentication and do not sweep its thread affinities: telling the user to // re-login is wrong advice that cannot fix a workspace grant. - upstreamHealth.set(accountId, { - consecutiveFailures: (upstreamHealth.get(accountId)?.consecutiveFailures ?? 0) + 1, + setAccountHealth(accountId, { + consecutiveFailures: (getAccountHealth(accountId)?.consecutiveFailures ?? 0) + 1, lastFailureStatus, lastFailureAt: now, }); @@ -3579,7 +1434,7 @@ export function recordCodexUpstreamOutcome( * Affinity sweeping needs no tag: an affinity entry already carries a credential generation and * self-invalidates on the next check, and re-adding swept entries would be a worse bug. */ - upstreamHealth.set(accountId, { + setAccountHealth(accountId, { consecutiveFailures: 1, lastFailureStatus, lastFailureAt: now, @@ -3588,7 +1443,7 @@ export function recordCodexUpstreamOutcome( ? { credentialFailureGeneration: meta.credentialGeneration } : {}), }); - quotaScopedHealth.delete(accountId); + deleteAllScopedHealth(accountId); // The reauth flag carries the same provenance, so a replacement landing after this call cannot // inherit a quarantine that was never about it. markAccountNeedsReauth(accountId, writerGeneration, meta.credentialGeneration); @@ -3649,13 +1504,13 @@ export function recordCodexUpstreamOutcome( if (scopedProbe && meta.probeQuotaScope && ownsProbeLease(scopedProbe, meta)) { setScopedHealth(accountId, meta.probeQuotaScope, withProbeLeaseReleased(scopedProbe, now)); } - const prior = upstreamHealth.get(accountId); + const prior = getAccountHealth(accountId); // Every cooldown write bumps the generation so a probe issued against the // previous cooldown can no longer clear this one (#433). const cooldownGeneration = (prior?.cooldownGeneration ?? 0) + 1; // A failed probe concludes its lease; an unrelated 429 leaves the live probe alone. const ownsLease = ownsProbeLease(prior, meta); - upstreamHealth.set(accountId, { + setAccountHealth(accountId, { consecutiveFailures: 0, lastFailureStatus, lastFailureAt: now, @@ -3695,7 +1550,7 @@ export function recordCodexUpstreamOutcome( } // transient (connect_error / timeout / 5xx) - const current = upstreamHealth.get(accountId); + const current = getAccountHealth(accountId); const scopedProbe = meta.probeQuotaScope ? scopedHealthFor(accountId, meta.probeQuotaScope) : undefined; @@ -3721,7 +1576,7 @@ export function recordCodexUpstreamOutcome( now + escalationMs, ) : undefined; - upstreamHealth.set(accountId, { + setAccountHealth(accountId, { ...preservedCooldownFields(transientBase), consecutiveFailures, lastFailureStatus, diff --git a/src/codex/routing/active-account.ts b/src/codex/routing/active-account.ts new file mode 100644 index 0000000000..e53ef06b26 --- /dev/null +++ b/src/codex/routing/active-account.ts @@ -0,0 +1,194 @@ +import { saveConfigPreservingClaudeCode } from "../../config"; +import { clearCodexAccountPin, pinnedCodexAccountId } from "../account-priority"; +import { + POOL_KEY_CODEX, + normalizeCodexAccountPoolStrategy, + seedPoolRotationAccount, +} from "../pool-rotation"; +import type { OcxConfig } from "../../types"; +import { clearThreadAccountMap } from "./thread-affinity"; +import { + NATIVE_MODEL_QUOTA_SCOPES, + codexPoolKeyForScope, + deleteAccountHealth, + deleteScopedHealth, + getAccountHealth, + isIndependentCodexQuotaScope, + listScopedHealthEntries, + preservedCooldownFields, + setAccountHealth, + setScopedHealth, + type CodexUpstreamHealth, +} from "./health-store"; + +/** + * Process-local cursor for automatic RR/fill-first (and quota-429 when not + * sync-writing) picks. Keeps unrelated `saveConfig` from persisting transient + * rotation as the operator's `activeCodexAccountId`. Manual selection clears it + * so disk/`config.activeCodexAccountId` remains authoritative. + */ +let runtimeActiveCodexAccountId: string | undefined; + +/** Manual selection resets transient routing evidence without bypassing a real 429 cooldown. */ +export function resetCodexRoutingForManualSelection(accountId: string): void { + clearThreadAccountMap(); + // Manual selection is the operator source of truth — drop any automatic runtime cursor. + runtimeActiveCodexAccountId = undefined; + // Record the pick as an unspent one-shot on the SHARED scope only. An independent scope + // gets no entry on purpose: every write site the guard protects is already skipped for + // independent scopes, so an entry there would be state nothing reads — and state nothing + // reads is what the next reader mistakes for a rule. + // + // Seeding happens ONLY here. A pool-driven promote must never create or move a preference, + // or the pool would manufacture an operator intent nobody expressed. + manualPreference.set(POOL_KEY_CODEX, accountId); + // Seed the RR ring so the next unbound new session honors the manually selected account + // under round-robin (affinity-cleared threads / null threadId). Fill-first already follows + // config.activeCodexAccountId, which the caller persists before invoking this. + seedPoolRotationAccount(POOL_KEY_CODEX, accountId); + for (const scope of new Set(Object.values(NATIVE_MODEL_QUOTA_SCOPES))) { + if (isIndependentCodexQuotaScope(scope)) { + seedPoolRotationAccount(codexPoolKeyForScope(scope), accountId); + } + } + // Quota avoidance is a preference, like the soft avoid dropped above, and an operator naming + // this account has overruled it. The hard cooldown is the part that survives. + const overrule = (health: CodexUpstreamHealth) => { + const { quotaAvoidUntil: _avoid, ...retained } = preservedCooldownFields(health); + return retained; + }; + const current = getAccountHealth(accountId); + if (current) { + const retained = overrule(current); + if (Object.keys(retained).length === 0) deleteAccountHealth(accountId); + else setAccountHealth(accountId, { consecutiveFailures: 0, ...retained }); + } + // A reset-derived refusal records its avoidance on the SCOPED map and returns before the + // account-wide entry is written, so naming the account has to reach that map too. Stopping + // at `upstreamHealth` — and returning early when it holds nothing — overruled nothing in + // the case that produces the avoidance this function exists to overrule. + for (const [scope, health] of [...(listScopedHealthEntries(accountId))]) { + const retained = overrule(health); + if (Object.keys(retained).length === 0) deleteScopedHealth(accountId, scope); + else setScopedHealth(accountId, scope, { consecutiveFailures: 0, ...retained }); + } +} + +/** Effective active: automatic runtime cursor, else operator/persisted selection. */ +/** + * Unspent operator selections, keyed by pool scope. + * + * Codex has no account-side equivalent of the Anthropic `selectionRevision`, so staleness + * cannot be detected by comparing values: a pool-driven promote legitimately moves the + * persisted active account, and reading that as staleness would silently spend the + * operator's one-shot. Invalidation is keyed to the OPERATOR path instead — another manual + * selection, the account leaving the pool, or a successful dispatch on it. + */ +const manualPreference = new Map(); + +/** + * Spend the one-shot for a pool scope once a dispatch on that account actually succeeded. + * This is the Codex analogue of `commitAnthropicSelectionRouting`, which Codex lacks. + * + * Wiring this BEFORE the guard below is not a style choice. Measured: with the guard in + * place and no consume site, the first manual selection freezes the automatic cursor + * permanently and 15 of 69 rotation tests fail. + */ +export function consumeManualPreference(accountId: string, poolKey: string): void { + if (manualPreference.get(poolKey) === accountId) manualPreference.delete(poolKey); +} + +/** + * Drop an account's preference in every scope. Pause and exclusion do not route through + * `resetCodexRoutingForManualSelection`, so without this a preference could outlive the + * account it names and keep suppressing the automatic cursor. + */ +export function forgetManualPreference(accountId: string): void { + for (const [poolKey, preferred] of manualPreference) { + if (preferred === accountId) manualPreference.delete(poolKey); + } +} + +/** + * True while an unspent operator selection for this scope names a DIFFERENT account than + * the automatic pick about to be recorded. + * + * Callers pass their own scope: an independent quota scope keeps its own entry and must + * never read the shared one. The failover promote does NOT consult this — see its call + * site for why. + */ +export function manualPreferenceBlocks(poolKey: string, accountId: string): boolean { + const preferred = manualPreference.get(poolKey); + return preferred !== undefined && preferred !== accountId; +} + +export function getEffectiveActiveCodexAccountId(config: OcxConfig): string | undefined { + return runtimeActiveCodexAccountId ?? config.activeCodexAccountId; +} + +/** + * Whether the account routing is currently on is there because an operator asked + * for it, rather than because a strategy landed on it. Surfaces read this instead + * of comparing the stored pin themselves, which would report a pin that a later + * automatic pick has already moved past. + */ +export function isEffectiveCodexAccountPinned(config: OcxConfig): boolean { + const pinned = pinnedCodexAccountId(config); + return pinned !== undefined && pinned === getEffectiveActiveCodexAccountId(config); +} + +/** + * Automatic strategy / failover cursor only — never mutates `config.activeCodexAccountId` + * so an unrelated `saveConfig` cannot persist transient rotation as operator selection. + */ +export function rememberActiveCodexAccount(_config: OcxConfig, accountId: string): void { + runtimeActiveCodexAccountId = accountId; +} + +/** + * End the manual pin when routing moves to a different account. Returns whether + * the pin changed so the caller can fold it into a write it was already making. + */ +function releaseCodexAccountPinFor(config: OcxConfig, accountId: string): boolean { + const pinned = pinnedCodexAccountId(config); + if (pinned === undefined || pinned === accountId) return false; + clearCodexAccountPin(config); + return true; +} + +/** Persist operator (or quota-strategy) active selection to config + disk. */ +export function setActiveCodexAccount(config: OcxConfig, accountId: string): void { + runtimeActiveCodexAccountId = undefined; + const releasedPin = releaseCodexAccountPinFor(config, accountId); + if (config.activeCodexAccountId === accountId && !releasedPin) return; + config.activeCodexAccountId = accountId; + saveConfigPreservingClaudeCode(config); +} + +/** Quota strategy persists; RR/fill-first keep a process-local cursor only. */ +export function promoteActiveCodexAccount(config: OcxConfig, accountId: string): void { + if (normalizeCodexAccountPoolStrategy(config.accountPoolStrategy) === "quota") { + setActiveCodexAccount(config, accountId); + return; + } + // Runtime-only, like the cursor itself: a caller that persists (pause, delete) + // saves this release with its own write; a transient failover does not, so the + // pin survives a restart that also clears the failure history behind it. + releaseCodexAccountPinFor(config, accountId); + rememberActiveCodexAccount(config, accountId); +} + +export function clearAllManualPreferences(): void { + manualPreference.clear(); +} + +export function forgetRuntimeActiveCodexAccount(): void { + runtimeActiveCodexAccountId = undefined; +} + +export function forgetRoutingPreferencesOutside(codexAccountIds: ReadonlySet): void { + for (const [poolKey, preferred] of manualPreference) { + if (codexAccountIds.has(preferred)) continue; + manualPreference.delete(poolKey); + } +} diff --git a/src/codex/routing/cooldown-math.ts b/src/codex/routing/cooldown-math.ts new file mode 100644 index 0000000000..123da5e3b1 --- /dev/null +++ b/src/codex/routing/cooldown-math.ts @@ -0,0 +1,275 @@ +import { + CODEX_EXHAUSTED_USAGE_PERCENT, + CODEX_UNKNOWN_USAGE_SCORE, + resetAtToMs, +} from "../quota"; +import { isThirtyDayOnlyCodexPlan } from "../plan"; +import type { CodexQuotaScope } from "./health-store"; + +export const CODEX_DEFAULT_QUOTA_COOLDOWN_MS = 60_000; +export const CODEX_MAX_QUOTA_COOLDOWN_MS = 24 * 60 * 60_000; +/** + * A weekly/monthly quota `resetAt` announces when the window refreshes; it is not + * a "come back after this" directive like Retry-After. Plan quota routinely frees + * up long before the advertised reset, so cap reset-derived cooldowns far below + * the Retry-After ceiling (#433). + */ +export const CODEX_MAX_RESET_DERIVED_COOLDOWN_MS = 15 * 60_000; +/** + * Ceiling on quota-refusal avoidance. Generous enough to cover a full five-hour burst window, + * tight enough that a weekly or monthly reset four days out cannot take an account out of + * rotation for the {@link CODEX_MAX_QUOTA_COOLDOWN_MS} day the Retry-After ceiling allows. + */ +export const CODEX_MAX_QUOTA_AVOID_MS = 6 * 60 * 60_000; +/** Minimum gap between probe leases for one cooled-down account. */ +export const CODEX_QUOTA_PROBE_INTERVAL_MS = 5 * 60_000; +export const CODEX_FAILURE_WINDOW_MS = 5 * 60_000; +/** + * How recently a 100% burst reading must have been OBSERVED to exclude an account when it + * carries no reset timestamp (#3425). Deliberately far tighter than the 6h disk-hydration + * horizon in `quota.ts`: shorter than any plausible five-hour burst window, so a persisted + * reading can never strand a recovered account, and long enough that a snapshot taken at + * admission is still fresh when selection reads it. + */ +export const TERMINAL_SHORT_WINDOW_FRESHNESS_MS = 5 * 60_000; +/** How long a transient failure keeps the account out of pool selection. */ +export const CODEX_TRANSIENT_SOFT_AVOID_MS = 30_000; +export const CODEX_TRANSIENT_SOFT_AVOID_ESCALATION_MS = [ + CODEX_TRANSIENT_SOFT_AVOID_MS, + 2 * 60_000, + 10 * 60_000, + 30 * 60_000, +] as const; + +export type CodexUpstreamOutcome = number | "connect_error" | "timeout" | "connect_neutral"; +export type CodexUpstreamOutcomeClass = "success" | "credential" + | "workspace" | "quota" | "transient" | "caller" | "neutral" | "unknown"; +export type CodexCooldownSource = "retry-after" | "reset-derived" | "default"; + +export type CodexUpstreamOutcomeMeta = { + retryAfter?: string | null; + resetAt?: unknown | unknown[]; + now?: number; + /** (provider, host) ledger key for account-neutral reachability failures (#914). */ + hostKey?: string; + /** + * Upstream denial evidence for a 403. A workspace/entitlement denial means the CREDENTIAL + * is fine and the account simply cannot reach this workspace, so it must not be quarantined + * for reauthentication (#1789). Absent evidence keeps the historical credential handling. + */ + denial?: "workspace" | "entitlement"; + /** Stable transport code recorded alongside a neutral host failure. */ + lastFailureCode?: string; + /** Native model selected for this request; used only for confirmed scoped quotas. */ + modelId?: string; + /** When set, clears affinity for this thread immediately on transient failure. */ + threadId?: string | null; + /** + * Suppress Pool rotation and quota/transient affinity mutations for an account-qualified + * request. Credential failures still sweep stale affinities because reauthentication is + * account-wide. + */ + fixedAccount?: boolean; + /** + * Probe lease held by this request, when it was admitted through an active + * quota cooldown. Only the outcome carrying the current lease may clear the + * cooldown (#433). + */ + probeLeaseId?: string; + /** Scope of `probeLeaseId` when it was granted against a model-scoped cooldown. */ + probeQuotaScope?: CodexQuotaScope; + /** + * Already-chosen alternate for same-request 429 retry. When set, promotion + * reuses this account instead of calling {@link pickAlternateCodexAccount} + * again (which would advance a round-robin ring twice). + */ + promoteAccountId?: string; + /** Generation captured when this routed account was selected. */ + writerGeneration?: number; + /** + * Credential generation this request's bearer was read at. Distinct from + * `writerGeneration`, which tracks the config store. + * + * A 401 that arrives after the credential was already replaced is evidence about a + * token nobody is using any more, so it must not quarantine the replacement. Absent + * means the caller cannot supply lineage and the historical unfenced handling stands. + */ + credentialGeneration?: number; +}; + +export function computeCodexUsageScore(quota: { + weeklyPercent?: number; + monthlyPercent?: number; + shortPercent?: number; + shortResetAt?: number; + shortObservedAt?: number; +} | null, plan?: unknown, now: number = Date.now()): number { + if (!quota) return CODEX_UNKNOWN_USAGE_SCORE; + const finite = (value: unknown): value is number => typeof value === "number" && Number.isFinite(value); + const longWindows = isThirtyDayOnlyCodexPlan(plan) + ? [quota.monthlyPercent] + : [quota.weeklyPercent, quota.monthlyPercent]; + const knownLong = longWindows.filter(finite); + // The short burst window only REFINES a known long-window position; it cannot stand in for + // one. A snapshot carrying just `shortPercent: 0` would otherwise score a flat 0 and make an + // account whose weekly/monthly usage is entirely unverified look like the emptiest in the + // pool, so `pickLowestUsageAmong` would send every request to it. Unknown has to stay + // unknown until a governing window is actually observed. + // + // A FULL burst window is the exception (#3029). It is not an optimistic guess about an + // unobserved window — it is a direct observation that the account cannot serve a request + // right now, whatever its monthly position turns out to be. Unknown-means-selectable is + // correct for uncertainty and wrong for a measured refusal: the account stays selected, + // `applyQuotaAutoSwitch` never fires, and the pool wedges on an exhausted credential. + if (knownLong.length === 0) { + return isTerminalShortWindow(quota, now) ? CODEX_EXHAUSTED_USAGE_PERCENT : CODEX_UNKNOWN_USAGE_SCORE; + } + const values = finite(quota.shortPercent) ? [...knownLong, quota.shortPercent] : knownLong; + return Math.max(...values); +} + +/** + * A short-only reading that proves the account is blocked NOW. + * + * Freshness is not optional. `getAccountQuota` performs no expiry check, partial updates + * carry a still-open short tuple forward, and disk hydration accepts a persisted reading for + * hours — so scoring 100 from `shortPercent` alone would keep excluding an account whose + * five-hour window has since reset. Merge no longer carries an elapsed shortResetAt, but an + * explicit incoming elapsed tuple is still stored, and a missing reset cannot be aged there. + * That is #3029 pointed the other way: the issue is that + * an exhausted account stays selected, and "a recovered account stays excluded" trades one + * unusable pool for another. + * + * A reading with no `shortResetAt` cannot be aged, so it stays unknown. The conservative + * direction here is the one that keeps an account selectable: a wrongly-selected account + * fails one request, while a wrongly-excluded one is invisible until someone reads the pool + * by hand. + * + * A missing reset can instead be aged by shortObservedAt (#3425). General updatedAt is not + * sufficient: credit-only updates preserve the old short tuple but advance that timestamp. + * Old disk snapshots without short-window provenance remain unknown. + */ +function isTerminalShortWindow( + quota: { shortPercent?: number; shortResetAt?: number; shortObservedAt?: number }, + now: number, +): boolean { + if (typeof quota.shortPercent !== "number" || !Number.isFinite(quota.shortPercent)) return false; + if (quota.shortPercent < CODEX_EXHAUSTED_USAGE_PERCENT) return false; + const resetAt = quota.shortResetAt; + if (typeof resetAt !== "number" || !Number.isFinite(resetAt) || resetAt <= 0) { + const observedAt = quota.shortObservedAt; + if (typeof observedAt !== "number" || !Number.isFinite(observedAt)) return false; + const age = now - observedAt; + return age >= 0 && age <= TERMINAL_SHORT_WINDOW_FRESHNESS_MS; + } + // Seconds and milliseconds both reach storage, so the split lives in one place next to the + // merge that also ages a stored reset instant (`resetAtToMs`, src/codex/quota.ts). + return resetAtToMs(resetAt) > now; +} + +export function classifyCodexUpstreamOutcome( + outcome: CodexUpstreamOutcome, + denial?: "workspace" | "entitlement", +): CodexUpstreamOutcomeClass { + if (outcome === "connect_neutral") return "neutral"; + if (outcome === "connect_error" || outcome === "timeout") return "transient"; + if (!Number.isFinite(outcome)) return "unknown"; + if (outcome >= 200 && outcome < 300) return "success"; + // Explicit 3xx policy (#914): a redirect response is relayed as-is and is + // never account or host health evidence — it proves the host is reachable + // and says nothing about the credential. Relayed as the neutral class so a + // stray 3xx cannot increment an account's transient streak. + if (outcome >= 300 && outcome < 400) return "neutral"; + // 401 is always a credential problem. A 403 is only a credential problem when nothing + // tells us otherwise: a workspace/entitlement denial (#1789) means the credential is valid + // and the account simply lacks access here, so quarantining it for reauth is wrong advice. + // Absent denial evidence the historical mapping stands, so the change fails safe. + if (outcome === 403 && denial !== undefined) return "workspace"; + if (outcome === 401 || outcome === 403) return "credential"; + // 402 Payment Required is treated as quota exhaustion for pool cooldown/failover + // (same-request alternate retry records this outcome for the depleted account). + if (outcome === 429 || outcome === 402) return "quota"; + if (outcome >= 400 && outcome < 500) return "caller"; + if (outcome >= 500 && outcome < 600) return "transient"; + return "unknown"; +} + +function clampCooldownMs(ms: number): number { + return Math.min(Math.max(ms, 1), CODEX_MAX_QUOTA_COOLDOWN_MS); +} + +export function parseRetryAfterMs(value: string | null | undefined, now = Date.now()): number | undefined { + const text = value?.trim(); + if (!text) return undefined; + if (/^\d+(?:\.\d+)?$/.test(text)) { + const seconds = Number(text); + if (Number.isFinite(seconds) && seconds > 0) return clampCooldownMs(Math.ceil(seconds * 1000)); + } + const timestamp = Date.parse(text); + if (!Number.isFinite(timestamp)) return undefined; + const delay = timestamp - now; + return delay > 0 ? clampCooldownMs(delay) : undefined; +} + +function resetTimestampMs(value: unknown): number | undefined { + const numeric = typeof value === "number" + ? value + : typeof value === "string" && value.trim() !== "" + ? Number(value) + : undefined; + if (typeof numeric !== "number" || !Number.isFinite(numeric) || numeric <= 0) return undefined; + return numeric < 1_000_000_000_000 ? numeric * 1000 : numeric; +} + +export function parseResetCooldownMs(resetAt: unknown | unknown[] | undefined, now = Date.now()): number | undefined { + const values = Array.isArray(resetAt) ? resetAt : [resetAt]; + let best: number | undefined; + for (const value of values) { + const timestamp = resetTimestampMs(value); + if (timestamp === undefined) continue; + const delay = timestamp - now; + if (delay <= 0) continue; + // A far-future reset must not pin the account for the full Retry-After + // ceiling: quota usually frees up well before the advertised window (#433). + const clamped = Math.min(clampCooldownMs(delay), CODEX_MAX_RESET_DERIVED_COOLDOWN_MS); + if (best === undefined || clamped < best) best = clamped; + } + return best; +} + +export function computeQuotaCooldown(meta: CodexUpstreamOutcomeMeta = {}): { + until: number; + source: CodexCooldownSource; +} { + const now = meta.now ?? Date.now(); + const retryAfterMs = parseRetryAfterMs(meta.retryAfter, now); + if (retryAfterMs !== undefined) return { until: now + retryAfterMs, source: "retry-after" }; + const resetCooldownMs = parseResetCooldownMs(meta.resetAt, now); + if (resetCooldownMs !== undefined) return { until: now + resetCooldownMs, source: "reset-derived" }; + return { until: now + CODEX_DEFAULT_QUOTA_COOLDOWN_MS, source: "default" }; +} + +/** + * When the pool should stop preferring an account after it refused on quota. + * + * The earliest window the refusal actually announced, bounded by {@link CODEX_MAX_QUOTA_AVOID_MS}, + * and never shorter than the cooldown the same refusal produced — a Retry-After directive that + * outlasts every announcement still governs. + */ +export function quotaAvoidUntilFor(meta: CodexUpstreamOutcomeMeta, now: number, cooldownUntil: number): number { + const values = Array.isArray(meta.resetAt) ? meta.resetAt : [meta.resetAt]; + let announced: number | undefined; + for (const value of values) { + const timestamp = resetTimestampMs(value); + if (timestamp === undefined) continue; + const delay = timestamp - now; + if (delay <= 0) continue; + const until = now + Math.min(delay, CODEX_MAX_QUOTA_AVOID_MS); + if (announced === undefined || until < announced) announced = until; + } + return Math.max(cooldownUntil, announced ?? 0); +} + +export function computeQuotaCooldownUntil(meta: CodexUpstreamOutcomeMeta = {}): number { + return computeQuotaCooldown(meta).until; +} diff --git a/src/codex/routing/health-store.ts b/src/codex/routing/health-store.ts new file mode 100644 index 0000000000..9c0d922b97 --- /dev/null +++ b/src/codex/routing/health-store.ts @@ -0,0 +1,402 @@ +import { isCodexAccountGenerationLive } from "../account-store"; +import { NATIVE_RESERVE_MODEL } from "../catalog/native-models"; +import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; +import { POOL_KEY_CODEX } from "../pool-rotation"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import type { OcxConfig } from "../../types"; +import type { CodexCooldownSource } from "./cooldown-math"; + +export type CodexUpstreamHealth = { + consecutiveFailures: number; + /** Consecutive healthy terminals observed while recovering from escalation level 2+. */ + consecutiveSuccesses?: number; + lastFailureStatus?: number; + lastFailureAt?: number; + /** Hard cooldown (quota 429). Survives a later 2xx; blocks auth + selection. */ + cooldownUntil?: number; + /** + * How long a quota refusal keeps selection away from this account (or this native quota + * group), as opposed to how long it is hard-blocked. + * + * The two are deliberately different lengths. {@link CODEX_MAX_RESET_DERIVED_COOLDOWN_MS} + * caps the hard cooldown at 15 minutes because a reset announcement is advisory and plan + * quota usually frees up before it — an account must stay reachable so the pool can find + * that out (#433). The window the refusal announced is not 15 minutes, though, so once the + * cooldown lapses the account is selectable again while its burst window is still spent, + * and the strategy picks it straight back: this proxy reads a weekly bar a burst limit never + * touches, so a refused account still scores as the coolest in the pool. Every request then + * earns the same 429 until the process restarts, which is the only thing that drops this map. + * + * So the announcement governs avoidance and the cap still governs blocking. Avoidance is soft + * in the {@link softAvoidUntil} sense: it reorders the pool and releases a bound thread, and + * the last-resort paths still reach the account when nothing else can serve, so one pessimistic + * announcement cannot stall routing. + */ + quotaAvoidUntil?: number; + /** When the current cooldown was recorded; origin of the probe interval clock. */ + cooldownSince?: number; + /** + * What produced the cooldown. An explicit Retry-After is a literal retry + * directive and is never probed; a quota resetAt only announces a window + * refresh, so it may be probed early (#433). + */ + cooldownSource?: CodexCooldownSource; + /** + * Bumped on every cooldown write. A probe lease records the generation it was + * issued for so a lease cannot clear a cooldown that a later 429 replaced. + */ + cooldownGeneration?: number; + /** + * Identity of the in-flight probe. A cooled-down account sends no traffic, so + * no organic 2xx can prove recovery; only the outcome carrying this id may + * clear the cooldown. + */ + probeLeaseId?: string; + /** Cooldown generation at the moment the lease was granted. */ + probeLeaseGeneration?: number; + /** Last probe grant or conclusion; paces the probe interval. */ + lastProbeAt?: number; + /** + * Soft avoid after connect_error / timeout / transient 5xx. Cleared on 2xx. + * Blocks pool selection + thread affinity reuse so a sticky session can leave a + * flaky account without throwing CodexAccountCooldownError (hard-only). + */ + softAvoidUntil?: number; + /** + * Credential generation a 401/403 quarantine was derived from (#2892 gap 4). + * + * Provenance lives ON the entry rather than in a side map keyed by account id. A side map spends + * "whatever health is current when the old credential is found dead", which deletes a later + * unrelated entry: a G1 401, then a G2 save, then a genuine G2 503 would lose the 503. Only the + * entry that carries this field can be spent, and any later write simply replaces it. + */ + credentialFailureGeneration?: number; +}; + +const upstreamHealth = new Map(); +/** + * Reset-derived 429s can describe a quota owned by one native model family, + * rather than the whole ChatGPT account. Keep those advisory cooldowns apart + * from account-wide Retry-After/default throttles and transient health. + */ +const quotaScopedHealth = new Map>(); +/** + * Spend a credential-failure health entry whose credential no longer exists (#2892 gap 4). + * + * A 401/403 describes one CREDENTIAL, not an account, and a replacement can land at any point after + * the outcome is recorded — so re-reading the store inside `recordCodexUpstreamOutcome` narrows the + * window without closing it. The reader decides instead, and it may only spend an entry that + * actually carries credential provenance: a later transient or quota write replaces the entry and + * with it the tag, so this can never delete evidence that belongs to a different failure. + */ +export function dropSpentCredentialFailure(accountId: string): void { + const health = upstreamHealth.get(accountId); + const generation = health?.credentialFailureGeneration; + if (health === undefined || generation === undefined) return; + if (isCodexAccountGenerationLive(accountId, generation)) return; + upstreamHealth.delete(accountId); +} +let lastReconciledGeneration = 0; +let liveHealthAccountIds = new Set(); + +/** + * Native Codex quota groups known to be independent upstream. Keep the mapping + * deliberately conservative: unlisted models share the normal native group. + * Add a new explicit group here only when its independent upstream quota is + * confirmed, so shared limits never receive cross-model bypasses. + */ +export type CodexQuotaScope = "shared" | "reserve"; + + +export const NATIVE_MODEL_QUOTA_SCOPES: Readonly> = { + [NATIVE_RESERVE_MODEL]: "reserve", +}; + +export function codexQuotaScopeForModel(modelId: string | undefined): CodexQuotaScope | undefined { + if (!modelId?.trim()) return undefined; + return NATIVE_MODEL_QUOTA_SCOPES[modelId.trim().toLowerCase()] ?? "shared"; +} + +/** Independent quota groups must not mutate the shared active-account cursor. */ +export function isIndependentCodexQuotaScope(quotaScope?: CodexQuotaScope): boolean { + return quotaScope !== undefined && quotaScope !== "shared"; +} + +export function codexPoolKeyForScope(quotaScope?: CodexQuotaScope): string { + return isIndependentCodexQuotaScope(quotaScope) ? `${POOL_KEY_CODEX}:${quotaScope}` : POOL_KEY_CODEX; +} + +export function listLiveCodexAccountIds(config: OcxConfig): ReadonlySet { + const ids = new Set((config.codexAccounts ?? []).map(account => account.id)); + const openai = config.providers.openai; + if (openai && openai.disabled !== true && isCanonicalOpenAiForwardProvider(openai)) { + ids.add(MAIN_CODEX_ACCOUNT_ID); + } + return ids; +} + +export function getCodexUpstreamHealth( + accountId: string, +): CodexUpstreamHealth | null { + dropSpentCredentialFailure(accountId); + return upstreamHealth.get(accountId) ?? null; +} + +export function scopedHealthFor(accountId: string, scope: CodexQuotaScope): CodexUpstreamHealth | undefined { + return quotaScopedHealth.get(accountId)?.get(scope); +} + +export function setScopedHealth(accountId: string, scope: CodexQuotaScope, health: CodexUpstreamHealth): void { + let scopes = quotaScopedHealth.get(accountId); + if (!scopes) { + scopes = new Map(); + quotaScopedHealth.set(accountId, scopes); + } + scopes.set(scope, health); +} + +export function deleteScopedHealth(accountId: string, scope: CodexQuotaScope): void { + const scopes = quotaScopedHealth.get(accountId); + if (!scopes) return; + scopes.delete(scope); + if (scopes.size === 0) quotaScopedHealth.delete(accountId); +} + +/** Live quota-refusal avoidance for an account, including the lane the request belongs to. */ +function codexQuotaAvoidUntil( + accountId: string, + quotaScope: CodexQuotaScope | undefined, + now: number, +): number | null { + const live = (value: number | undefined): number | null => + typeof value === "number" && Number.isFinite(value) && value > now ? value : null; + const account = live(upstreamHealth.get(accountId)?.quotaAvoidUntil); + const scoped = quotaScope === undefined + ? null + : live(scopedHealthFor(accountId, quotaScope)?.quotaAvoidUntil); + if (account === null) return scoped; + return scoped === null ? account : Math.max(account, scoped); +} + +export function isCodexQuotaAvoided( + accountId: string, + quotaScope: CodexQuotaScope | undefined, + now: number, +): boolean { + return codexQuotaAvoidUntil(accountId, quotaScope, now) !== null; +} + +/** + * Hard-cooldown bookkeeping that ordinary success/transient transitions rebuild + * their health object from. Dropping these would let one late unrelated response + * erase a Retry-After source, a cooldown generation, or someone else's live probe. + */ +export function preservedCooldownFields(health: CodexUpstreamHealth | undefined): Partial { + if (!health) return {}; + // `credentialFailureGeneration` is provenance for ONE credential failure, so it must not survive + // into a later transient or quota entry — otherwise that entry inherits the tag and gets spent + // when the old credential dies, deleting evidence that was never about it (#2892 gap 4 review). + const { + consecutiveFailures: _f, consecutiveSuccesses: _s, lastFailureStatus: _st, lastFailureAt: _at, + softAvoidUntil: _sa, credentialFailureGeneration: _cg, ...cooldownFields + } = health; + return cooldownFields; +} + +export function getCodexAccountCooldownUntil(accountId: string, now = Date.now()): number | null { + const cooldownUntil = upstreamHealth.get(accountId)?.cooldownUntil; + return typeof cooldownUntil === "number" && Number.isFinite(cooldownUntil) && cooldownUntil > now ? cooldownUntil : null; +} + +/** Read-only cooldown snapshot for shared OAuth health projection (no write side effects). */ +export function getCodexAccountHealthSnapshot(accountId: string, now = Date.now()): { + cooldownUntil?: number; + cooldownSource?: CodexCooldownSource; +} | null { + const cooldownUntil = getCodexAccountCooldownUntil(accountId, now); + if (cooldownUntil === null) return null; + const source = upstreamHealth.get(accountId)?.cooldownSource; + return { + cooldownUntil, + ...(source ? { cooldownSource: source } : {}), + }; +} + +/** + * Read the cooldown relevant to a routed native model. Account-wide cooldowns + * (Retry-After/default) always win; reset-derived scoped state applies only to + * its confirmed quota group. + */ +export function getCodexQuotaHealthSnapshot( + accountId: string, + quotaScope: CodexQuotaScope | undefined, + now = Date.now(), +): { + cooldownUntil?: number; + cooldownSource?: CodexCooldownSource; + quotaScope?: CodexQuotaScope; +} | null { + const account = getCodexAccountHealthSnapshot(accountId, now); + if (account) return account; + if (!quotaScope) return null; + const scoped = scopedHealthFor(accountId, quotaScope); + const cooldownUntil = scoped?.cooldownUntil; + if (typeof cooldownUntil !== "number" || !Number.isFinite(cooldownUntil) || cooldownUntil <= now) return null; + return { + cooldownUntil, + ...(scoped?.cooldownSource ? { cooldownSource: scoped.cooldownSource } : {}), + quotaScope, + }; +} + +export function isCodexAccountInCooldown(accountId: string, now = Date.now()): boolean { + return getCodexAccountCooldownUntil(accountId, now) !== null; +} + +/** + * Manually lift a hard quota cooldown without touching failure history. + * + * Injected Codex routing makes this proxy the ONLY model path for Codex Desktop, so a + * cooldown that outlives the real upstream limit reads to the user as "the whole app is + * broken" with no escape but editing config.toml. This is that escape hatch. + * + * Deliberately narrow: + * - Failure counters and softAvoid survive. Clearing a cooldown says "the quota window + * moved", not "this account is healthy"; failover must keep its knowledge. + * - Dropping `probeLeaseId` is what stops a stale in-flight probe from later "proving" + * recovery against a NEWER cooldown: {@link ownsProbeLease} needs the id to match. + * `cooldownGeneration` is preserved and bumped as redundancy only — a fresh 429 already + * bumps it in {@link recordCodexUpstreamOutcome}, so the bump here is not load-bearing + * today and is kept so the invariant survives a future change that retains the lease. + * + * Returns false when the account carried neither a live cooldown nor a live avoidance window. + * The window outlives the cooldown by design — the cooldown caps at fifteen minutes and the + * window runs up to six hours — so the moment an operator actually reaches for this escape + * hatch is usually after the cooldown lapsed and only the window is still keeping the account + * out of rotation. Refusing to look at the window then would leave the hatch shut in the one + * case it exists for. + */ +export function clearCodexAccountCooldown(accountId: string, now = Date.now()): boolean { + const clear = (health: CodexUpstreamHealth): CodexUpstreamHealth | null => { + const cooldownUntil = health.cooldownUntil; + const liveCooldown = typeof cooldownUntil === "number" && Number.isFinite(cooldownUntil) && cooldownUntil > now; + const avoidUntil = health.quotaAvoidUntil; + const liveAvoidance = typeof avoidUntil === "number" && Number.isFinite(avoidUntil) && avoidUntil > now; + if (!liveCooldown && !liveAvoidance) return null; + const { + cooldownUntil: _until, + cooldownSince: _since, + cooldownSource: _source, + probeLeaseId: _leaseId, + probeLeaseGeneration: _leaseGeneration, + // Same reasoning as the probe recovery above: "the quota window moved" is a statement + // about the whole refusal, so the avoidance it announced goes with the block it + // produced. Keeping it would leave this escape hatch not escaping, because selection + // would still pass over the account for as long as the announced window runs. + quotaAvoidUntil: _avoid, + ...rest + } = health; + return { + ...rest, + cooldownGeneration: (health.cooldownGeneration ?? 0) + 1, + lastProbeAt: now, + }; + }; + + let cleared = false; + const accountHealth = upstreamHealth.get(accountId); + if (accountHealth) { + const next = clear(accountHealth); + if (next) { + upstreamHealth.set(accountId, next); + cleared = true; + } + } + for (const [scope, health] of quotaScopedHealth.get(accountId) ?? []) { + const next = clear(health); + if (next) { + setScopedHealth(accountId, scope, next); + cleared = true; + } + } + return cleared; +} + +export function getCodexAccountSoftAvoidUntil(accountId: string, now = Date.now()): number | null { + const softAvoidUntil = upstreamHealth.get(accountId)?.softAvoidUntil; + return typeof softAvoidUntil === "number" && Number.isFinite(softAvoidUntil) && softAvoidUntil > now + ? softAvoidUntil + : null; +} + +export function isCodexAccountSoftAvoided(accountId: string, now = Date.now()): boolean { + return getCodexAccountSoftAvoidUntil(accountId, now) !== null; +} + +/** + * Closed package-internal accessors for the account-wide health maps. Selection, + * the probe lease, and the active cursor mutate health only through these; the + * Map bindings themselves never leave this module. + */ +export function getAccountHealth(accountId: string): CodexUpstreamHealth | undefined { + return upstreamHealth.get(accountId); +} + +export function setAccountHealth(accountId: string, health: CodexUpstreamHealth): void { + upstreamHealth.set(accountId, health); +} + +export function deleteAccountHealth(accountId: string): void { + upstreamHealth.delete(accountId); +} + +export function listScopedHealthEntries(accountId: string): Array<[CodexQuotaScope, CodexUpstreamHealth]> { + return [...(quotaScopedHealth.get(accountId) ?? [])]; +} + +export function deleteAllScopedHealth(accountId: string): void { + quotaScopedHealth.delete(accountId); +} + +export function isHealthAccountAdmissible(accountId: string, writerGeneration: number): boolean { + return writerGeneration >= lastReconciledGeneration || liveHealthAccountIds.has(accountId); +} + +export function isHealthGenerationReconciled(generation: number): boolean { + return generation <= lastReconciledGeneration; +} + +export function pruneHealthAccountsForContext(codexAccountIds: ReadonlySet): number { + let removed = 0; + for (const accountId of upstreamHealth.keys()) { + if (codexAccountIds.has(accountId)) continue; + upstreamHealth.delete(accountId); + removed += 1; + } + for (const accountId of quotaScopedHealth.keys()) { + if (codexAccountIds.has(accountId)) continue; + quotaScopedHealth.delete(accountId); + removed += 1; + } + return removed; +} + +export function commitHealthReconcile(generation: number, codexAccountIds: ReadonlySet): void { + liveHealthAccountIds = new Set(codexAccountIds); + lastReconciledGeneration = generation; +} + +export function clearUpstreamHealthState(): void { + upstreamHealth.clear(); + quotaScopedHealth.clear(); +} + +export function resetHealthReconcileState(): void { + lastReconciledGeneration = 0; + liveHealthAccountIds = new Set(); +} + +export function deleteAllHealthForAccount(accountId: string): void { + upstreamHealth.delete(accountId); + quotaScopedHealth.delete(accountId); +} diff --git a/src/codex/routing/probe-lease.ts b/src/codex/routing/probe-lease.ts new file mode 100644 index 0000000000..0ae865ac47 --- /dev/null +++ b/src/codex/routing/probe-lease.ts @@ -0,0 +1,358 @@ +import { randomUUID } from "node:crypto"; +import { isCodexAccountGenerationLive, readCodexAccountRecord, type CodexRefreshProvenance } from "../account-store"; +import { isCodexAccountPaused } from "../account-pause"; +import { isSelectableCodexPoolAccount } from "../account-id"; +import { isAccountNeedsReauth } from "../account-runtime-state"; +import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; +import type { OcxConfig } from "../../types"; +import { CODEX_QUOTA_PROBE_INTERVAL_MS, type CodexUpstreamOutcomeMeta } from "./cooldown-math"; +import { + deleteScopedHealth, + getAccountHealth, + listScopedHealthEntries, + scopedHealthFor, + setAccountHealth, + setScopedHealth, + type CodexQuotaScope, + type CodexUpstreamHealth, +} from "./health-store"; + +export type CodexQuotaRecoveryProbeClaim = { + accountId: string; + scope?: CodexQuotaScope; + leaseId: string; + cooldownGeneration: number; + credentialGeneration: number; + /** Claim-time `replacedAt`; unchanged after a probe-owned refresh, stamped on external replacement. */ + credentialReplacedAt?: number; +}; + +export type CodexQuotaRecoveryProbeProof = { + credentialGeneration?: number; +}; + +/** + * Grant at most one probe lease per interval for a cooled-down account. + * + * A cooled-down account is short-circuited locally, so it never sends traffic and + * no organic 2xx can prove that upstream quota recovered — the cooldown can only + * end by expiry or a proxy restart (#433). Releasing a single probe breaks that + * deadlock. Explicit Retry-After cooldowns are excluded: those are literal retry + * directives, not window announcements. + * + * Returns the lease id, or null when no probe may go out right now. + */ +export function tryAcquireCodexQuotaProbeLease(accountId: string, now = Date.now()): string | null { + if (!canAcquireCodexQuotaProbeLease(accountId, now)) return null; + const health = getAccountHealth(accountId)!; + const probeLeaseId = randomUUID(); + setAccountHealth(accountId, { + ...health, + probeLeaseId, + probeLeaseGeneration: health.cooldownGeneration ?? 0, + lastProbeAt: now, + }); + return probeLeaseId; +} + +/** Side-effect-free check mirroring {@link tryAcquireCodexQuotaProbeLease} eligibility. */ +export function canAcquireCodexQuotaProbeLease(accountId: string, now = Date.now()): boolean { + return canAcquireQuotaProbeLease(getAccountHealth(accountId), now); +} + +function canAcquireQuotaProbeLease(health: CodexUpstreamHealth | undefined, now: number): boolean { + if (!health) return false; + const cooldownUntil = health.cooldownUntil; + if (typeof cooldownUntil !== "number" || !Number.isFinite(cooldownUntil) || cooldownUntil <= now) return false; + if (health.cooldownSource === "retry-after") return false; + if (health.probeLeaseId !== undefined) return false; + const origin = health.lastProbeAt ?? health.cooldownSince ?? cooldownUntil; + return now - origin >= CODEX_QUOTA_PROBE_INTERVAL_MS; +} + +/** + * Claim due reset-derived cooldown probes without consulting account selection. + * Added Pool credentials only; owned main usage recovery is handled separately. + */ +export function claimDueCodexQuotaRecoveryProbes( + config: OcxConfig, + limit: number, + now = Date.now(), +): CodexQuotaRecoveryProbeClaim[] { + const boundedLimit = Math.max(0, Math.floor(limit)); + if (boundedLimit === 0) return []; + const candidates: Array<{ + accountId: string; + scope?: CodexQuotaScope; + health: CodexUpstreamHealth; + credentialGeneration: number; + credentialReplacedAt?: number; + order: number; + }> = []; + for (const [order, account] of (config.codexAccounts ?? []).entries()) { + if (!isSelectableCodexPoolAccount(account) + || isCodexAccountPaused(config, account.id) + || isAccountNeedsReauth(account.id)) continue; + const record = readCodexAccountRecord(account.id); + if (!record?.credential || record.deletedAt != null) continue; + const due = [ + { scope: undefined, health: getAccountHealth(account.id) }, + ...[...(listScopedHealthEntries(account.id))].map(([scope, health]) => ({ scope, health })), + ].filter((entry): entry is { scope?: CodexQuotaScope; health: CodexUpstreamHealth } => + // Generic WHAM evidence can recover only ordinary quota, never Reserve. + // Do not spend this account's one claim per pass on an independent scope and + // delay the shared scope that the response can actually recover. + (entry.scope === undefined || entry.scope === "shared") + && entry.health?.cooldownSource === "reset-derived" + && canAcquireQuotaProbeLease(entry.health, now)) + .sort((a, b) => + (a.health.lastProbeAt ?? a.health.cooldownSince ?? 0) + - (b.health.lastProbeAt ?? b.health.cooldownSince ?? 0)); + const candidate = due[0]; + if (candidate) candidates.push({ + accountId: account.id, + ...(candidate.scope ? { scope: candidate.scope } : {}), + health: candidate.health, + credentialGeneration: record.generation, + ...(record.replacedAt !== undefined ? { credentialReplacedAt: record.replacedAt } : {}), + order, + }); + } + candidates.sort((a, b) => { + const age = (a.health.lastProbeAt ?? a.health.cooldownSince ?? 0) + - (b.health.lastProbeAt ?? b.health.cooldownSince ?? 0); + return age || a.order - b.order; + }); + return candidates.slice(0, boundedLimit).map(candidate => { + const leaseId = randomUUID(); + const next = { + ...candidate.health, + probeLeaseId: leaseId, + probeLeaseGeneration: candidate.health.cooldownGeneration ?? 0, + lastProbeAt: now, + }; + if (candidate.scope) setScopedHealth(candidate.accountId, candidate.scope, next); + else setAccountHealth(candidate.accountId, next); + return { + accountId: candidate.accountId, + ...(candidate.scope ? { scope: candidate.scope } : {}), + leaseId, + cooldownGeneration: candidate.health.cooldownGeneration ?? 0, + credentialGeneration: candidate.credentialGeneration, + ...(candidate.credentialReplacedAt !== undefined + ? { credentialReplacedAt: candidate.credentialReplacedAt } + : {}), + }; + }); +} + +type CooldownRecoveryLease = Pick; + +export type ManualResetCooldownClaim = + | { kind: "pool"; probe: CodexQuotaRecoveryProbeClaim } + | { kind: "main"; probe: CooldownRecoveryLease }; + +function manualResetAccountEligible(config: OcxConfig, accountId: string): boolean { + return !isCodexAccountPaused(config, accountId) && !isAccountNeedsReauth(accountId) + && (accountId === MAIN_CODEX_ACCOUNT_ID + || (config.codexAccounts ?? []).some(account => account.id === accountId && isSelectableCodexPoolAccount(account))); +} + +/** Explicit reset bypasses probe pacing, never another owner's lease or quota scope. */ +export function claimManualResetCooldowns( + config: OcxConfig, + accountId: string, + now = Date.now(), + expectedPoolGeneration?: number, +): ManualResetCooldownClaim[] { + if (!manualResetAccountEligible(config, accountId)) return []; + const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); + if (accountId !== MAIN_CODEX_ACCOUNT_ID && (!record?.credential || record.deletedAt != null)) return []; + if (record && expectedPoolGeneration !== undefined && record.generation !== expectedPoolGeneration) return []; + const claims: ManualResetCooldownClaim[] = []; + for (const scope of [undefined, "shared"] as const) { + const health = scope ? scopedHealthFor(accountId, scope) : getAccountHealth(accountId); + if (!health || health.cooldownSource !== "reset-derived" || health.probeLeaseId !== undefined + || !Number.isFinite(health.cooldownUntil) || !(health.cooldownUntil! > now)) continue; + const leaseId = randomUUID(); + const cooldownGeneration = health.cooldownGeneration ?? 0; + const next = { ...health, probeLeaseId: leaseId, probeLeaseGeneration: cooldownGeneration, lastProbeAt: now }; + if (scope) setScopedHealth(accountId, scope, next); + else setAccountHealth(accountId, next); + const probe = { accountId, scope, leaseId, cooldownGeneration }; + claims.push(record ? { kind: "pool", probe: { + ...probe, credentialGeneration: record.generation, credentialReplacedAt: record.replacedAt, + } } : { kind: "main", probe }); + } + return claims; +} + +export type ManualResetRefreshLineage = Readonly<{ + fromGeneration: number; + toGeneration: number; + provenance: CodexRefreshProvenance; +}>; + +type ManualResetQuotaProof = CodexQuotaRecoveryProbeProof & { + refreshLineage?: ManualResetRefreshLineage; +}; + +/** Main proof is checked by the already-owned auth operation, never by a Pool record. */ +export function settleManualResetCooldown( + config: OcxConfig, + claim: ManualResetCooldownClaim, + recovered: boolean, + proof: ManualResetQuotaProof = {}, + now = Date.now(), +): boolean { + if (!recovered) return settleCooldownRecoveryLease(claim.probe, false, now); + const eligible = manualResetAccountEligible(config, claim.probe.accountId); + if (claim.kind === "main") return settleCooldownRecoveryLease(claim.probe, eligible, now); + const lineage = proof.refreshLineage; + // Equal wall-clock replacement stamps do not establish ancestry. Manual +1 + // recovery additionally needs the actual forced-refresh result for this edge. + const ownedGeneration = proof.credentialGeneration === claim.probe.credentialGeneration + || (proof.credentialGeneration === claim.probe.credentialGeneration + 1 + && lineage?.fromGeneration === claim.probe.credentialGeneration + && lineage.toGeneration === proof.credentialGeneration + && (lineage.provenance === "self-refresh" || lineage.provenance === "joined-lineage")); + return settleCodexQuotaRecoveryProbe(claim.probe, eligible && ownedGeneration, proof, now); +} + +/** Settle one background recovery claim without mutating account-wide outcome state. */ +export function settleCodexQuotaRecoveryProbe( + claim: CodexQuotaRecoveryProbeClaim, + recovered: boolean, + proof: CodexQuotaRecoveryProbeProof, + now = Date.now(), +): boolean { + const health = claim.scope + ? scopedHealthFor(claim.accountId, claim.scope) + : getAccountHealth(claim.accountId); + if (!health || health.probeLeaseId !== claim.leaseId) return false; + const currentRecord = readCodexAccountRecord(claim.accountId); + const proofGeneration = proof.credentialGeneration; + // A probe-owned token refresh (getValidCodexToken) advances the credential generation by + // exactly one while preserving `replacedAt`; an external credential replacement bumps the + // generation too but stamps a fresh `replacedAt`. Accept the +1 transition only when the + // claim-time lineage is intact AND the generation the fresh quota was proven under is live. + const generationFenced = proofGeneration !== undefined + && (proofGeneration === claim.credentialGeneration + ? isCodexAccountGenerationLive(claim.accountId, proofGeneration) + : proofGeneration === claim.credentialGeneration + 1 + && currentRecord?.replacedAt === claim.credentialReplacedAt + && isCodexAccountGenerationLive(claim.accountId, proofGeneration)); + return settleCooldownRecoveryLease(claim, recovered && generationFenced, now); +} + +function settleCooldownRecoveryLease(claim: CooldownRecoveryLease, recovered: boolean, now: number): boolean { + const health = claim.scope ? scopedHealthFor(claim.accountId, claim.scope) : getAccountHealth(claim.accountId); + if (!health || health.probeLeaseId !== claim.leaseId) return false; + const fenced = (claim.scope === undefined || claim.scope === "shared") + && health.cooldownSource === "reset-derived" + && (health.cooldownGeneration ?? 0) === claim.cooldownGeneration + && (health.probeLeaseGeneration ?? 0) === claim.cooldownGeneration; + if (!recovered || !fenced) { + const released = withProbeLeaseReleased(health, now); + if (claim.scope) setScopedHealth(claim.accountId, claim.scope, released); + else setAccountHealth(claim.accountId, released); + return false; + } + if (claim.scope) { + deleteScopedHealth(claim.accountId, claim.scope); + } else { + const { + cooldownUntil: _until, + cooldownSince: _since, + cooldownSource: _source, + probeLeaseId: _leaseId, + probeLeaseGeneration: _leaseGeneration, + // "The quota window moved" is a statement about the whole refusal, so the avoidance it + // announced goes with the block it produced. Leaving it would make this escape hatch stop + // escaping: the account would still be passed over by every selection it is meant to win. + quotaAvoidUntil: _avoid, + ...rest + } = health; + setAccountHealth(claim.accountId, { + ...rest, + cooldownGeneration: claim.cooldownGeneration + 1, + lastProbeAt: now, + }); + } + return true; +} + +/** Acquire the recovery probe for one confirmed model-specific quota group. */ +export function tryAcquireCodexQuotaScopeProbeLease( + accountId: string, + scope: CodexQuotaScope, + now = Date.now(), +): string | null { + const health = scopedHealthFor(accountId, scope); + if (!canAcquireQuotaProbeLease(health, now)) return null; + const probeLeaseId = randomUUID(); + setScopedHealth(accountId, scope, { + ...health!, + probeLeaseId, + probeLeaseGeneration: health!.cooldownGeneration ?? 0, + lastProbeAt: now, + }); + return probeLeaseId; +} + +/** Side-effect-free check for a confirmed model-specific quota probe. */ +export function canAcquireCodexQuotaScopeProbeLease( + accountId: string, + scope: CodexQuotaScope, + now = Date.now(), +): boolean { + return canAcquireQuotaProbeLease(scopedHealthFor(accountId, scope), now); +} + +/** + * Hand a probe lease back without recording an upstream outcome. Used by paths + * that take a lease and then fail before any request reaches upstream. + */ +export function releaseCodexQuotaProbeLease(accountId: string, leaseId: string, now = Date.now()): void { + const health = getAccountHealth(accountId); + if (!health || health.probeLeaseId !== leaseId) return; + setAccountHealth(accountId, withProbeLeaseReleased(health, now)); +} + +/** Release a model-specific quota probe when the request never reaches upstream. */ +export function releaseCodexQuotaScopeProbeLease( + accountId: string, + scope: CodexQuotaScope, + leaseId: string, + now = Date.now(), +): void { + const health = scopedHealthFor(accountId, scope); + if (!health || health.probeLeaseId !== leaseId) return; + setScopedHealth(accountId, scope, withProbeLeaseReleased(health, now)); +} + +/** + * True when this outcome belongs to the account's in-flight probe. The + * undefined-id guard matters: without it an outcome carrying no lease would match + * an account holding no lease and be mistaken for the probe owner. + */ +export function ownsProbeLease(health: CodexUpstreamHealth | undefined, meta: CodexUpstreamOutcomeMeta): boolean { + return meta.probeLeaseId !== undefined && meta.probeLeaseId === health?.probeLeaseId; +} + +/** + * True when the owning probe may still clear the cooldown. A later 429 bumps the + * generation, so a probe that started under an older cooldown must not erase the + * newer restriction (which may carry an explicit Retry-After). + */ +export function probeMayClearCooldown(health: CodexUpstreamHealth | undefined, meta: CodexUpstreamOutcomeMeta): boolean { + return ownsProbeLease(health, meta) + && (health!.probeLeaseGeneration ?? 0) === (health!.cooldownGeneration ?? 0); +} + +/** Strip the in-flight lease while preserving every hard-cooldown field. */ +export function withProbeLeaseReleased(health: CodexUpstreamHealth, now: number): CodexUpstreamHealth { + const { probeLeaseId: _id, probeLeaseGeneration: _gen, ...rest } = health; + return { ...rest, lastProbeAt: now }; +} diff --git a/src/codex/routing/selection.ts b/src/codex/routing/selection.ts new file mode 100644 index 0000000000..9272636e99 --- /dev/null +++ b/src/codex/routing/selection.ts @@ -0,0 +1,703 @@ +import { isCodexAccountPaused } from "../account-pause"; +import { codexAccountPriorityLookup, pinnedCodexAccountId } from "../account-priority"; +import { isSelectableCodexPoolAccount } from "../account-id"; +import { isAccountNeedsReauth } from "../account-runtime-state"; +import { isCodexAccountUsable, type CodexAccountUsabilityOptions } from "../account-usability"; +import { isCodexPoolRefreshCooling } from "../pool-refresh-backoff"; +import { + normalizeAccountPoolStickyLimit, + normalizeCodexAccountPoolStrategy, + notePoolRotationSuccess, + peekRoundRobinAccount, + pickRoundRobinAccount, + selectPriorityTier, +} from "../pool-rotation"; +import { CODEX_UNKNOWN_USAGE_SCORE, getAccountQuota, resetAtToMs } from "../quota"; +import { codexPlanKey } from "../plan"; +import { MAIN_CODEX_ACCOUNT_ID, getMainAccountPlan, hasMainAccountRefreshGrant } from "../main-account"; +import type { OcxConfig } from "../../types"; +import { CODEX_FAILURE_WINDOW_MS, computeCodexUsageScore } from "./cooldown-math"; +import { + codexPoolKeyForScope, + dropSpentCredentialFailure, + getAccountHealth, + getCodexQuotaHealthSnapshot, + isCodexAccountSoftAvoided, + isCodexQuotaAvoided, + isIndependentCodexQuotaScope, + type CodexQuotaScope, +} from "./health-store"; +import { bindThreadAffinity, type CodexAffinityReason } from "./thread-affinity"; +import { + getEffectiveActiveCodexAccountId, + manualPreferenceBlocks, + promoteActiveCodexAccount, + rememberActiveCodexAccount, + setActiveCodexAccount, +} from "./active-account"; + +/** + * Plan keys the operator excluded from automatic rotation. Absent or empty means no policy, so an + * existing install rotates exactly as before. Compared with `codexPlanKey` because the stored plan + * is an unrestricted provider string whose casing this repository does not control. + */ +function excludedCodexPoolPlanKeys(config: OcxConfig): ReadonlySet | undefined { + const configured = config.codexPool?.excludedPlans; + if (!configured?.length) return undefined; + const keys = configured + .map(plan => codexPlanKey(plan)) + .filter((key): key is string => key !== undefined); + return keys.length > 0 ? new Set(keys) : undefined; +} + +/** + * Whether the operator's plan policy removes this account from automatic selection. + * + * Modelled on pause rather than usability: an excluded account keeps its credential, quota history, + * and affinity, stays visible on the account surface, and is still reachable by explicit account + * selection. Only automatic rotation skips it, which is the distinction #4211 asked for. + * + * It is checked in the same two places pause is checked, and that is not redundancy. The eligible + * list is consulted only when routing picks a NEW account; an already-active or already-affined + * account is served straight from {@link isCodexAccountSelectable}. A lapsed subscription leaves + * behind exactly that account, so a policy that filtered only the eligible list would miss the case + * it exists for. + * + * `__main__` is exempt. {@link getPoolAccountPlanForSelection} withholds the main plan during a + * selection-only drain so routing never reads the fenced native credential for it, so a rule that + * covered main would disagree with itself between drain and ordinary routing. + */ +export function isCodexAccountPlanExcluded( + config: OcxConfig, + accountId: string, + precomputed?: ReadonlySet, +): boolean { + if (accountId === MAIN_CODEX_ACCOUNT_ID) return false; + // Callers that test a whole list pass the set once rather than rebuilding it per row. + const excluded = precomputed ?? excludedCodexPoolPlanKeys(config); + if (!excluded) return false; + const plan = codexPlanKey(getPoolAccountPlan(config, accountId)); + return plan !== undefined && excluded.has(plan); +} + +export function isCodexAccountSelectable( + config: OcxConfig, + accountId: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): boolean { + return !isCodexAccountPaused(config, accountId) + && !isCodexAccountPlanExcluded(config, accountId) + && getCodexQuotaHealthSnapshot(accountId, quotaScope, now) === null + && !isCodexQuotaAvoided(accountId, quotaScope, now) + && !isCodexAccountSoftAvoided(accountId, now) + && !isCodexPoolRefreshCooling(accountId, now) + && isCodexAccountUsable(config, accountId, selectionOptions); +} + +/** + * Which guard in {@link isCodexAccountSelectable} refused this account, if any. + * + * Deliberately the same predicates in the same order as that function, because the point is to + * REPORT the guard that actually fired rather than to re-derive a plausible-looking cause. An + * earlier version of the release reason checked only a subset and let a paused, plan-excluded, + * cooled-down or quota-avoided release fall through to a quota fallback, which named something + * routing never used -- a diagnostic that is confidently wrong in exactly the cases an operator + * would consult it for (#4598). + */ +export function codexAccountBlockReason( + config: OcxConfig, + accountId: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): CodexAffinityReason | undefined { + if (isCodexAccountPaused(config, accountId)) return "paused"; + if (isCodexAccountPlanExcluded(config, accountId)) return "plan_excluded"; + if (getCodexQuotaHealthSnapshot(accountId, quotaScope, now) !== null) return "cooldown"; + if (isCodexQuotaAvoided(accountId, quotaScope, now)) return "quota_avoided"; + if (isCodexAccountSoftAvoided(accountId, now)) return "transient"; + if (isCodexPoolRefreshCooling(accountId, now)) return "transient"; + if (!isCodexAccountUsable(config, accountId, selectionOptions)) return "unusable"; + return undefined; +} + +export function getEligiblePoolAccounts( + config: OcxConfig, + excludeId?: string, + now = Date.now(), + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, + skipFailoverReadyCandidates = false, +): readonly string[] { + const excludedPlans = excludedCodexPoolPlanKeys(config); + const ids = (config.codexAccounts ?? []) + .filter(account => isSelectableCodexPoolAccount(account) + && account.id !== excludeId + && !isCodexAccountPaused(config, account.id) + && !isCodexAccountPlanExcluded(config, account.id, excludedPlans) + && !isAccountNeedsReauth(account.id) + && (!skipFailoverReadyCandidates || !shouldFailover(config, account.id, now))) + .filter(account => getCodexQuotaHealthSnapshot(account.id, quotaScope, now) === null) + .filter(account => !isCodexAccountSoftAvoided(account.id, now)) + .filter(account => !isCodexQuotaAvoided(account.id, quotaScope, now)) + .filter(account => !isCodexPoolRefreshCooling(account.id, now)) + .filter(account => isCodexAccountUsable(config, account.id, selectionOptions)) + .map(account => account.id); + // The main Codex account is not stored in config.codexAccounts; include it as a + // first-class rotation candidate when its read-only token is usable (Option A). + if ( + excludeId !== MAIN_CODEX_ACCOUNT_ID + && !isCodexAccountPaused(config, MAIN_CODEX_ACCOUNT_ID) + && (!isAccountNeedsReauth(MAIN_CODEX_ACCOUNT_ID) || hasMainAccountRefreshGrant()) + && getCodexQuotaHealthSnapshot(MAIN_CODEX_ACCOUNT_ID, quotaScope, now) === null + && !isCodexAccountSoftAvoided(MAIN_CODEX_ACCOUNT_ID, now) + // The main login is not in `config.codexAccounts`, so it never passes through the + // filters above and this is the only place an avoidance window can exclude it. Without + // this the window a refusal announced applies to the pool but not to the account that + // earned it: the cooldown caps at fifteen minutes, the window runs up to six hours, and + // in between the main account returns as a first-class candidate. + && !isCodexQuotaAvoided(MAIN_CODEX_ACCOUNT_ID, quotaScope, now) + && !isCodexPoolRefreshCooling(MAIN_CODEX_ACCOUNT_ID, now) + && (!skipFailoverReadyCandidates || !shouldFailover(config, MAIN_CODEX_ACCOUNT_ID, now)) + && isCodexAccountUsable(config, MAIN_CODEX_ACCOUNT_ID, selectionOptions) + ) { + ids.unshift(MAIN_CODEX_ACCOUNT_ID); + } + // Single choke point for selection order: every strategy, failover, and preview + // reaches the pool through here, so tiering applies once rather than per picker. + // Eligibility above is unchanged — this only narrows an already-eligible list. + return selectPriorityTier( + ids, + codexAccountPriorityLookup(config), + id => hasCodexQuotaHeadroom(config, id, selectionOptions, now), + pinnedCodexAccountId(config), + ); +} + +function listEligibleCodexAccountIds( + config: OcxConfig, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): readonly string[] { + return getEligiblePoolAccounts(config, undefined, now, quotaScope, selectionOptions); +} + +/** Shared reset timestamps are not evidence for independent model-quota groups. */ +export function accountPoolStrategyForScope(config: OcxConfig, quotaScope?: CodexQuotaScope) { + const strategy = normalizeCodexAccountPoolStrategy(config.accountPoolStrategy); + return strategy === "reset-first" && isIndependentCodexQuotaScope(quotaScope) ? "quota" : strategy; +} + +function stickyLimitForConfig(config: OcxConfig): number { + return normalizeAccountPoolStickyLimit(config.accountPoolStickyLimit); +} + +/** + * Whether an account still has quota to give under the auto-switch threshold. + * + * Fill-first and the priority tier filter share this predicate, and share both of + * its escape hatches. A disabled threshold means only health, pause, and reauth + * may drain an account; unknown usage is a guess, so it must neither force + * fill-first off the active account nor drain a tier that was simply never + * primed. A genuinely exhausted account 429s into cooldown and leaves + * eligibility on its own. + */ +export function hasCodexQuotaHeadroom( + config: OcxConfig, + accountId: string, + selectionOptions?: CodexAccountUsabilityOptions, + now: number = Date.now(), +): boolean { + const threshold = config.autoSwitchThreshold ?? 80; + if (threshold <= 0) return true; + const usage = computeCodexUsageScore( + getAccountQuota(accountId), + getPoolAccountPlanForSelection(config, accountId, selectionOptions), + now, + ); + if (isUnknownUsage(usage)) return true; + return usage < threshold; +} + +/** + * Is a live binding held for its prompt cache? + * + * Unset means yes. Cache affinity shipped as an opt-in flag (#4292) and then #4546 measured + * what the default costs: a pool whose accounts all sit in the 80-99% band hands a bound + * conversation from account to account, and because provider prompt caches are account-isolated + * every hop re-sends the entire prefix. An install that has never heard of this flag is exactly + * the install that gets hurt by it, so the protection cannot be something you have to find. + * + * `false` restores capacity-first routing byte-for-byte. It is a real choice -- a pinned thread + * on a busy account pays latency -- and it stays available; it is just no longer the default. + */ +export function isCacheAffinityEnabled(config: OcxConfig): boolean { + return config.pool?.cacheAffinity !== false; +} + +/** Earliest future shared short/weekly reset; missing evidence and ties use usage order. */ +export function pickResetFirstCodexAccount( + config: OcxConfig, + ids: readonly string[], + now: number, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + const available = ids.filter(id => hasCodexQuotaHeadroom(config, id, selectionOptions, now)); + if (available.length === 0) return pickLowestUsageAmong(config, ids, selectionOptions, now); + let earliest = Number.POSITIVE_INFINITY; + let candidates: string[] = []; + for (const id of available) { + const quota = getAccountQuota(id); + const resets = [quota?.shortResetAt, quota?.weeklyResetAt] + .filter((reset): reset is number => typeof reset === "number" && Number.isFinite(reset)) + .map(resetAtToMs) + .filter(reset => reset > now); + const next = Math.min(...resets); + if (next < earliest) { + earliest = next; + candidates = [id]; + } else if (next === earliest) candidates.push(id); + } + return pickLowestUsageAmong(config, candidates, selectionOptions, now); +} + +/** + * Fill-first: keep selectable active under threshold; otherwise advance to the next + * eligible id in stable sorted order after the current active (wrapping). + */ +function pickFillFirstCodexAccount( + config: OcxConfig, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + const eligible = listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions); + if (eligible.length === 0) return null; + + const active = getEffectiveActiveCodexAccountId(config); + if (active && eligible.includes(active) && hasCodexQuotaHeadroom(config, active, selectionOptions, now)) { + return active; + } + + return pickNextFillFirstCodexAccount(config, active ?? null, eligible, now, selectionOptions); +} + +/** Next eligible account in stable order after `afterId` (wrapping). */ +function pickNextFillFirstCodexAccount( + config: OcxConfig, + afterId: string | null, + eligible: readonly string[] = listEligibleCodexAccountIds(config, Date.now()), + now = Date.now(), + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + if (eligible.length === 0) return null; + const ordered = [...eligible].sort((a, b) => a.localeCompare(b)); + if (!afterId) { + // Prefer an under-threshold account when starting with no active cursor. + for (const id of ordered) { + if (hasCodexQuotaHeadroom(config, id, selectionOptions, now)) return id; + } + return ordered[0] ?? null; + } + + const allConfigured = [ + ...(isCodexAccountUsable(config, MAIN_CODEX_ACCOUNT_ID, selectionOptions) || afterId === MAIN_CODEX_ACCOUNT_ID + ? [MAIN_CODEX_ACCOUNT_ID] + : []), + ...(config.codexAccounts ?? []).filter(account => !account.isMain).map(account => account.id), + ]; + const stableAll = [...new Set(allConfigured)].sort((a, b) => a.localeCompare(b)); + const startIdx = stableAll.indexOf(afterId); + if (startIdx < 0) { + for (const id of ordered) { + if (hasCodexQuotaHeadroom(config, id, selectionOptions, now)) return id; + } + return ordered[0] ?? null; + } + + // Skip successors that are also at/above threshold (known drained usage). + let fallback: string | null = null; + for (let step = 1; step <= stableAll.length; step++) { + const candidate = stableAll[(startIdx + step) % stableAll.length]!; + if (!eligible.includes(candidate)) continue; + if (!fallback) fallback = candidate; + if (hasCodexQuotaHeadroom(config, candidate, selectionOptions, now)) return candidate; + } + return fallback ?? ordered[0] ?? null; +} + +/** + * Unbound new-session pick for round-robin / fill-first. Returns null to fall through + * to the legacy quota path (or when the strategy is quota). + * + * When `commit` is true (resolve path), advances RR state. `commitSharedActive` + * and `commitAffinity` independently control the two cross-request side effects: + * model-scoped entitlement selection can bind a new task without replacing an + * existing task binding or global active choice. Preview remains a dry-run peek. + * + * Automatic strategy picks never sync-write config; only manual selection persists active. + * + * Known limitation (follow-up): when a subagent preview peeks an RR account and the request + * then falls back to a non-Codex provider, the ring is not reserved/committed. Prefer seeding + * the peeked account if that path becomes load-bearing. + */ +export function pickUnboundStrategyAccount( + config: OcxConfig, + threadId: string | null, + now: number, + commit: boolean, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, + commitSharedActive = commit, + commitAffinity = commit, +): string | null { + const strategy = accountPoolStrategyForScope(config, quotaScope); + if (strategy === "quota") return null; + const poolKey = codexPoolKeyForScope(quotaScope); + + let picked: string | null = null; + if (strategy === "round-robin") { + const eligible = listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions); + const limit = stickyLimitForConfig(config); + if (!commit) { + return peekRoundRobinAccount(poolKey, eligible, limit); + } + picked = pickRoundRobinAccount(poolKey, eligible, limit); + if (!picked) return null; + if (commitSharedActive) { + if (!isIndependentCodexQuotaScope(quotaScope) + && !manualPreferenceBlocks(codexPoolKeyForScope(quotaScope), picked)) { + rememberActiveCodexAccount(config, picked); + } + } + if (commitAffinity && threadId) bindThreadAffinity(threadId, picked, now, quotaScope); + notePoolRotationSuccess(poolKey, picked, limit); + return picked; + } + + if (strategy === "fill-first" || strategy === "reset-first") { + picked = strategy === "reset-first" + ? pickResetFirstCodexAccount(config, listEligibleCodexAccountIds(config, now, quotaScope, selectionOptions), now, selectionOptions) + : pickFillFirstCodexAccount(config, now, quotaScope, selectionOptions); + if (!picked) return null; + if (commitSharedActive) { + if (!isIndependentCodexQuotaScope(quotaScope) + && !manualPreferenceBlocks(codexPoolKeyForScope(quotaScope), picked)) { + rememberActiveCodexAccount(config, picked); + } + } + if (commitAffinity && threadId) bindThreadAffinity(threadId, picked, now, quotaScope); + return picked; + } + + return null; +} + +export function getPoolAccountPlan(config: OcxConfig, accountId: string): string | undefined { + if (accountId === MAIN_CODEX_ACCOUNT_ID) return getMainAccountPlan(); + return (config.codexAccounts ?? []) + .find(account => isSelectableCodexPoolAccount(account) && account.id === accountId)?.plan; +} + +/** Selection-only main routing must not lazily read the fenced native credential for its plan. */ +export function getPoolAccountPlanForSelection( + config: OcxConfig, + accountId: string, + selectionOptions?: CodexAccountUsabilityOptions, +): string | undefined { + if (accountId === MAIN_CODEX_ACCOUNT_ID && selectionOptions?.nativeMainSelectionOnly === true) { + return undefined; + } + return getPoolAccountPlan(config, accountId); +} + +/** Shared routing state must ignore a request-scoped entitlement roster. */ +export function sharedStateSelectionOptions( + selectionOptions?: CodexAccountUsabilityOptions, +): Pick< + CodexAccountUsabilityOptions, + "nativeMainSelectionOnly" | "isMainAccountTokenLive" +> | undefined { + if (!selectionOptions) return undefined; + return { + ...(selectionOptions.nativeMainSelectionOnly !== undefined + ? { nativeMainSelectionOnly: selectionOptions.nativeMainSelectionOnly } + : {}), + ...(selectionOptions.isMainAccountTokenLive + ? { isMainAccountTokenLive: selectionOptions.isMainAccountTokenLive } + : {}), + }; +} + +export function pickLowerUsageAccount( + config: OcxConfig, + active: string, + activeUsage: number, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, + skipFailoverReadyCandidates = false, +): string { + let best = active; + let bestUsage = activeUsage; + for (const id of getEligiblePoolAccounts( + config, + active, + now, + quotaScope, + selectionOptions, + skipFailoverReadyCandidates, + )) { + const usage = computeCodexUsageScore( + getAccountQuota(id), + getPoolAccountPlanForSelection(config, id, selectionOptions), + now, + ); + if (usage < bestUsage) { + best = id; + bestUsage = usage; + } + } + return best; +} + +/** Coolest account in an already-selected candidate list; first index wins ties. */ +export function pickLowestUsageAmong( + config: OcxConfig, + ids: readonly string[], + selectionOptions?: CodexAccountUsabilityOptions, + now: number = Date.now(), +): string | null { + let best: string | null = null; + let bestUsage = Number.POSITIVE_INFINITY; + for (const id of ids) { + const usage = computeCodexUsageScore( + getAccountQuota(id), + getPoolAccountPlanForSelection(config, id, selectionOptions), + now, + ); + if (usage < bestUsage) { + best = id; + bestUsage = usage; + } + } + return best; +} + +export function pickLowestUsageCodexAccount( + config: OcxConfig, + excludeId?: string, + now = Date.now(), + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + return pickLowestUsageAmong( + config, + getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions), + selectionOptions, + now, + ); +} + +/** + * Strategy-aware alternate after a cooled/excluded account (same-request 429 retry + * and active promotion). Quota keeps lowest-usage; fill-first advances stable order; + * round-robin takes the next ring pick (caller should have noted the failure). + */ +export function pickAlternateCodexAccount( + config: OcxConfig, + excludeId: string, + now = Date.now(), + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + const strategy = accountPoolStrategyForScope(config, quotaScope); + // The exclusion is passed into eligibility rather than post-filtered off its + // result: when the excluded account is the only healthy member of the top + // tier, the tier walk must be free to descend instead of selecting that tier + // and then handing back an empty list. + if (strategy === "round-robin") { + const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); + return pickRoundRobinAccount(codexPoolKeyForScope(quotaScope), eligible, stickyLimitForConfig(config)); + } + if (strategy === "fill-first") { + const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); + return pickNextFillFirstCodexAccount(config, excludeId, eligible, now, selectionOptions); + } + if (strategy === "reset-first") { + return pickResetFirstCodexAccount(config, getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions), now, selectionOptions); + } + return pickLowestUsageCodexAccount(config, excludeId, now, quotaScope, selectionOptions); +} + +/** + * The account {@link pickAlternateCodexAccount} WOULD return, without returning it. + * + * Only the round-robin branch has a side effect -- `pickRoundRobinAccount` commits the pick and + * advances the ring -- so every other strategy delegates rather than growing a second copy of + * the selection rule that could drift from it. + * + * This exists because preview and resolve have to agree on the FIRST transient detour, not just + * on later ones. Preview feeds subagent model-availability scoring, so a preview that reported + * the bound account while resolve was about to serve from a cool sibling could retire a model + * over usage the request would never have touched. + */ +export function peekAlternateCodexAccount( + config: OcxConfig, + excludeId: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + if (accountPoolStrategyForScope(config, quotaScope) === "round-robin") { + const eligible = getEligiblePoolAccounts(config, excludeId, now, quotaScope, selectionOptions); + return peekRoundRobinAccount(codexPoolKeyForScope(quotaScope), eligible, stickyLimitForConfig(config)); + } + return pickAlternateCodexAccount(config, excludeId, now, quotaScope, selectionOptions); +} + +export function isUnknownUsage(usage: number): boolean { + return usage >= CODEX_UNKNOWN_USAGE_SCORE; +} + +/** + * Move an unbound request back up when a higher tier regains headroom — the + * weekly-reset case. Returns null when nothing should change. + * + * Downward moves are deliberately left to {@link applyQuotaAutoSwitch}: this only + * fires when the tier filter has already excluded `active`, and only toward a + * tier that strictly outranks it. Threads bound by affinity never reach here. + */ +export function pickPriorityPreemption( + config: OcxConfig, + active: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, +): string | null { + const eligible = getEligiblePoolAccounts(config, undefined, now, quotaScope, selectionOptions); + if (eligible.length === 0 || eligible.includes(active)) return null; + const pinned = pinnedCodexAccountId(config); + // A live pin already lowered the tier ceiling; never preempt past an explicit + // operator choice. Same liveness test the tier filter applies, so preview and + // resolve agree even before the pin is garbage-collected. + if ( + pinned !== undefined + && eligible.includes(pinned) + && hasCodexQuotaHeadroom(config, pinned, selectionOptions, now) + ) return null; + const priorityOf = codexAccountPriorityLookup(config); + if (priorityOf(eligible[0]!) <= priorityOf(active)) return null; + // Members without headroom are in the tier only because a sibling has some; + // picking one would hand the request straight back to a drained account. + return pickLowestUsageAmong( + config, + eligible.filter(id => hasCodexQuotaHeadroom(config, id, selectionOptions, now)), + selectionOptions, + now, + ); +} + +export function applyQuotaAutoSwitch( + config: OcxConfig, + active: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, + commitSharedSelection = true, +): string { + const threshold = config.autoSwitchThreshold ?? 80; + if (threshold <= 0) return active; + const quota = getAccountQuota(active); + const activeUsage = computeCodexUsageScore( + quota, + getPoolAccountPlanForSelection(config, active, selectionOptions), + now, + ); + // Unknown usage is not evidence that a user's explicit selection crossed the + // threshold. Wait for quota priming instead of rotating among guesses. + if (isUnknownUsage(activeUsage)) return active; + if (activeUsage < threshold) return active; + const best = pickLowerUsageAccount(config, active, activeUsage, now, quotaScope, selectionOptions); + if (best !== active) { + if (commitSharedSelection && !isIndependentCodexQuotaScope(quotaScope)) { + setActiveCodexAccount(config, best); + } + return best; + } + + return active; +} + +export function shouldFailover(config: OcxConfig, accountId: string, now: number): boolean { + const threshold = config.upstreamFailoverThreshold ?? 3; + if (threshold <= 0) return false; + dropSpentCredentialFailure(accountId); + const health = getAccountHealth(accountId); + if (health?.lastFailureAt && now - health.lastFailureAt > CODEX_FAILURE_WINDOW_MS) return false; + return !!health && health.consecutiveFailures >= threshold; +} + +export function isHealthySharedCodexSelection( + config: OcxConfig, + accountId: string, + now: number, + quotaScope: CodexQuotaScope | undefined, + selectionOptions: CodexAccountUsabilityOptions | undefined, +): boolean { + return isCodexAccountSelectable(config, accountId, now, quotaScope, selectionOptions) + && hasCodexQuotaHeadroom(config, accountId, selectionOptions, now) + && !shouldFailover(config, accountId, now); +} + +export function strategySelectionOptionsForModelDetour( + config: OcxConfig, + now: number, + quotaScope: CodexQuotaScope | undefined, + selectionOptions: CodexAccountUsabilityOptions | undefined, +): CodexAccountUsabilityOptions | undefined { + if (selectionOptions?.modelEligibleAccountIds === undefined) return selectionOptions; + const sharedSelectionOptions = sharedStateSelectionOptions(selectionOptions) ?? {}; + return { + ...selectionOptions, + modelEligibleAccountIds: new Set( + [...selectionOptions.modelEligibleAccountIds].filter(accountId => + isHealthySharedCodexSelection( + config, + accountId, + now, + quotaScope, + sharedSelectionOptions, + ) + ), + ), + }; +} + +export function applyFailureFailover( + config: OcxConfig, + active: string, + now: number, + quotaScope?: CodexQuotaScope, + selectionOptions?: CodexAccountUsabilityOptions, + commitSharedSelection = true, +): string { + if (!shouldFailover(config, active, now)) return active; + const best = pickAlternateCodexAccount(config, active, now, quotaScope, selectionOptions); + if (best) { + // The scope still routes away from the failing account — that is this request's + // own decision — but an independent one must not persist a new shared active + // account. recordCodexUpstreamOutcome only suppresses the promotion it makes at + // the moment of the failure; the streak outlives the soft avoid, so a later + // scoped resolve reaches here with the streak still tripped and would otherwise + // move the shared cursor after all. + if (commitSharedSelection && !isIndependentCodexQuotaScope(quotaScope)) { + promoteActiveCodexAccount(config, best); + } + return best; + } + return active; +} diff --git a/src/codex/routing/thread-affinity.ts b/src/codex/routing/thread-affinity.ts new file mode 100644 index 0000000000..d8d2f5cbfb --- /dev/null +++ b/src/codex/routing/thread-affinity.ts @@ -0,0 +1,538 @@ +import { isCodexAccountGenerationLive, readCodexAccountRecord } from "../account-store"; +import { MAIN_CODEX_ACCOUNT_ID } from "../main-account"; +import { retainedUtf8Bytes } from "../../lib/admission"; +import { clearAllCodexPoolRefreshFailures } from "../pool-refresh-backoff"; +import type { CodexThreadLineage } from "../lineage"; +import type { CodexQuotaScope } from "./health-store"; + +export type ThreadAffinityEntry = { + accountId: string; + generation: number; + createdAt: number; + lastUsedAt: number; + // Last time the bound account's quota threshold was re-evaluated for this + // thread (interval-gated to avoid per-request flapping). See REEVAL_INTERVAL_MS. + lastReevalAt: number; + // When a transient failure streak first forced this thread onto another account + // while the binding was HELD (#4546). Cleared the moment the bound account serves + // again; once it ages past CODEX_TRANSIENT_AFFINITY_HOLD_MS the binding is + // released through the ordinary path instead of detouring forever. + transientHoldSince?: number; + // Which account is serving this thread while its own is held under a transient hold. + // Remembered rather than re-picked per request: under round-robin a fresh pick each turn + // would walk the ring and start cold on every hop, which is the behaviour the hold exists + // to prevent. Cleared with transientHoldSince when the bound account serves again. + transientDetourAccountId?: string; +}; + +export type CodexThreadResolution = + | { status: "selected"; accountId: string; affinity?: CodexAffinityDecision } + | { status: "none"; affinity?: CodexAffinityDecision } + | { status: "expired"; accountId: string; affinity?: CodexAffinityDecision }; + +/** What happened to this thread's binding on this request (#4546). */ +export type CodexAffinityMove = + /** Served by its own bound account, which was healthy. */ + | "reused" + /** Served by its own bound account while something transient was wrong with it. */ + | "held" + /** Served by another account while the binding stayed put. */ + | "detour" + /** The binding was released and a different account took the thread. */ + | "rebound" + /** There was no live binding; this request established one. */ + | "new_bind" + /** The binding was released without a replacement on this request. */ + | "cleared"; + +/** + * Why. A move is the expensive event -- it discards the prompt-cache prefix warmed on the old + * account -- so the operator should not have to infer it from account labels across log lines, + * which is how #4546 had to be diagnosed. + */ +export type CodexAffinityReason = + | "healthy" + | "quota_headroom" + | "quota_refusal" + | "transient" + | "transient_hold_expired" + | "unusable" + | "paused" + | "plan_excluded" + | "cooldown" + | "quota_avoided" + | "generation" + | "expired" + | "model_lane" + /** First placement followed the parent's CURRENT serving account (#4546, wp8). */ + | "lineage_parent" + /** First placement followed a compatible sibling's current serving account. */ + | "lineage_sibling"; + +export interface CodexAffinityDecision { + move: CodexAffinityMove; + reason: CodexAffinityReason; +} + +/** The decision to report once a binding has been released and selection starts over. */ +export function affinityAfterRelease( + threadId: string | null, + releaseReason: CodexAffinityReason | undefined, +): CodexAffinityDecision { + // Reported now, so it must not be reported again by the next request. + clearPendingReleaseReason(threadId); + return releaseReason === undefined + ? { move: "new_bind", reason: "healthy" } + : { move: "rebound", reason: releaseReason }; +} + +/** + * What to report when selection produced no account at all. The binding is gone and nothing took + * it, which is a `cleared`, and the pending reason is deliberately NOT consumed: a no-account + * result reaches no auth context and therefore no usage entry, so the next resolve that does + * produce one is the first place this release can actually be seen. + */ +export function affinityOnNoAccount( + threadId: string | null, + releaseReason: CodexAffinityReason | undefined, +): CodexAffinityDecision | undefined { + if (releaseReason === undefined) return undefined; + // Hand it forward as well as reporting it. A reason derived from the entry this request just + // released lives only in a local, so without this the next resolve finds no entry and no + // pending reason and calls the rebind a fresh healthy bind. + notePendingReleaseReason(threadId, releaseReason); + return { move: "cleared", reason: releaseReason }; +} + +export const CODEX_THREAD_AFFINITY_IDLE_TTL_MS = 24 * 60 * 60_000; +export const CODEX_THREAD_AFFINITY_MAX_ENTRIES = 2048; +const MAX_AFFINITY_COMPONENT_BYTES = 512; +// Min interval between quota threshold re-evaluations for a single bound thread. +// Well under the 5h/weekly quota windows, but enough to stop per-request flapping. +export const CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS = 60_000; + +/** + * How long a live binding outlives a TRANSIENT failure streak on its own account (#4546). + * + * Being unable to send right now is not the same as losing ownership of the conversation. + * A 5xx streak is frequently provider-wide rather than account-specific, and deleting the + * binding for it discards a prompt-cache prefix that the next turn then pays for again -- + * the same cost the quota threshold used to impose, arriving through a different door. + * So the request detours to another account while the binding is held here. + * + * Bounded, because an unbounded hold is its own defect: an account that never recovers + * would keep a thread detouring indefinitely while the conversation's real warm prefix + * accumulates somewhere else. Ten minutes is longer than the whole soft-avoid escalation + * ladder up to its final step, so an ordinary outage resolves inside the hold and a + * genuine one converts to a real rebind instead of a permanent detour. + */ +export const CODEX_TRANSIENT_AFFINITY_HOLD_MS = 10 * 60_000; + +/** + * Requests without a resolved native model retain the historic one-account-per- + * thread behavior. Requests with a known quota scope get an independent + * affinity so a Reserve failover cannot displace the same thread's Terra/Luna + * account (and vice versa). + */ +type BaseThreadAffinityScope = CodexQuotaScope | "legacy"; +type ModelDetourAffinityScope = `model-detour:${BaseThreadAffinityScope}:${string}`; +type ThreadAffinityScope = BaseThreadAffinityScope | ModelDetourAffinityScope; + +function isModelDetourAffinityScope(scope: ThreadAffinityScope): scope is ModelDetourAffinityScope { + return scope.startsWith("model-detour:"); +} +const LEGACY_THREAD_AFFINITY_SCOPE = "legacy" as const; +const threadAccountMap = new Map>(); +let threadAffinityEntryTotal = 0; + +/** + * Which pool account minted the conversation's carried OpenAI state + * (`previous_response_id`, encrypted reasoning, provider conversation/file ids). + * Keyed by the same affinity key as {@link threadAccountMap}, bounded the same + * way, and process-local — raw account ids never reach a log. + */ +type ConversationStateIssuerEntry = { + accountId: string; + lastUsedAt: number; +}; +const conversationStateIssuerMap = new Map(); + +export function clearThreadAccountMap(): void { + threadAccountMap.clear(); + threadAffinityEntryTotal = 0; + // A refresh cooldown is per-account runtime state learned alongside these bindings. Leaving it + // behind here keeps an account out of selection after the roster it belonged to is gone. + clearAllCodexPoolRefreshFailures(); + conversationStateIssuerMap.clear(); +} + +export function clearConversationStateIssuerMap(): void { + conversationStateIssuerMap.clear(); +} + +export function clearThreadAccountMapForAccount( + accountId: string, + reason: CodexAffinityReason = "unusable", +): void { + for (const [threadId, affinities] of threadAccountMap) { + for (const [scope, entry] of affinities) { + if (entry.accountId === accountId && affinities.delete(scope)) { + threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); + notePendingReleaseReason(threadId, reason); + } + } + if (affinities.size === 0) threadAccountMap.delete(threadId); + } +} + +function pruneConversationStateIssuers(now: number): void { + for (const [key, entry] of conversationStateIssuerMap) { + if (now - entry.lastUsedAt > CODEX_THREAD_AFFINITY_IDLE_TTL_MS) { + conversationStateIssuerMap.delete(key); + } + } + while (conversationStateIssuerMap.size > CODEX_THREAD_AFFINITY_MAX_ENTRIES) { + let oldestKey: string | null = null; + let oldestAt = Number.POSITIVE_INFINITY; + for (const [key, entry] of conversationStateIssuerMap) { + if (entry.lastUsedAt < oldestAt) { + oldestAt = entry.lastUsedAt; + oldestKey = key; + } + } + if (!oldestKey) break; + conversationStateIssuerMap.delete(oldestKey); + } +} + +/** + * Record the pool account that just issued carried conversation state for this + * binding key. In-memory only; the id is never written to a request log. + */ +export function rememberConversationStateIssuer( + bindingKey: string, + accountId: string, + now = Date.now(), +): void { + if (!bindingKey.trim() || !accountId.trim()) return; + if (!admissibleAffinityComponent(bindingKey) || !admissibleAffinityComponent(accountId)) return; + pruneConversationStateIssuers(now); + conversationStateIssuerMap.set(bindingKey, { accountId, lastUsedAt: now }); + pruneConversationStateIssuers(now); +} + +/** Last account that minted carried state for this binding, if still in the TTL window. */ +export function peekConversationStateIssuer( + bindingKey: string, + now = Date.now(), +): string | undefined { + if (!bindingKey.trim() || !admissibleAffinityComponent(bindingKey)) return undefined; + pruneConversationStateIssuers(now); + const entry = conversationStateIssuerMap.get(bindingKey); + if (!entry) return undefined; + entry.lastUsedAt = now; + return entry.accountId; +} + +/** + * Why a binding was released, held until that thread's next resolve can report it (#4546). + * + * A release and the request that pays for it are two different moments: a 429 clears the pin + * inside the outcome recorder, and the next request arrives with nothing left to explain why it + * is starting cold. Bounded, because it is a diagnostic and must not become a leak. + */ +const pendingReleaseReasons = new Map(); +const MAX_PENDING_RELEASE_REASONS = 4096; + +function notePendingReleaseReason(threadId: string | null, reason: CodexAffinityReason): void { + if (threadId === null) return; + if (!pendingReleaseReasons.has(threadId) && pendingReleaseReasons.size >= MAX_PENDING_RELEASE_REASONS) { + const oldest = pendingReleaseReasons.keys().next(); + if (!oldest.done) pendingReleaseReasons.delete(oldest.value); + } + pendingReleaseReasons.set(threadId, reason); +} + +export function peekPendingReleaseReason(threadId: string | null): CodexAffinityReason | undefined { + if (threadId === null) return undefined; + return pendingReleaseReasons.get(threadId); +} + +/** + * Forget a release only once it has actually been reported. + * + * Consuming it at derivation time lost it whenever selection then failed to produce an account: + * a no-account return carries no payload, so the release went unrecorded and the next successful + * resolve claimed a fresh healthy bind (#4598). A release survives until some resolve reports it. + */ +function clearPendingReleaseReason(threadId: string | null): void { + if (threadId !== null) pendingReleaseReasons.delete(threadId); +} + +function threadAffinityScope(quotaScope?: CodexQuotaScope): BaseThreadAffinityScope { + return quotaScope ?? LEGACY_THREAD_AFFINITY_SCOPE; +} + +function admissibleAffinityComponent(value: string): boolean { + return retainedUtf8Bytes(value) <= MAX_AFFINITY_COMPONENT_BYTES; +} + +function modelDetourAffinityScope( + modelId: string | undefined, + quotaScope?: CodexQuotaScope, +): ModelDetourAffinityScope | undefined { + const canonicalModelId = modelId?.trim().toLowerCase(); + if (!canonicalModelId || !admissibleAffinityComponent(canonicalModelId)) return undefined; + return `model-detour:${threadAffinityScope(quotaScope)}:${canonicalModelId}`; +} + +function getThreadAffinityForScope( + threadId: string, + scope: ThreadAffinityScope, +): ThreadAffinityEntry | undefined { + if (!admissibleAffinityComponent(threadId)) return undefined; + return threadAccountMap.get(threadId)?.get(scope); +} + +export function getThreadAffinity(threadId: string, quotaScope?: CodexQuotaScope): ThreadAffinityEntry | undefined { + return getThreadAffinityForScope(threadId, threadAffinityScope(quotaScope)); +} + +export function getModelDetourAffinity( + threadId: string, + modelId: string | undefined, + quotaScope?: CodexQuotaScope, +): ThreadAffinityEntry | undefined { + const scope = modelDetourAffinityScope(modelId, quotaScope); + return scope ? getThreadAffinityForScope(threadId, scope) : undefined; +} + +function deleteThreadAffinityForScope(threadId: string, scope: ThreadAffinityScope): void { + if (!admissibleAffinityComponent(threadId)) return; + const affinities = threadAccountMap.get(threadId); + if (!affinities) return; + if (affinities.delete(scope)) { + threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); + } + if (affinities.size === 0) threadAccountMap.delete(threadId); +} + +export function deleteThreadAffinity(threadId: string, quotaScope?: CodexQuotaScope): void { + deleteThreadAffinityForScope(threadId, threadAffinityScope(quotaScope)); +} + +export function deleteModelDetourAffinity( + threadId: string, + modelId: string | undefined, + quotaScope?: CodexQuotaScope, +): void { + const scope = modelDetourAffinityScope(modelId, quotaScope); + if (scope) deleteThreadAffinityForScope(threadId, scope); +} + +/** Remove only the matching failed account's affinities for one thread. */ +export function deleteThreadAffinitiesForAccount(threadId: string, accountId: string): void { + if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; + const affinities = threadAccountMap.get(threadId); + if (!affinities) return; + for (const [scope, entry] of affinities) { + if (entry.accountId === accountId && affinities.delete(scope)) { + threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); + } + } + if (affinities.size === 0) threadAccountMap.delete(threadId); +} + +function threadAffinityEntryCount(): number { + return threadAffinityEntryTotal; +} + +export function isThreadAffinityExpired(entry: ThreadAffinityEntry, now: number): boolean { + return now - entry.lastUsedAt > CODEX_THREAD_AFFINITY_IDLE_TTL_MS; +} + +export function isThreadAffinityGenerationLive(entry: ThreadAffinityEntry): boolean { + if (entry.accountId === MAIN_CODEX_ACCOUNT_ID) return entry.generation === 0; + return isCodexAccountGenerationLive(entry.accountId, entry.generation); +} + +/** Generations this account's affinity entries are bound at. Test observability only. */ +export function debugCodexAffinityGenerations(accountId: string): number[] { + const generations: number[] = []; + for (const affinities of threadAccountMap.values()) { + for (const entry of affinities.values()) { + if (entry.accountId === accountId) generations.push(entry.generation); + } + } + return generations; +} + +/** + * Advance this account's affinity entries from the generation a rejected credential + * was bound under to the generation its own refresh produced. + * + * A 401 refresh-and-replay keeps the request on the same account, but the CAS write + * moves the credential from G to G+1, and {@link isThreadAffinityGenerationLive} + * demands exact equality — so without this the entry the replay just preserved is + * dead on the next request. Not quarantining an account is not the same as keeping + * its affinity. + * + * Lineage is proven by the CALLER, which must pass only a generation its own refresh + * produced. Re-deriving it here from `replacedAt` cannot work: the caller reads that + * field after the refresh and this function would re-read the same record, so the + * comparison is tautological and an external replacement passes it. An external + * replacement must retire the affinity, because that credential may belong to a + * different upstream identity. + */ +export function handOffThreadAffinityGeneration( + accountId: string, + fromGeneration: number, + toGeneration: number, +): boolean { + if (accountId === MAIN_CODEX_ACCOUNT_ID) return false; + if (toGeneration !== fromGeneration + 1) return false; + const record = readCodexAccountRecord(accountId); + if (!record?.credential || record.deletedAt != null) return false; + if (record.generation !== toGeneration) return false; + let handedOff = false; + for (const affinities of threadAccountMap.values()) { + for (const entry of affinities.values()) { + if (entry.accountId !== accountId || entry.generation !== fromGeneration) continue; + entry.generation = toGeneration; + handedOff = true; + } + } + return handedOff; +} + +function pruneExpiredThreadAffinities(now: number): void { + for (const [threadId, affinities] of threadAccountMap) { + for (const [scope, entry] of affinities) { + if (isThreadAffinityExpired(entry, now) && affinities.delete(scope)) { + threadAffinityEntryTotal = Math.max(0, threadAffinityEntryTotal - 1); + } + } + if (affinities.size === 0) threadAccountMap.delete(threadId); + } +} + +function pruneLruThreadAffinities(): void { + if (threadAffinityEntryCount() <= CODEX_THREAD_AFFINITY_MAX_ENTRIES) return; + while (threadAffinityEntryCount() > CODEX_THREAD_AFFINITY_MAX_ENTRIES) { + let oldestThreadId: string | null = null; + let oldestScope: ThreadAffinityScope | null = null; + let oldestLastUsedAt = Number.POSITIVE_INFINITY; + let oldestIsDetour = false; + for (const [threadId, affinities] of threadAccountMap) { + for (const [scope, entry] of affinities) { + const candidateIsDetour = isModelDetourAffinityScope(scope); + if ( + (candidateIsDetour && !oldestIsDetour) + || (candidateIsDetour === oldestIsDetour && entry.lastUsedAt < oldestLastUsedAt) + ) { + oldestThreadId = threadId; + oldestScope = scope; + oldestLastUsedAt = entry.lastUsedAt; + oldestIsDetour = candidateIsDetour; + } + } + } + if (!oldestThreadId || !oldestScope) return; + deleteThreadAffinityForScope(oldestThreadId, oldestScope); + } +} + +function bindThreadAffinityForScope( + threadId: string, + accountId: string, + now: number, + scope: ThreadAffinityScope, +): void { + if (!admissibleAffinityComponent(threadId) || !admissibleAffinityComponent(accountId)) return; + const record = accountId === MAIN_CODEX_ACCOUNT_ID ? undefined : readCodexAccountRecord(accountId); + if (accountId !== MAIN_CODEX_ACCOUNT_ID && (!record?.credential || record.deletedAt != null)) return; + pruneExpiredThreadAffinities(now); + const affinities = threadAccountMap.get(threadId) ?? new Map(); + const previous = affinities.get(scope); + affinities.set(scope, { + accountId, + generation: accountId === MAIN_CODEX_ACCOUNT_ID ? 0 : record!.generation, + createdAt: previous?.createdAt ?? now, + lastUsedAt: now, + lastReevalAt: now, + }); + if (!previous) threadAffinityEntryTotal += 1; + threadAccountMap.set(threadId, affinities); + pruneLruThreadAffinities(); +} + +export function bindThreadAffinity( + threadId: string, + accountId: string, + now: number, + quotaScope?: CodexQuotaScope, +): void { + bindThreadAffinityForScope(threadId, accountId, now, threadAffinityScope(quotaScope)); +} + +export function bindModelDetourAffinity( + threadId: string, + accountId: string, + now: number, + modelId: string | undefined, + quotaScope?: CodexQuotaScope, +): void { + const scope = modelDetourAffinityScope(modelId, quotaScope); + if (scope) bindThreadAffinityForScope(threadId, accountId, now, scope); +} + +/** Read-only view of one thread's scope-keyed affinity entries. */ +export function getThreadAffinityScopes( + threadId: string, +): ReadonlyMap | undefined { + return threadAccountMap.get(threadId); +} + +/** + * Move one scope's binding from the pre-#4546 RAW parent key onto the key this thread uses now. + * + * Bindings and the key that derives them are process-local, so an ordinary restart already + * discards every binding and there is nothing to migrate. The case this exists for is the + * narrow one: a code swap under a live conversation, where the map still holds entries made by + * the old rule. Rebinding those cold is precisely the defect the lineage work exists to prevent, + * so the conversation keeps its account and the legacy entry is retired in the same step. + * + * One way, once. The legacy entry is deleted even when it was dead on arrival, because nothing + * can reach it again under the new rule and an orphan only spends an LRU slot a live + * conversation needs. Only the account moves: a transient hold describes a failure happening + * right now, and the ordinary path re-derives it on this very request. + */ +function adoptLegacyAffinityForScope( + threadId: string, + legacyKey: string, + now: number, + scope: ThreadAffinityScope, +): void { + if (getThreadAffinityForScope(threadId, scope) !== undefined) return; + const legacy = getThreadAffinityForScope(legacyKey, scope); + if (legacy === undefined) return; + if (!isThreadAffinityExpired(legacy, now) && isThreadAffinityGenerationLive(legacy)) { + bindThreadAffinityForScope(threadId, legacy.accountId, now, scope); + } + deleteThreadAffinityForScope(legacyKey, scope); +} + +/** Both lanes of the legacy migration: the ordinary binding and this request's model detour. */ +export function adoptLegacyLineageAffinity( + threadId: string, + lineage: CodexThreadLineage | undefined, + now: number, + quotaScope?: CodexQuotaScope, + modelId?: string, +): void { + const legacyKey = lineage?.legacyConversationKey; + if (legacyKey === undefined || legacyKey === threadId) return; + adoptLegacyAffinityForScope(threadId, legacyKey, now, threadAffinityScope(quotaScope)); + const detourScope = modelDetourAffinityScope(modelId, quotaScope); + if (detourScope) adoptLegacyAffinityForScope(threadId, legacyKey, now, detourScope); +} diff --git a/src/codex/shim-fingerprint.ts b/src/codex/shim-fingerprint.ts new file mode 100644 index 0000000000..619d95eb05 --- /dev/null +++ b/src/codex/shim-fingerprint.ts @@ -0,0 +1,223 @@ +import { + closeSync, + existsSync, + lstatSync, + linkSync, + openSync, + readSync, + readlinkSync, + statSync, + symlinkSync, + unlinkSync, +} from "node:fs"; +import { posix, win32 } from "node:path"; +import { SHIM_MARKER, UNIX_SHIM_REVISION_MARKER } from "./shim-templates"; +import type { ShimFileState } from "./shim-state-file"; + +const CODEX_SHIM_PROBE_BYTES = 16 * 1024; + +interface ShimPathFingerprint { + dev: number; + ino: number; + kind: "file" | "symlink"; + mode: number; + size: number; + mtimeMs: number; + ctimeMs: number; + target?: Omit; +} + +interface StableShimPathProbe { + fingerprint: ShimPathFingerprint; + prefix: string; +} + +function readShimProbePrefix(path: string): string { + const fd = openSync(path, "r"); + try { + const buffer = Buffer.allocUnsafe(CODEX_SHIM_PROBE_BYTES); + const bytesRead = readSync(fd, buffer, 0, buffer.length, 0); + return buffer.toString("utf8", 0, bytesRead); + } finally { + closeSync(fd); + } +} + +function statFingerprint(path: string, follow: boolean): Omit | null { + try { + const stat = follow ? statSync(path) : lstatSync(path); + if (follow ? !stat.isFile() : (!stat.isFile() && !stat.isSymbolicLink())) return null; + return { + dev: stat.dev, + ino: stat.ino, + kind: stat.isSymbolicLink() ? "symlink" : "file", + mode: stat.mode, + size: stat.size, + mtimeMs: stat.mtimeMs, + ctimeMs: stat.ctimeMs, + }; + } catch { + return null; + } +} + +function sameFingerprint( + left: ShimPathFingerprint | Omit, + right: ShimPathFingerprint | Omit, +): boolean { + return left.dev === right.dev + && left.ino === right.ino + && left.kind === right.kind + && left.mode === right.mode + && left.size === right.size + && left.mtimeMs === right.mtimeMs + && left.ctimeMs === right.ctimeMs + && (!("target" in left) || !("target" in right) + ? true + : left.target === undefined && right.target === undefined + ? true + : left.target !== undefined && right.target !== undefined + ? sameFingerprint(left.target, right.target) + : false); +} + +function sameFingerprintAfterRename(left: ShimPathFingerprint, right: ShimPathFingerprint): boolean { + // rename changes the outer directory entry ctime on macOS; every other field, + // including a symlink target fingerprint, must remain identical. + return sameFingerprint({ ...left, ctimeMs: 0 }, { ...right, ctimeMs: 0 }); +} + +function stableShimPathProbe(path: string): StableShimPathProbe | null { + const before = statFingerprint(path, false); + if (!before) return null; + const targetBefore = before.kind === "symlink" ? statFingerprint(path, true) : undefined; + if (before.kind === "symlink" && !targetBefore) return null; + let prefix: string; + try { + prefix = readShimProbePrefix(path); + } catch { + return null; + } + const targetAfter = before.kind === "symlink" ? statFingerprint(path, true) : undefined; + const after = statFingerprint(path, false); + if (!after || !sameFingerprint(before, after)) return null; + if (before.kind === "symlink") { + if (!targetBefore || !targetAfter || !sameFingerprint(targetBefore, targetAfter)) return null; + } + const fingerprint: ShimPathFingerprint = { + ...before, + ...(targetBefore ? { target: targetBefore } : {}), + }; + const contentSize = fingerprint.target?.size ?? fingerprint.size; + return contentSize > 0 ? { fingerprint, prefix } : null; +} + +function sameStableShimPathProbe(left: StableShimPathProbe, right: StableShimPathProbe): boolean { + return left.prefix === right.prefix && sameFingerprint(left.fingerprint, right.fingerprint); +} + +/** + * Identity of whatever sits at `path`, read from metadata alone. + * + * `stableShimPathProbe` answers a different question: it reads content to decide + * whether a launcher looks like a healthy shim, and it deliberately returns null + * for a zero-byte file. That makes it the wrong instrument for rollback + * bookkeeping. A user can legitimately own an empty `codex` launcher, and a fresh + * install moves it aside before writing our wrapper; if the move is recorded + * without a fingerprint, rollback cannot prove the backup is still the file it + * set aside and refuses to restore it — the launcher stays lost (#1625). + * + * Content is irrelevant to that proof, so this reads dev/ino/mode/size/times and + * re-reads them to reject a path that changed under us, following a symlink to + * fingerprint its target as well. + */ +function shimPathFingerprint(path: string): ShimPathFingerprint | null { + const before = statFingerprint(path, false); + if (!before) return null; + if (before.kind !== "symlink") { + const after = statFingerprint(path, false); + return after && sameFingerprint(before, after) ? before : null; + } + const targetBefore = statFingerprint(path, true); + if (!targetBefore) return null; + const targetAfter = statFingerprint(path, true); + const after = statFingerprint(path, false); + if (!targetAfter || !after + || !sameFingerprint(targetBefore, targetAfter) + || !sameFingerprint(before, after)) return null; + return { ...before, target: targetBefore }; +} + +/** + * Move `from` onto `to` without ever replacing an existing entry. + * + * `renameSync` silently clobbers the destination on POSIX, which is wrong for a + * rollback restore: `sourceOccupied` is sampled before the fingerprint check, so + * a concurrent installer can publish its own launcher at the original path in + * between, and the restore would delete it. `link` fails EEXIST instead, which + * is the no-replace primitive we need and needs no native helper. + * + * `link` follows a symlink to its target rather than preserving the link, so a + * symlink launcher is republished with `symlink`, which is also no-replace: it + * fails EEXIST on an occupied destination. Checking existence and then renaming + * would reintroduce exactly the race this function exists to close. + */ +function restoreWithoutReplacing(from: string, to: string): void { + const source = lstatSync(from); + if (source.isSymbolicLink()) { + symlinkSync(readlinkSync(from), to); + unlinkSync(from); + return; + } + linkSync(from, to); + unlinkSync(from); +} + +function isHealthyShimProbe(probe: StableShimPathProbe, platform: NodeJS.Platform): boolean { + if (probe.prefix.length < 180 || !probe.prefix.includes(SHIM_MARKER) || !probe.prefix.includes("ensure")) return false; + const mode = probe.fingerprint.target?.mode ?? probe.fingerprint.mode; + return platform === "win32" || (mode & 0o111) !== 0; +} + +function isCurrentUnixShimProbe(probe: StableShimPathProbe): boolean { + return probe.prefix.includes(UNIX_SHIM_REVISION_MARKER); +} + +function hasUsableBackingPath(file: ShimFileState): boolean { + return [existsSync(file.backupPath) ? file.backupPath : undefined, file.realPath] + .some(path => { + if (!path) return false; + const fingerprint = statFingerprint(path, true); + return fingerprint !== null && fingerprint.size > 0; + }); +} + +/** + * True when a Codex binary lives inside a version manager's install tree. + * + * These trees are rewritten in place on upgrade, which destroys both the shim + * and the sibling .opencodex-real backup it restores from (#2412). The tempting + * repair — adopt the newly installed binary as a fresh original — is wrong + * twice: it records a provenance that never happened, and the next upgrade wipes + * it again, so the repair silently un-repairs on the version manager's schedule. + * + * Scope is the three managers named in the report. nvm/fnm/npm-prefix are + * deliberately excluded: a false positive here refuses a restore that would + * otherwise be correct. + */ +export function isVersionManagerOwnedCodexPath( + path: string, + platform: NodeJS.Platform = process.platform, +): boolean { + const normalized = (platform === "win32" + ? win32.normalize(path).replace(/\\/g, "/") + : posix.normalize(path)).toLowerCase(); + return normalized.includes("/mise/installs/") + || normalized.includes("/mise/shims/") + || normalized.includes("/.asdf/installs/") + || normalized.includes("/.asdf/shims/") + || normalized.includes("/.volta/"); +} + +export type { ShimPathFingerprint, StableShimPathProbe }; +export { statFingerprint, sameFingerprint, sameFingerprintAfterRename, stableShimPathProbe, sameStableShimPathProbe, shimPathFingerprint, restoreWithoutReplacing, isHealthyShimProbe, isCurrentUnixShimProbe, hasUsableBackingPath }; diff --git a/src/codex/shim-inspect.ts b/src/codex/shim-inspect.ts new file mode 100644 index 0000000000..dd696d5bcb --- /dev/null +++ b/src/codex/shim-inspect.ts @@ -0,0 +1,175 @@ +import { lstatSync } from "node:fs"; +import { extname, join, posix, win32 } from "node:path"; +import { getConfigDir } from "../config"; +import { fileErrorCode, readStateResult, stateFiles } from "./shim-state-file"; +import { + isHealthyShimProbe, + isVersionManagerOwnedCodexPath, + shimPathFingerprint, + stableShimPathProbe, + statFingerprint, + type ShimPathFingerprint, +} from "./shim-fingerprint"; +import { gitBashPath, psString, shQuote, windowsBatchSet } from "./shim-templates"; + +export type CodexShimBackingForCommand = + | Readonly<{ status: "not-tracked" }> + | Readonly<{ + status: "matched"; + selectedRole: "wrapper" | "backing"; + backingPath: string; + backingKind: "backup" | "real"; + }> + | Readonly<{ + status: "unknown"; + reason: + | "state_invalid" + | "platform_mismatch" + | "ambiguous_match" + | "preserve_only" + | "backing_missing" + | "backing_mismatch" + | "binding_unavailable" + | "wrapper_unhealthy" + | "version_manager_refused"; + }>; + +export function isLocalAbsoluteInspectionPath(path: string, platform: NodeJS.Platform): boolean { + if (platform !== "win32") return posix.isAbsolute(path); + const normalized = path.replace(/\//g, "\\"); + // UNC and device namespaces can initiate remote I/O while a nominally local + // inspection is resolving user-controlled paths. Root-relative paths are + // drive-context dependent, so require an explicit local drive as well. + return win32.isAbsolute(path) + && /^[a-z]:\\/i.test(normalized) + && !normalized.startsWith("\\\\"); +} + +function windowsShimInspectionIsDeferred(platform: NodeJS.Platform): boolean { + return platform === "win32"; +} + +/** Resolve one selected command through already-recorded shim state, without repair. */ +export function inspectCodexShimBackingForCommand( + selectedCommand: string, + platform: NodeJS.Platform = process.platform, + configDir: string = getConfigDir(), +): CodexShimBackingForCommand { + // Pathname prechecks cannot prevent a writable Windows ancestor from being + // replaced with a remote reparse point before the later state/fingerprint + // reads. Keep the exported read-only helper fail-closed until those reads are + // performed through a handle-bound Windows provenance layer. + if (windowsShimInspectionIsDeferred(platform)) { + return Object.freeze({ status: "unknown" as const, reason: "binding_unavailable" as const }); + } + if (!isLocalAbsoluteInspectionPath(configDir, platform)) { + return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); + } + const stateFile = join(configDir, "codex-shim.json"); + try { + const stateEntry = lstatSync(stateFile); + if (stateEntry.isSymbolicLink()) { + return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); + } + } catch (error) { + if (fileErrorCode(error) !== "ENOENT") { + return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); + } + } + const result = readStateResult(stateFile); + if (!result.state) { + return result.present + ? Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }) + : Object.freeze({ status: "not-tracked" as const }); + } + const pathApi = platform === "win32" ? win32 : posix; + const samePath = (left: string, right: string): boolean => { + const normalizedLeft = pathApi.resolve(left); + const normalizedRight = pathApi.resolve(right); + return platform === "win32" + ? normalizedLeft.toLowerCase() === normalizedRight.toLowerCase() + : normalizedLeft === normalizedRight; + }; + const files = stateFiles(result.state); + if (files.some(file => !file.wrapperPath || !file.originalPath || !file.backupPath + || ![file.wrapperPath, file.originalPath, file.backupPath, file.realPath] + .filter((path): path is string => typeof path === "string") + .every(path => isLocalAbsoluteInspectionPath(path, platform)))) { + return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); + } + const wrapperKeys = files.map(file => platform === "win32" + ? pathApi.resolve(file.wrapperPath).toLowerCase() + : pathApi.resolve(file.wrapperPath)); + if (new Set(wrapperKeys).size !== wrapperKeys.length) { + return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); + } + const selectedFingerprint = shimPathFingerprint(selectedCommand); + if (!selectedFingerprint) { + return Object.freeze({ status: "unknown" as const, reason: "binding_unavailable" as const }); + } + const selectedIdentity = selectedFingerprint.target ?? selectedFingerprint; + const sameEffectiveIdentity = (fingerprint: ShimPathFingerprint | null): boolean => { + if (!fingerprint) return false; + const identity = fingerprint.target ?? fingerprint; + return identity.dev === selectedIdentity.dev && identity.ino === selectedIdentity.ino; + }; + const matches = files.flatMap(file => { + const backingPath = file.realPath ?? file.backupPath; + const roles: Array<"wrapper" | "backing"> = []; + if (samePath(file.wrapperPath, selectedCommand) + || sameEffectiveIdentity(shimPathFingerprint(file.wrapperPath))) { + roles.push("wrapper"); + } + if (samePath(backingPath, selectedCommand) + || sameEffectiveIdentity(shimPathFingerprint(backingPath))) { + roles.push("backing"); + } + return roles.map(selectedRole => ({ file, backingPath, selectedRole })); + }); + if (matches.length === 0) return Object.freeze({ status: "not-tracked" as const }); + if (result.state.platform !== platform) { + return Object.freeze({ status: "unknown" as const, reason: "platform_mismatch" as const }); + } + if (matches.length !== 1) { + return Object.freeze({ status: "unknown" as const, reason: "ambiguous_match" as const }); + } + const { file, backingPath, selectedRole } = matches[0]!; + if (file.preserveOnly === true) { + return Object.freeze({ status: "unknown" as const, reason: "preserve_only" as const }); + } + const backing = statFingerprint(backingPath, true); + if (!backing || backing.size <= 0 || samePath(backingPath, file.wrapperPath)) { + return Object.freeze({ status: "unknown" as const, reason: "backing_missing" as const }); + } + const wrapperProbe = stableShimPathProbe(file.wrapperPath); + if (!wrapperProbe || !isHealthyShimProbe(wrapperProbe, result.state.platform)) { + return Object.freeze({ + status: "unknown" as const, + reason: isVersionManagerOwnedCodexPath(file.wrapperPath) + ? "version_manager_refused" as const + : "wrapper_unhealthy" as const, + }); + } + const wrapperIdentity = wrapperProbe.fingerprint.target ?? wrapperProbe.fingerprint; + if (backing.dev === wrapperIdentity.dev && backing.ino === wrapperIdentity.ino) { + return Object.freeze({ status: "unknown" as const, reason: "backing_mismatch" as const }); + } + const wrapperExt = extname(file.wrapperPath).toLowerCase(); + const invokesBacking = platform !== "win32" + ? wrapperProbe.prefix.includes(`exec ${shQuote(backingPath)} "$@"`) + : wrapperExt === ".cmd" || wrapperExt === ".bat" + ? wrapperProbe.prefix.includes(windowsBatchSet("OCX_REAL_CODEX", backingPath)) + && wrapperProbe.prefix.includes('"%OCX_REAL_CODEX%" %*') + : wrapperExt === ".ps1" + ? wrapperProbe.prefix.includes(`& ${psString(backingPath)} @args`) + : wrapperProbe.prefix.includes(`exec ${shQuote(gitBashPath(backingPath))} "$@"`); + if (!invokesBacking) { + return Object.freeze({ status: "unknown" as const, reason: "backing_mismatch" as const }); + } + return Object.freeze({ + status: "matched" as const, + selectedRole, + backingPath, + backingKind: file.realPath !== undefined ? "real" as const : "backup" as const, + }); +} diff --git a/src/codex/shim-probe.ts b/src/codex/shim-probe.ts new file mode 100644 index 0000000000..c99a573b7e --- /dev/null +++ b/src/codex/shim-probe.ts @@ -0,0 +1,367 @@ +import { spawnSync } from "node:child_process"; +import { tmpdir } from "node:os"; +import { + chmodSync, + existsSync, + lstatSync, + mkdtempSync, + readFileSync, + rmSync, +} from "node:fs"; +import { join } from "node:path"; +import { CODEX_SHIM_REENTRY_EXIT_CODE, CODEX_SHIM_REENTRY_DIAGNOSTIC } from "./shim-templates"; +import type { ShimFileState } from "./shim-state-file"; + +const CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS = 5_000; +const CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS = 1_000; + +const CODEX_SHIM_INSTALL_PROBE_SCRIPT = ` +const { spawn } = require("node:child_process"); +const { readFileSync, writeFileSync } = require("node:fs"); +const [markerPath, reentryPath, groupPath, stderrPath, launcherShellPath, wrapperPath, timeoutRaw, stderrLimitRaw, stderrDrainRaw, observationRaw] = process.argv.slice(1); +const timeoutMs = Number.parseInt(timeoutRaw, 10); +const stderrLimit = Number.parseInt(stderrLimitRaw, 10); +const stderrDrainMs = Number.parseInt(stderrDrainRaw, 10); +const observationMs = Number.parseInt(observationRaw, 10); +const probeStartedAt = Date.now(); +const stderrChunks = []; +let stderrBytes = 0; +let launcher; +let probeLease; +let timer; +let stderrDrainTimer; +let observationTimer; +let reentryPollTimer; +let marker = ""; +let finished = false; + +function writeExclusive(path, value) { + writeFileSync(path, value, { flag: "wx", mode: 0o600 }); +} + +function appendStderr(value) { + if (stderrBytes >= stderrLimit) return; + const bytes = Buffer.from(value); + const retained = bytes.subarray(0, stderrLimit - stderrBytes); + stderrChunks.push(retained); + stderrBytes += retained.byteLength; +} + +function groupAlive() { + if (!launcher || !launcher.pid) return false; + try { + process.kill(-launcher.pid, 0); + return true; + } catch (error) { + return error && error.code !== "ESRCH"; + } +} + +function killGroup() { + if (!launcher || !launcher.pid) return; + try { process.kill(-launcher.pid, "SIGKILL"); } catch (error) { + if (!error || error.code !== "ESRCH") appendStderr(String(error)); + } +} + +function setMarker(value) { + if (marker) return; + marker = value; + try { writeExclusive(markerPath, value + "\\n"); } catch (error) { appendStderr(String(error)); } +} + +function reentryDetected() { + try { return readFileSync(reentryPath, "utf8").trim() === "recursive"; } catch { return false; } +} + +function checkReentry() { + if (finished || !reentryDetected()) return; + setMarker("recursive"); + killGroup(); + finish(126); +} + +function finish(status) { + if (finished) return; + finished = true; + if (timer) clearTimeout(timer); + if (stderrDrainTimer) clearTimeout(stderrDrainTimer); + if (observationTimer) clearTimeout(observationTimer); + if (reentryPollTimer) clearInterval(reentryPollTimer); + if (!marker && reentryDetected()) setMarker("recursive"); + if (!marker && groupAlive()) { + setMarker("descendants"); + killGroup(); + } + try { writeExclusive(stderrPath, Buffer.concat(stderrChunks)); } catch { /* parent fails closed */ } + process.exit(marker === "timeout" ? 124 : marker === "descendants" ? 125 : marker === "recursive" ? 126 : status); +} + +function finishAfterStderr(status) { + if (finished) return; + if (timer) { + clearTimeout(timer); + timer = undefined; + } + if (!launcher || !launcher.stderr || !probeLease) { + finish(status); + return; + } + let stderrEnded = launcher.stderr.readableEnded; + let leaseEnded = probeLease.readableEnded; + let observationElapsed = false; + const finishWhenReady = () => { + if (stderrEnded && leaseEnded && observationElapsed) finish(status); + }; + launcher.stderr.once("end", () => { + stderrEnded = true; + finishWhenReady(); + }); + probeLease.once("end", () => { + leaseEnded = true; + finishWhenReady(); + }); + stderrDrainTimer = setTimeout(() => { + stderrEnded = true; + if (!marker && groupAlive()) { + setMarker("descendants"); + killGroup(); + finish(125); + return; + } + finishWhenReady(); + }, stderrDrainMs); + const remainingObservationMs = Math.max(0, observationMs - (Date.now() - probeStartedAt)); + observationTimer = setTimeout(() => { + observationElapsed = true; + if (!leaseEnded) { + setMarker(groupAlive() ? "descendants" : "timeout"); + killGroup(); + finish(marker === "descendants" ? 125 : 124); + return; + } + finishWhenReady(); + }, remainingObservationMs); + finishWhenReady(); +} + +try { + launcher = spawn(launcherShellPath, [wrapperPath, "--version"], { + detached: true, + env: process.env, + stdio: ["ignore", "ignore", "pipe", "pipe"], + }); + if (!launcher.pid) throw new Error("Codex shim probe launcher has no pid"); + probeLease = launcher.stdio[3]; + if (!probeLease) throw new Error("Codex shim probe launcher has no descendant lease pipe"); + writeExclusive(groupPath, String(launcher.pid) + "\\n"); + launcher.stderr.on("data", appendStderr); + reentryPollTimer = setInterval(checkReentry, 10); + launcher.once("error", error => { + appendStderr(String(error)); + finishAfterStderr(127); + }); + launcher.once("exit", code => finishAfterStderr(Number.isInteger(code) ? code : 127)); + timer = setTimeout(() => { + setMarker("timeout"); + killGroup(); + finish(124); + }, timeoutMs); +} catch (error) { + appendStderr(String(error)); + killGroup(); + finish(127); +} +`; +const MAX_DIAGNOSTIC_VALUE_BYTES = 8 * 1024; + +type UnixShimProbeCleanupPhase = "marker" | "reentry" | "group" | "stderr" | "group-id" | "termination" | "spawn" | "exception"; +interface UnixShimProbeCleanup { + kind: "cleanup"; + phase: UnixShimProbeCleanupPhase; + code: string; + status: number | null; + signal: string; +} +type UnixShimProbeResult = UnixShimProbeCleanup | "descendants" | "failed" | "recursive" | "timeout" | null; + +const SHIM_PROBE_ERROR_CODES = new Set([ + "EACCES", "EAGAIN", "EBADF", "ECANCELED", "EINTR", "EIO", "EMFILE", "ENFILE", + "ENOENT", "ENOEXEC", "ENOMEM", "ENOSPC", "EPERM", "EPIPE", "ESRCH", "ETIMEDOUT", "ETXTBSY", +]); +const SHIM_PROBE_SIGNALS = new Set([ + "SIGABRT", "SIGBUS", "SIGHUP", "SIGILL", "SIGINT", "SIGKILL", "SIGPIPE", "SIGQUIT", + "SIGSEGV", "SIGTERM", "SIGTRAP", "SIGXCPU", "SIGXFSZ", +]); + +/** Diagnostics cross a CLI boundary: never stringify arbitrary errors or metadata. */ +function shimProbeCleanup( + phase: UnixShimProbeCleanupPhase, error?: unknown, status?: unknown, signal?: unknown, +): UnixShimProbeCleanup { + let code = error === undefined ? "none" : "unknown"; + if (error !== null && typeof error === "object") { + try { + const value = Object.getOwnPropertyDescriptor(error, "code")?.value; + if (typeof value === "string" && SHIM_PROBE_ERROR_CODES.has(value)) code = value; + } catch { /* hostile accessors/proxies cannot turn diagnostics into an exception */ } + } + return { + kind: "cleanup", phase, code, + status: typeof status === "number" && Number.isInteger(status) && status >= 0 && status <= 255 ? status : null, + signal: typeof signal === "string" && SHIM_PROBE_SIGNALS.has(signal) ? signal : "none", + }; +} + +let codexShimProbeHookForTests: (() => void) | null = null; +let codexShimProbeShellForTests: string | null = null; + +let codexShimProbeObservationMs = CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS; + +/** Narrow deterministic seam for transaction rollback tests. */ +export function setCodexShimProbeHookForTests(hook: (() => void) | null): void { + codexShimProbeHookForTests = hook; +} + +/** Selects a POSIX shell only for cross-shell probe regression tests. */ +export function setCodexShimProbeShellForTests(path: string | null): void { + codexShimProbeShellForTests = path; +} + +/** Shortens the successful-launcher observation window only for focused tests. */ +export function setCodexShimProbeObservationMsForTests(value: number | null): void { + codexShimProbeObservationMs = value ?? CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS; +} + +function readProbeMetadata(path: string, maxBytes: number): string | null { + try { + if (!existsSync(path)) return ""; + const stat = lstatSync(path); + if (!stat.isFile() || stat.size > maxBytes) return null; + return readFileSync(path, "utf8").trim(); + } catch { + return null; + } +} + +function probeUnixShimInstall(wrapperPath: string): UnixShimProbeResult { + if (process.platform === "win32") return null; + const probeDir = mkdtempSync(join(tmpdir(), "opencodex-shim-probe-")); + const markerPath = join(probeDir, "result"); + const reentryPath = join(probeDir, "reentry"); + const groupPath = join(probeDir, "group"); + const stderrPath = join(probeDir, "stderr"); + const env: NodeJS.ProcessEnv = { + ...process.env, + OCX_SHIM_BYPASS: "1", + OCX_SHIM_PROBE: "1", + OCX_SHIM_PROBE_REENTRY_PATH: reentryPath, + }; + delete env.OCX_SHIM_ACTIVE_PID; + delete env.OCX_SHIM_ACTIVE_DEPTH; + delete env.OCX_SHIM_PROBE_ACTIVE; + let groupId = 0; + let probeStatus: unknown; + let probeSignal: unknown; + try { + chmodSync(probeDir, 0o700); + const result = spawnSync(process.execPath, [ + "-e", + CODEX_SHIM_INSTALL_PROBE_SCRIPT, + markerPath, + reentryPath, + groupPath, + stderrPath, + codexShimProbeShellForTests ?? "/bin/sh", + wrapperPath, + String(CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS), + String(MAX_DIAGNOSTIC_VALUE_BYTES), + String(CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS), + String(codexShimProbeObservationMs), + ], { + encoding: "utf8", + env, + timeout: CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS + CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS, + killSignal: "SIGKILL", + }); + probeStatus = result.status; + probeSignal = result.signal; + const timedOut = (result.error as NodeJS.ErrnoException | undefined)?.code === "ETIMEDOUT"; + const marker = readProbeMetadata(markerPath, 64); + const reentryMarker = readProbeMetadata(reentryPath, 64); + const groupText = readProbeMetadata(groupPath, 64); + const launcherStderr = readProbeMetadata(stderrPath, MAX_DIAGNOSTIC_VALUE_BYTES); + groupId = groupText === null ? 0 : Number.parseInt(groupText, 10); + if (marker === null) return shimProbeCleanup("marker", result.error, probeStatus, probeSignal); + if (reentryMarker === null) return shimProbeCleanup("reentry", result.error, probeStatus, probeSignal); + if (groupText === null) return shimProbeCleanup("group", result.error, probeStatus, probeSignal); + if (launcherStderr === null) return shimProbeCleanup("stderr", result.error, probeStatus, probeSignal); + if (!Number.isInteger(groupId) || groupId <= 0) return shimProbeCleanup("group-id", result.error, probeStatus, probeSignal); + const groupSurvived = unixProcessGroupAlive(groupId); + if (timedOut || marker || reentryMarker || groupSurvived) { + try { + terminateUnixProcessGroup(groupId); + } catch (error) { + return shimProbeCleanup("termination", error, probeStatus, probeSignal); + } + } + if (result.error && !timedOut) return shimProbeCleanup("spawn", result.error, probeStatus, probeSignal); + if (timedOut || marker === "timeout") return "timeout"; + if (marker === "recursive" || reentryMarker === "recursive") return "recursive"; + if (reentryMarker !== "") return shimProbeCleanup("reentry", undefined, probeStatus, probeSignal); + if (marker === "descendants") return "descendants"; + if (groupSurvived) return "descendants"; + if (result.status === CODEX_SHIM_REENTRY_EXIT_CODE && launcherStderr.includes(CODEX_SHIM_REENTRY_DIAGNOSTIC)) { + return "recursive"; + } + if (result.status !== 0) return "failed"; + return null; + } catch (error) { + if (Number.isInteger(groupId) && groupId > 0) { + try { terminateUnixProcessGroup(groupId); } catch { /* cleanup classification below */ } + } + return shimProbeCleanup("exception", error, probeStatus, probeSignal); + } finally { + try { rmSync(probeDir, { recursive: true, force: true }); } catch { /* best-effort cleanup */ } + } +} + +function probeUnixShimFiles(files: readonly ShimFileState[]): UnixShimProbeResult { + if (process.platform === "win32") return null; + codexShimProbeHookForTests?.(); + return files + .filter(file => !file.preserveOnly) + .map(file => probeUnixShimInstall(file.wrapperPath)) + .find(result => result !== null) ?? null; +} + +function unixProcessGroupAlive(groupId: number): boolean { + try { + process.kill(-groupId, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code !== "ESRCH"; + } +} + +function terminateUnixProcessGroup(groupId: number): void { + let permissionError: unknown; + try { + process.kill(-groupId, "SIGKILL"); + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (code === "EPERM") permissionError = error; + else if (code !== "ESRCH") throw error; + } + // A concurrently exiting group can briefly reject a second signal. Only + // observed disappearance clears that uncertainty; never send another signal. + const deadline = Date.now() + CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS; + while (Date.now() < deadline && unixProcessGroupAlive(groupId)) Bun.sleepSync(10); + if (unixProcessGroupAlive(groupId)) { + if (permissionError) throw permissionError; + throw new Error(`Codex shim install probe process group ${groupId} did not terminate`); + } +} + +export { CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS, MAX_DIAGNOSTIC_VALUE_BYTES }; +export type { UnixShimProbeResult }; +export { probeUnixShimFiles }; diff --git a/src/codex/shim-restore-lock.ts b/src/codex/shim-restore-lock.ts new file mode 100644 index 0000000000..3ee2e843ac --- /dev/null +++ b/src/codex/shim-restore-lock.ts @@ -0,0 +1,169 @@ +import { randomUUID } from "node:crypto"; +import { + closeSync, + existsSync, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readdirSync, + rmdirSync, + unlinkSync, + writeFileSync, + type Stats, +} from "node:fs"; +import { join } from "node:path"; +import { getConfigDir } from "../config"; +import { isProcessAlive } from "../lib/process-control"; +import { sameFingerprint, stableShimPathProbe, type ShimPathFingerprint } from "./shim-fingerprint"; +import { fileErrorCode } from "./shim-state-file"; + +const CODEX_SHIM_RESTORE_LOCK_STALE_MS = 30_000; + +interface ShimRestoreLock { + release(): void; +} + +interface ShimRestoreLockRecord { + version: 1; + token: string; + pid: number; + createdAt: number; +} + +interface ShimRestoreLockSnapshot { + record: ShimRestoreLockRecord; + ownerPath: string; + lockIdentity: Pick; + fingerprint: ShimPathFingerprint; +} + +function restoreLockPath(): string { + return join(getConfigDir(), "codex-shim.autorestore.lock"); +} + +function sameFileIdentity(left: Pick, right: Pick): boolean { + return left.dev === right.dev && left.ino === right.ino; +} + +function readShimRestoreLockSnapshot(path: string): ShimRestoreLockSnapshot | null { + let lockIdentity: Stats; + let entries: string[]; + try { + lockIdentity = lstatSync(path); + if (!lockIdentity.isDirectory()) return null; + entries = readdirSync(path); + } catch { + return null; + } + if (entries.length !== 1 || !entries[0].endsWith(".json")) return null; + const ownerPath = join(path, entries[0]); + const probe = stableShimPathProbe(ownerPath); + if (!probe || probe.fingerprint.kind !== "file" || probe.fingerprint.size > 4096) return null; + try { + const value = JSON.parse(probe.prefix) as Partial; + if (value.version !== 1 || typeof value.token !== "string" || value.token.length === 0 + || typeof value.pid !== "number" || !Number.isSafeInteger(value.pid) || value.pid <= 0 + || typeof value.createdAt !== "number" || !Number.isFinite(value.createdAt)) return null; + if (entries[0] !== `${value.token}.json`) return null; + const currentLockIdentity = lstatSync(path); + if (!currentLockIdentity.isDirectory() || !sameFileIdentity(lockIdentity, currentLockIdentity)) return null; + return { + record: value as ShimRestoreLockRecord, + ownerPath, + lockIdentity, + fingerprint: probe.fingerprint, + }; + } catch { + return null; + } +} + +function sameShimRestoreLock(left: ShimRestoreLockSnapshot, right: ShimRestoreLockSnapshot): boolean { + return left.record.token === right.record.token + && sameFileIdentity(left.lockIdentity, right.lockIdentity) + && sameFingerprint(left.fingerprint, right.fingerprint); +} + +function reclaimStaleRestoreLock(path: string, beforeDelete?: () => void): boolean { + const observed = readShimRestoreLockSnapshot(path); + if (!observed) return false; + const createdAt = Math.max(observed.record.createdAt, observed.fingerprint.mtimeMs); + if (Date.now() - createdAt <= CODEX_SHIM_RESTORE_LOCK_STALE_MS) return false; + if (isProcessAlive(observed.record.pid)) return false; + const current = readShimRestoreLockSnapshot(path); + if (!current || !sameShimRestoreLock(observed, current)) return false; + beforeDelete?.(); + try { + // The token is part of the owner filename. Even if the lock directory is + // replaced after the comparison, this unlink cannot target a successor's + // differently named owner record. + unlinkSync(observed.ownerPath); + rmdirSync(path); + return true; + } catch { + return false; + } +} + +function tryAcquireShimRestoreLock(beforeStaleDelete?: () => void): ShimRestoreLock | null { + const dir = getConfigDir(); + if (!existsSync(dir)) mkdirSync(dir, { recursive: true, mode: 0o700 }); + const path = restoreLockPath(); + for (let attempt = 0; attempt < 2; attempt += 1) { + let fd: number | null = null; + let identity: Stats | null = null; + let createdDirectory = false; + const record: ShimRestoreLockRecord = { + version: 1, + token: `${process.pid}-${Date.now()}-${randomUUID()}`, + pid: process.pid, + createdAt: Date.now(), + }; + const ownerPath = join(path, `${record.token}.json`); + try { + mkdirSync(path, { mode: 0o700 }); + createdDirectory = true; + fd = openSync(ownerPath, "wx", 0o600); + identity = fstatSync(fd); + writeFileSync(fd, `${JSON.stringify(record)}\n`, "utf8"); + identity = fstatSync(fd); + let released = false; + return { + release(): void { + if (released) return; + released = true; + try { closeSync(fd!); } catch { /* stale recovery handles an uncertain lock */ } + try { + const current = readShimRestoreLockSnapshot(path); + if (identity && current && current.record.token === record.token + && sameFileIdentity(identity, current.fingerprint)) { + unlinkSync(ownerPath); + rmdirSync(path); + } + } catch { /* stale recovery handles release failures */ } + }, + }; + } catch (error) { + if (fd !== null) { + try { closeSync(fd); } catch { /* best-effort close before ownership cleanup */ } + try { + const current = readShimRestoreLockSnapshot(path); + if (identity && current && current.record.token === record.token + && sameFileIdentity(identity, current.fingerprint)) { + unlinkSync(ownerPath); + rmdirSync(path); + } + } catch { /* leave an uncertain lock for stale recovery */ } + } else if (createdDirectory) { + try { rmdirSync(path); } catch { /* another owner exists or cleanup is uncertain */ } + } + if (fileErrorCode(error) !== "EEXIST") throw error; + if (attempt === 0 && reclaimStaleRestoreLock(path, beforeStaleDelete)) continue; + return null; + } + } + return null; +} + +export { tryAcquireShimRestoreLock, reclaimStaleRestoreLock }; diff --git a/src/codex/shim-state-file.ts b/src/codex/shim-state-file.ts new file mode 100644 index 0000000000..d5c790cdcf --- /dev/null +++ b/src/codex/shim-state-file.ts @@ -0,0 +1,151 @@ +import { + closeSync, + existsSync, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readSync, + writeFileSync, + type Stats, +} from "node:fs"; +import { join } from "node:path"; +import { getConfigDir } from "../config"; +import { recordOwnedConfigPath } from "../lib/config-ownership"; + +export const CODEX_SHIM_STATE_MAX_BYTES = 1024 * 1024; + +interface ShimState { + platform: NodeJS.Platform; + wrapperPath: string; + originalPath: string; + backupPath: string; + wrappers?: ShimFileState[]; +} + +interface ShimFileState { + wrapperPath: string; + originalPath: string; + backupPath: string; + realPath?: string; + preserveOnly?: boolean; +} + +interface ShimStateReadResult { + state: ShimState | null; + present: boolean; + warning?: string; +} + +function fileErrorCode(error: unknown): string | undefined { + return error && typeof error === "object" && "code" in error + ? String((error as { code?: unknown }).code) + : undefined; +} + +function readBoundedRegularFile(path: string, maxBytes: number): { bytes: Buffer; content: string } | { warning: string } | null { + let lexicalBefore: Stats; + try { + lexicalBefore = lstatSync(path); + if (lexicalBefore.isSymbolicLink() || !lexicalBefore.isFile()) { + return { warning: `Codex shim state is not a direct regular file at ${path}; auto-restore skipped.` }; + } + } catch (error) { + if (fileErrorCode(error) === "ENOENT") return null; + return { warning: `Codex shim state could not be inspected at ${path}.` }; + } + let fd: number; + try { + fd = openSync(path, "r"); + } catch (error) { + if (fileErrorCode(error) === "ENOENT") return null; + return { warning: `Codex shim state could not be opened as a regular file at ${path}.` }; + } + try { + const before = fstatSync(fd); + if (!before.isFile()) return { warning: `Codex shim state is not a regular file at ${path}; auto-restore skipped.` }; + if (before.size > maxBytes) { + return { warning: `Codex shim state exceeds the 1 MiB startup limit at ${path}; auto-restore skipped.` }; + } + const buffer = Buffer.allocUnsafe(before.size); + let offset = 0; + while (offset < buffer.length) { + const bytesRead = readSync(fd, buffer, offset, buffer.length - offset, offset); + if (bytesRead === 0) return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; + offset += bytesRead; + } + const extra = Buffer.allocUnsafe(1); + if (readSync(fd, extra, 0, 1, offset) !== 0) { + return { warning: `Codex shim state exceeds the 1 MiB startup limit at ${path}; auto-restore skipped.` }; + } + const after = fstatSync(fd); + let lexicalAfter: Stats; + try { + lexicalAfter = lstatSync(path); + } catch { + return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; + } + if (before.dev !== after.dev || before.ino !== after.ino || before.size !== after.size + || before.mtimeMs !== after.mtimeMs || before.ctimeMs !== after.ctimeMs + || lexicalBefore.dev !== before.dev || lexicalBefore.ino !== before.ino + || lexicalAfter.isSymbolicLink() || lexicalAfter.dev !== after.dev || lexicalAfter.ino !== after.ino) { + return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; + } + return { bytes: buffer, content: buffer.toString("utf8") }; + } finally { + closeSync(fd); + } +} + +function readStateResult(path = statePath()): ShimStateReadResult { + const bounded = readBoundedRegularFile(path, CODEX_SHIM_STATE_MAX_BYTES); + if (!bounded) return { state: null, present: false }; + if ("warning" in bounded) return { state: null, present: true, warning: bounded.warning }; + try { + const value = JSON.parse(bounded.content) as unknown; + if (!value || typeof value !== "object") return { state: null, present: true }; + const state = value as Record; + if (typeof state.platform !== "string") return { state: null, present: true }; + const validFile = (item: unknown): item is ShimFileState => { + if (!item || typeof item !== "object") return false; + const file = item as Record; + return typeof file.wrapperPath === "string" + && typeof file.originalPath === "string" + && typeof file.backupPath === "string" + && (file.realPath === undefined || typeof file.realPath === "string") + && (file.preserveOnly === undefined || typeof file.preserveOnly === "boolean"); + }; + if (state.wrappers !== undefined) { + if (!Array.isArray(state.wrappers) || state.wrappers.length === 0 || !state.wrappers.every(validFile)) return { state: null, present: true }; + } else if (!validFile(state)) { + return { state: null, present: true }; + } + return { state: state as unknown as ShimState, present: true }; + } catch { + return { state: null, present: true }; + } +} + +function readState(): ShimState | null { + return readStateResult().state; +} + +function statePath(): string { + return join(getConfigDir(), "codex-shim.json"); +} + +function writeState(state: ShimState): void { + const path = statePath(); + recordOwnedConfigPath(getConfigDir(), path); + if (!existsSync(getConfigDir())) mkdirSync(getConfigDir(), { recursive: true }); + writeFileSync(path, JSON.stringify(state, null, 2) + "\n", "utf8"); +} + +function stateFiles(state: ShimState): ShimFileState[] { + return state.wrappers?.length + ? state.wrappers + : [{ wrapperPath: state.wrapperPath, originalPath: state.originalPath, backupPath: state.backupPath }]; +} + +export type { ShimState, ShimFileState }; +export { fileErrorCode, readStateResult, readState, statePath, writeState, stateFiles }; diff --git a/src/codex/shim-templates.ts b/src/codex/shim-templates.ts new file mode 100644 index 0000000000..5ed2cb6b4a --- /dev/null +++ b/src/codex/shim-templates.ts @@ -0,0 +1,265 @@ +import { BUN_RUNTIME_PATH_ENV, BUN_RUNTIME_SOURCE_ENV } from "../lib/bun-runtime"; +import type { BunRuntimeSource } from "../lib/bun-runtime"; +import { serviceApiTokenFilePath } from "../lib/service-secrets"; +import { windowsEnvIndirectBatchValue } from "../lib/win-paths"; + +const SHIM_MARKER = "opencodex codex autostart shim"; +const UNIX_SHIM_REVISION_MARKER = "opencodex unix codex shim revision 2"; + +const CODEX_SHIM_REENTRY_EXIT_CODE = 126; +const CODEX_SHIM_REENTRY_DIAGNOSTIC = "opencodex: saved Codex launcher resolved back to the autostart shim; run ocx codex-shim uninstall and reinstall Codex before enabling codexAutoStart."; + +const CODEX_INTERNAL_COMMANDS = [ + "app-server", + "archive", + "apply", + "cloud", + "completion", + "debug", + "delete", + "doctor", + "exec-server", + "features", + "fork", + "help", + "login", + "logout", + "mcp", + "plugin", + "sandbox", + "unarchive", + "update", +]; + +// Codex accepts global options before a subcommand. The shim must skip the value belonging to +// these options before it decides which first positional token is the real subcommand. Keep this +// list aligned with `codex --help`; `--option=value` and attached short forms stay one token. +const CODEX_GLOBAL_OPTIONS_WITH_VALUE = [ + "-c", "--config", + "--enable", "--disable", + "--remote", "--remote-auth-token-env", + "-i", "--image", + "-m", "--model", + "--local-provider", + "-p", "--profile", + "-s", "--sandbox", + "-C", "--cd", + "--add-dir", + "-a", "--ask-for-approval", +]; + +function shQuote(value: string): string { + return `'${value.replace(/'/g, "'\\''")}'`; +} + +// Provenance is required rather than defaulted: a default would let a caller pass an +// override binary and silently label it something else, which is precisely the +// path/marker disagreement this feature exists to prevent. +// +// The marker is scoped to the `ensure` invocation in every flavor below and is never +// exported into the shim's own environment. A shim wraps the real `codex`, so an +// exported marker would be inherited by Codex and everything it spawns — a shell that +// then ran a *different* Bun directly would carry a provenance describing a binary it +// is not executing. + +export function buildUnixCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource, tokenFile = serviceApiTokenFilePath()): string { + const internalCommands = CODEX_INTERNAL_COMMANDS.join("|"); + const valueOptions = CODEX_GLOBAL_OPTIONS_WITH_VALUE.join("|"); + return `#!/usr/bin/env sh +# ${SHIM_MARKER} +# ${UNIX_SHIM_REVISION_MARKER} +if [ "\${OCX_SHIM_PROBE:-}" = "1" ]; then + if [ "\${OCX_SHIM_PROBE_ACTIVE:-}" = "1" ]; then + if [ -n "\${OCX_SHIM_PROBE_REENTRY_PATH:-}" ]; then + (umask 077; printf '%s\n' recursive > "$OCX_SHIM_PROBE_REENTRY_PATH") 2>/dev/null || true + fi + printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 + exit ${CODEX_SHIM_REENTRY_EXIT_CODE} + fi + OCX_SHIM_PROBE_ACTIVE=1 + export OCX_SHIM_PROBE_ACTIVE +fi +if [ "\${OCX_SHIM_ACTIVE_PID:-}" = "$$" ]; then + printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 + exit ${CODEX_SHIM_REENTRY_EXIT_CODE} +fi +case "\${OCX_SHIM_ACTIVE_DEPTH:-0}" in + 0) + OCX_SHIM_ACTIVE_DEPTH=1 + ;; + 1) + OCX_SHIM_ACTIVE_DEPTH=2 + ;; + *) + printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 + exit ${CODEX_SHIM_REENTRY_EXIT_CODE} + ;; +esac +# Dynamic launchers such as mise exec -- codex may resolve the command name +# back to this wrapper. An exec chain keeps the same PID. A legitimate nested +# Codex invocation may enter once with a new PID; repeated child-process +# redispatch reaches depth 2 and is rejected before it can form an infinite chain. +OCX_SHIM_ACTIVE_PID=$$ +export OCX_SHIM_ACTIVE_PID OCX_SHIM_ACTIVE_DEPTH +if [ -z "$OPENCODEX_API_AUTH_TOKEN" ] && [ -f ${shQuote(tokenFile)} ]; then + OPENCODEX_API_AUTH_TOKEN="$(cat ${shQuote(tokenFile)})" + export OPENCODEX_API_AUTH_TOKEN +fi +ocx_subcommand="" +ocx_skip_next=0 +for ocx_arg in "$@"; do + if [ "$ocx_skip_next" -eq 1 ]; then + ocx_skip_next=0 + continue + fi + case "$ocx_arg" in + --) + break + ;; + ${valueOptions}) + ocx_skip_next=1 + ;; + --help|-h|--version|-V) + ocx_subcommand="$ocx_arg" + break + ;; + -*) + ;; + *) + ocx_subcommand="$ocx_arg" + break + ;; + esac +done +case "$ocx_subcommand" in + ${internalCommands}|--help|-h|--version|-V) + ;; + *) + if [ -z "$OCX_SHIM_BYPASS" ]; then + ${BUN_RUNTIME_SOURCE_ENV}=${shQuote(bunRuntimeSource)} ${BUN_RUNTIME_PATH_ENV}=${shQuote(bunPath)} ${shQuote(bunPath)} ${shQuote(cliPath)} ensure >/dev/null 2>&1 || true + fi + ;; +esac +exec ${shQuote(realCodexPath)} "$@" +`; +} + +function windowsBatchValue(value: string): string { + return value + .replace(/%/g, "%%") + .replace(/\^/g, "^^") + .replace(/"/g, "") + .replace(/[\r\n]/g, ""); +} + +function windowsBatchSet(name: string, value: string): string { + // Paths are rewritten to %USERPROFILE%-style env indirection: cmd.exe parses .cmd + // files in the OEM codepage, so a literal non-ASCII profile prefix (Korean/Chinese + // usernames) written as UTF-8 turns to mojibake. The env token expands natively in + // the right codepage at parse time; no `chcp` here — this shim runs in the USER's + // console and must not leak a codepage change into it. + return `set "${name}=${windowsEnvIndirectBatchValue(value, windowsBatchValue)}"`; +} + +export function buildWindowsCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource): string { + const internalCommandChecks = CODEX_INTERNAL_COMMANDS.map(command => `if /I "%~1"=="${command}" goto run_codex`).join("\r\n"); + const valueOptionChecks = CODEX_GLOBAL_OPTIONS_WITH_VALUE.map(option => `if /I "%~1"=="${option}" goto skip_option_value`).join("\r\n"); + return `@echo off\r +rem ${SHIM_MARKER}\r +setlocal\r +${windowsBatchSet("OCX_REAL_CODEX", realCodexPath)}\r +${windowsBatchSet("OCX_BUN", bunPath)}\r +${windowsBatchSet("OCX_CLI", cliPath)}\r +${windowsBatchSet("OCX_API_TOKEN_FILE", serviceApiTokenFilePath())}\r +if "%OPENCODEX_API_AUTH_TOKEN%"=="" if exist "%OCX_API_TOKEN_FILE%" set /p OPENCODEX_API_AUTH_TOKEN=<"%OCX_API_TOKEN_FILE%"\r +if not "%OCX_SHIM_BYPASS%"=="" goto run_codex\r +goto scan_codex_args\r +:scan_codex_args\r +if "%~1"=="" goto ensure_ocx\r +if "%~1"=="--" goto ensure_ocx\r +${valueOptionChecks}\r +${internalCommandChecks}\r +if /I "%~1"=="--help" goto run_codex\r +if /I "%~1"=="-h" goto run_codex\r +if /I "%~1"=="--version" goto run_codex\r +if /I "%~1"=="-V" goto run_codex\r +set "OCX_SCAN_ARG=%~1"\r +if "%OCX_SCAN_ARG:~0,1%"=="-" goto shift_codex_arg\r +goto ensure_ocx\r +:skip_option_value\r +shift\r +if "%~1"=="" goto ensure_ocx\r +:shift_codex_arg\r +shift\r +goto scan_codex_args\r +:ensure_ocx\r +setlocal\r +${windowsBatchSet(BUN_RUNTIME_SOURCE_ENV, bunRuntimeSource)}\r +${windowsBatchSet(BUN_RUNTIME_PATH_ENV, bunPath)}\r +"%OCX_BUN%" "%OCX_CLI%" ensure >nul 2>nul\r +endlocal\r +:run_codex\r +"%OCX_REAL_CODEX%" %*\r +`; +} + +function psString(value: string): string { + return `'${value.replace(/'/g, "''")}'`; +} + +export function buildWindowsPowerShellCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource): string { + const internalCommands = CODEX_INTERNAL_COMMANDS.map(command => psString(command)).join(", "); + const valueOptions = CODEX_GLOBAL_OPTIONS_WITH_VALUE.map(option => psString(option)).join(", "); + const tokenFile = serviceApiTokenFilePath(); + return `#!/usr/bin/env pwsh +# ${SHIM_MARKER} +$hadApiAuthToken = Test-Path Env:\\OPENCODEX_API_AUTH_TOKEN +$priorApiAuthToken = $env:OPENCODEX_API_AUTH_TOKEN +try { +if (-not $env:OPENCODEX_API_AUTH_TOKEN -and (Test-Path -LiteralPath ${psString(tokenFile)})) { + $env:OPENCODEX_API_AUTH_TOKEN = (Get-Content -Raw -LiteralPath ${psString(tokenFile)}).Trim() +} +$internalCommands = @(${internalCommands}) +$valueOptions = @(${valueOptions}) +$subcommand = "" +$skipNext = $false +foreach ($argValue in $args) { + $argText = [string]$argValue + if ($skipNext) { $skipNext = $false; continue } + if ($argText -eq "--") { break } + if ($valueOptions -contains $argText) { $skipNext = $true; continue } + if (@("--help", "-h", "--version", "-V") -contains $argText) { $subcommand = $argText; break } + if ($argText.StartsWith("-")) { continue } + $subcommand = $argText + break +} +$skipEnsure = $env:OCX_SHIM_BYPASS -or $internalCommands -contains $subcommand -or @("--help", "-h", "--version", "-V") -contains $subcommand +if (-not $skipEnsure) { + $priorRuntimeSource = $env:${BUN_RUNTIME_SOURCE_ENV} + $priorRuntimePath = $env:${BUN_RUNTIME_PATH_ENV} + $env:${BUN_RUNTIME_SOURCE_ENV} = ${psString(bunRuntimeSource)} + $env:${BUN_RUNTIME_PATH_ENV} = ${psString(bunPath)} + try { & ${psString(bunPath)} ${psString(cliPath)} ensure *> $null } + finally { + if ($null -eq $priorRuntimeSource) { Remove-Item Env:\\${BUN_RUNTIME_SOURCE_ENV} -ErrorAction SilentlyContinue } + else { $env:${BUN_RUNTIME_SOURCE_ENV} = $priorRuntimeSource } + if ($null -eq $priorRuntimePath) { Remove-Item Env:\\${BUN_RUNTIME_PATH_ENV} -ErrorAction SilentlyContinue } + else { $env:${BUN_RUNTIME_PATH_ENV} = $priorRuntimePath } + } +} +& ${psString(realCodexPath)} @args +$codexExitCode = $LASTEXITCODE +} finally { + if ($hadApiAuthToken) { $env:OPENCODEX_API_AUTH_TOKEN = $priorApiAuthToken } + else { Remove-Item Env:\\OPENCODEX_API_AUTH_TOKEN -ErrorAction SilentlyContinue } +} +exit $codexExitCode +`; +} + +/** Git-Bash accepts `C:/...` but not backslashed paths inside sh scripts. */ +function gitBashPath(path: string): string { + return path.replace(/\\/g, "/"); +} + +export { SHIM_MARKER, UNIX_SHIM_REVISION_MARKER, CODEX_SHIM_REENTRY_EXIT_CODE, CODEX_SHIM_REENTRY_DIAGNOSTIC, shQuote, windowsBatchSet, psString, gitBashPath }; diff --git a/src/codex/shim.ts b/src/codex/shim.ts index 57f46cfb0d..6eebf8d61f 100644 --- a/src/codex/shim.ts +++ b/src/codex/shim.ts @@ -1,306 +1,73 @@ import { randomUUID } from "node:crypto"; -import { spawnSync } from "node:child_process"; -import { tmpdir } from "node:os"; -import { basename, delimiter, dirname, extname, join, posix, win32 } from "node:path"; import { chmodSync, - closeSync, existsSync, - fstatSync, lstatSync, - linkSync, - mkdirSync, - mkdtempSync, - openSync, readFileSync, - readdirSync, - readlinkSync, - readSync, renameSync, - rmSync, - rmdirSync, - statSync, - symlinkSync, - type Stats, unlinkSync, writeFileSync, } from "node:fs"; -import { getConfigDir } from "../config"; -import { BUN_RUNTIME_PATH_ENV, BUN_RUNTIME_SOURCE_ENV, durableBunRuntime } from "../lib/bun-runtime"; +import { basename, delimiter, dirname, extname, join, posix } from "node:path"; +import { durableBunRuntime } from "../lib/bun-runtime"; import type { BunRuntimeSource } from "../lib/bun-runtime"; -import { isProcessAlive } from "../lib/process-control"; import { serviceApiTokenFilePath } from "../lib/service-secrets"; -import { recordOwnedConfigPath } from "../lib/config-ownership"; -import { windowsEnvIndirectBatchValue } from "../lib/win-paths"; import { isWslRuntime, wslAutomountRoot } from "./home"; import { truncateRetainedUtf8 } from "../lib/admission"; +import { + buildUnixCodexShim, + buildWindowsCodexShim, + buildWindowsPowerShellCodexShim, + gitBashPath, + SHIM_MARKER, + UNIX_SHIM_REVISION_MARKER, +} from "./shim-templates"; +import { + hasUsableBackingPath, + isCurrentUnixShimProbe, + isHealthyShimProbe, + isVersionManagerOwnedCodexPath, + restoreWithoutReplacing, + sameFingerprint, + sameFingerprintAfterRename, + sameStableShimPathProbe, + shimPathFingerprint, + stableShimPathProbe, + type ShimPathFingerprint, + type StableShimPathProbe, +} from "./shim-fingerprint"; +import { + fileErrorCode, + readState, + readStateResult, + stateFiles, + statePath, + writeState, + type ShimFileState, + type ShimState, +} from "./shim-state-file"; +import { + CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS, + MAX_DIAGNOSTIC_VALUE_BYTES, + probeUnixShimFiles, + type UnixShimProbeResult, +} from "./shim-probe"; +import { tryAcquireShimRestoreLock } from "./shim-restore-lock"; + +export { buildUnixCodexShim, buildWindowsCodexShim, buildWindowsPowerShellCodexShim } from "./shim-templates"; +export { isVersionManagerOwnedCodexPath } from "./shim-fingerprint"; +export { CODEX_SHIM_STATE_MAX_BYTES } from "./shim-state-file"; +export { setCodexShimProbeHookForTests, setCodexShimProbeShellForTests, setCodexShimProbeObservationMsForTests } from "./shim-probe"; +export type { CodexShimBackingForCommand } from "./shim-inspect"; +export { isLocalAbsoluteInspectionPath, inspectCodexShimBackingForCommand } from "./shim-inspect"; -const SHIM_MARKER = "opencodex codex autostart shim"; -const UNIX_SHIM_REVISION_MARKER = "opencodex unix codex shim revision 2"; -const CODEX_SHIM_PROBE_BYTES = 16 * 1024; export const CODEX_SHIM_REPLACEMENT_STABLE_MS = 100; -export const CODEX_SHIM_STATE_MAX_BYTES = 1024 * 1024; -const CODEX_SHIM_RESTORE_LOCK_STALE_MS = 30_000; -const CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS = 5_000; -const CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS = 1_000; -const CODEX_SHIM_REENTRY_EXIT_CODE = 126; -const CODEX_SHIM_REENTRY_DIAGNOSTIC = "opencodex: saved Codex launcher resolved back to the autostart shim; run ocx codex-shim uninstall and reinstall Codex before enabling codexAutoStart."; -const CODEX_SHIM_INSTALL_PROBE_SCRIPT = ` -const { spawn } = require("node:child_process"); -const { readFileSync, writeFileSync } = require("node:fs"); -const [markerPath, reentryPath, groupPath, stderrPath, launcherShellPath, wrapperPath, timeoutRaw, stderrLimitRaw, stderrDrainRaw, observationRaw] = process.argv.slice(1); -const timeoutMs = Number.parseInt(timeoutRaw, 10); -const stderrLimit = Number.parseInt(stderrLimitRaw, 10); -const stderrDrainMs = Number.parseInt(stderrDrainRaw, 10); -const observationMs = Number.parseInt(observationRaw, 10); -const probeStartedAt = Date.now(); -const stderrChunks = []; -let stderrBytes = 0; -let launcher; -let probeLease; -let timer; -let stderrDrainTimer; -let observationTimer; -let reentryPollTimer; -let marker = ""; -let finished = false; - -function writeExclusive(path, value) { - writeFileSync(path, value, { flag: "wx", mode: 0o600 }); -} - -function appendStderr(value) { - if (stderrBytes >= stderrLimit) return; - const bytes = Buffer.from(value); - const retained = bytes.subarray(0, stderrLimit - stderrBytes); - stderrChunks.push(retained); - stderrBytes += retained.byteLength; -} - -function groupAlive() { - if (!launcher || !launcher.pid) return false; - try { - process.kill(-launcher.pid, 0); - return true; - } catch (error) { - return error && error.code !== "ESRCH"; - } -} - -function killGroup() { - if (!launcher || !launcher.pid) return; - try { process.kill(-launcher.pid, "SIGKILL"); } catch (error) { - if (!error || error.code !== "ESRCH") appendStderr(String(error)); - } -} - -function setMarker(value) { - if (marker) return; - marker = value; - try { writeExclusive(markerPath, value + "\\n"); } catch (error) { appendStderr(String(error)); } -} -function reentryDetected() { - try { return readFileSync(reentryPath, "utf8").trim() === "recursive"; } catch { return false; } -} - -function checkReentry() { - if (finished || !reentryDetected()) return; - setMarker("recursive"); - killGroup(); - finish(126); -} - -function finish(status) { - if (finished) return; - finished = true; - if (timer) clearTimeout(timer); - if (stderrDrainTimer) clearTimeout(stderrDrainTimer); - if (observationTimer) clearTimeout(observationTimer); - if (reentryPollTimer) clearInterval(reentryPollTimer); - if (!marker && reentryDetected()) setMarker("recursive"); - if (!marker && groupAlive()) { - setMarker("descendants"); - killGroup(); - } - try { writeExclusive(stderrPath, Buffer.concat(stderrChunks)); } catch { /* parent fails closed */ } - process.exit(marker === "timeout" ? 124 : marker === "descendants" ? 125 : marker === "recursive" ? 126 : status); -} - -function finishAfterStderr(status) { - if (finished) return; - if (timer) { - clearTimeout(timer); - timer = undefined; - } - if (!launcher || !launcher.stderr || !probeLease) { - finish(status); - return; - } - let stderrEnded = launcher.stderr.readableEnded; - let leaseEnded = probeLease.readableEnded; - let observationElapsed = false; - const finishWhenReady = () => { - if (stderrEnded && leaseEnded && observationElapsed) finish(status); - }; - launcher.stderr.once("end", () => { - stderrEnded = true; - finishWhenReady(); - }); - probeLease.once("end", () => { - leaseEnded = true; - finishWhenReady(); - }); - stderrDrainTimer = setTimeout(() => { - stderrEnded = true; - if (!marker && groupAlive()) { - setMarker("descendants"); - killGroup(); - finish(125); - return; - } - finishWhenReady(); - }, stderrDrainMs); - const remainingObservationMs = Math.max(0, observationMs - (Date.now() - probeStartedAt)); - observationTimer = setTimeout(() => { - observationElapsed = true; - if (!leaseEnded) { - setMarker(groupAlive() ? "descendants" : "timeout"); - killGroup(); - finish(marker === "descendants" ? 125 : 124); - return; - } - finishWhenReady(); - }, remainingObservationMs); - finishWhenReady(); -} - -try { - launcher = spawn(launcherShellPath, [wrapperPath, "--version"], { - detached: true, - env: process.env, - stdio: ["ignore", "ignore", "pipe", "pipe"], - }); - if (!launcher.pid) throw new Error("Codex shim probe launcher has no pid"); - probeLease = launcher.stdio[3]; - if (!probeLease) throw new Error("Codex shim probe launcher has no descendant lease pipe"); - writeExclusive(groupPath, String(launcher.pid) + "\\n"); - launcher.stderr.on("data", appendStderr); - reentryPollTimer = setInterval(checkReentry, 10); - launcher.once("error", error => { - appendStderr(String(error)); - finishAfterStderr(127); - }); - launcher.once("exit", code => finishAfterStderr(Number.isInteger(code) ? code : 127)); - timer = setTimeout(() => { - setMarker("timeout"); - killGroup(); - finish(124); - }, timeoutMs); -} catch (error) { - appendStderr(String(error)); - killGroup(); - finish(127); -} -`; -const MAX_DIAGNOSTIC_VALUE_BYTES = 8 * 1024; let lastShimDiscoveryError: string | null = null; /** Last human-readable reason discovery returned null (exposed for doctor/tests). */ export function lastCodexDiscoveryError(): string | null { return lastShimDiscoveryError; } -const CODEX_INTERNAL_COMMANDS = [ - "app-server", - "archive", - "apply", - "cloud", - "completion", - "debug", - "delete", - "doctor", - "exec-server", - "features", - "fork", - "help", - "login", - "logout", - "mcp", - "plugin", - "sandbox", - "unarchive", - "update", -]; - -// Codex accepts global options before a subcommand. The shim must skip the value belonging to -// these options before it decides which first positional token is the real subcommand. Keep this -// list aligned with `codex --help`; `--option=value` and attached short forms stay one token. -const CODEX_GLOBAL_OPTIONS_WITH_VALUE = [ - "-c", "--config", - "--enable", "--disable", - "--remote", "--remote-auth-token-env", - "-i", "--image", - "-m", "--model", - "--local-provider", - "-p", "--profile", - "-s", "--sandbox", - "-C", "--cd", - "--add-dir", - "-a", "--ask-for-approval", -]; - -interface ShimState { - platform: NodeJS.Platform; - wrapperPath: string; - originalPath: string; - backupPath: string; - wrappers?: ShimFileState[]; -} - -interface ShimFileState { - wrapperPath: string; - originalPath: string; - backupPath: string; - realPath?: string; - preserveOnly?: boolean; -} - -export type CodexShimBackingForCommand = - | Readonly<{ status: "not-tracked" }> - | Readonly<{ - status: "matched"; - selectedRole: "wrapper" | "backing"; - backingPath: string; - backingKind: "backup" | "real"; - }> - | Readonly<{ - status: "unknown"; - reason: - | "state_invalid" - | "platform_mismatch" - | "ambiguous_match" - | "preserve_only" - | "backing_missing" - | "backing_mismatch" - | "binding_unavailable" - | "wrapper_unhealthy" - | "version_manager_refused"; - }>; - -interface ShimPathFingerprint { - dev: number; - ino: number; - kind: "file" | "symlink"; - mode: number; - size: number; - mtimeMs: number; - ctimeMs: number; - target?: Omit; -} - -interface StableShimPathProbe { - fingerprint: ShimPathFingerprint; - prefix: string; -} interface InstallCodexShimInternalOptions { expectedReplacements?: ReadonlyMap; @@ -348,166 +115,6 @@ function isHealthyShim(path: string, platform: NodeJS.Platform): boolean { } } -function readShimProbePrefix(path: string): string { - const fd = openSync(path, "r"); - try { - const buffer = Buffer.allocUnsafe(CODEX_SHIM_PROBE_BYTES); - const bytesRead = readSync(fd, buffer, 0, buffer.length, 0); - return buffer.toString("utf8", 0, bytesRead); - } finally { - closeSync(fd); - } -} - -function statFingerprint(path: string, follow: boolean): Omit | null { - try { - const stat = follow ? statSync(path) : lstatSync(path); - if (follow ? !stat.isFile() : (!stat.isFile() && !stat.isSymbolicLink())) return null; - return { - dev: stat.dev, - ino: stat.ino, - kind: stat.isSymbolicLink() ? "symlink" : "file", - mode: stat.mode, - size: stat.size, - mtimeMs: stat.mtimeMs, - ctimeMs: stat.ctimeMs, - }; - } catch { - return null; - } -} - -function sameFingerprint( - left: ShimPathFingerprint | Omit, - right: ShimPathFingerprint | Omit, -): boolean { - return left.dev === right.dev - && left.ino === right.ino - && left.kind === right.kind - && left.mode === right.mode - && left.size === right.size - && left.mtimeMs === right.mtimeMs - && left.ctimeMs === right.ctimeMs - && (!("target" in left) || !("target" in right) - ? true - : left.target === undefined && right.target === undefined - ? true - : left.target !== undefined && right.target !== undefined - ? sameFingerprint(left.target, right.target) - : false); -} - -function sameFingerprintAfterRename(left: ShimPathFingerprint, right: ShimPathFingerprint): boolean { - // rename changes the outer directory entry ctime on macOS; every other field, - // including a symlink target fingerprint, must remain identical. - return sameFingerprint({ ...left, ctimeMs: 0 }, { ...right, ctimeMs: 0 }); -} - -function stableShimPathProbe(path: string): StableShimPathProbe | null { - const before = statFingerprint(path, false); - if (!before) return null; - const targetBefore = before.kind === "symlink" ? statFingerprint(path, true) : undefined; - if (before.kind === "symlink" && !targetBefore) return null; - let prefix: string; - try { - prefix = readShimProbePrefix(path); - } catch { - return null; - } - const targetAfter = before.kind === "symlink" ? statFingerprint(path, true) : undefined; - const after = statFingerprint(path, false); - if (!after || !sameFingerprint(before, after)) return null; - if (before.kind === "symlink") { - if (!targetBefore || !targetAfter || !sameFingerprint(targetBefore, targetAfter)) return null; - } - const fingerprint: ShimPathFingerprint = { - ...before, - ...(targetBefore ? { target: targetBefore } : {}), - }; - const contentSize = fingerprint.target?.size ?? fingerprint.size; - return contentSize > 0 ? { fingerprint, prefix } : null; -} - -function sameStableShimPathProbe(left: StableShimPathProbe, right: StableShimPathProbe): boolean { - return left.prefix === right.prefix && sameFingerprint(left.fingerprint, right.fingerprint); -} - -/** - * Identity of whatever sits at `path`, read from metadata alone. - * - * `stableShimPathProbe` answers a different question: it reads content to decide - * whether a launcher looks like a healthy shim, and it deliberately returns null - * for a zero-byte file. That makes it the wrong instrument for rollback - * bookkeeping. A user can legitimately own an empty `codex` launcher, and a fresh - * install moves it aside before writing our wrapper; if the move is recorded - * without a fingerprint, rollback cannot prove the backup is still the file it - * set aside and refuses to restore it — the launcher stays lost (#1625). - * - * Content is irrelevant to that proof, so this reads dev/ino/mode/size/times and - * re-reads them to reject a path that changed under us, following a symlink to - * fingerprint its target as well. - */ -function shimPathFingerprint(path: string): ShimPathFingerprint | null { - const before = statFingerprint(path, false); - if (!before) return null; - if (before.kind !== "symlink") { - const after = statFingerprint(path, false); - return after && sameFingerprint(before, after) ? before : null; - } - const targetBefore = statFingerprint(path, true); - if (!targetBefore) return null; - const targetAfter = statFingerprint(path, true); - const after = statFingerprint(path, false); - if (!targetAfter || !after - || !sameFingerprint(targetBefore, targetAfter) - || !sameFingerprint(before, after)) return null; - return { ...before, target: targetBefore }; -} - -/** - * Move `from` onto `to` without ever replacing an existing entry. - * - * `renameSync` silently clobbers the destination on POSIX, which is wrong for a - * rollback restore: `sourceOccupied` is sampled before the fingerprint check, so - * a concurrent installer can publish its own launcher at the original path in - * between, and the restore would delete it. `link` fails EEXIST instead, which - * is the no-replace primitive we need and needs no native helper. - * - * `link` follows a symlink to its target rather than preserving the link, so a - * symlink launcher is republished with `symlink`, which is also no-replace: it - * fails EEXIST on an occupied destination. Checking existence and then renaming - * would reintroduce exactly the race this function exists to close. - */ -function restoreWithoutReplacing(from: string, to: string): void { - const source = lstatSync(from); - if (source.isSymbolicLink()) { - symlinkSync(readlinkSync(from), to); - unlinkSync(from); - return; - } - linkSync(from, to); - unlinkSync(from); -} - -function isHealthyShimProbe(probe: StableShimPathProbe, platform: NodeJS.Platform): boolean { - if (probe.prefix.length < 180 || !probe.prefix.includes(SHIM_MARKER) || !probe.prefix.includes("ensure")) return false; - const mode = probe.fingerprint.target?.mode ?? probe.fingerprint.mode; - return platform === "win32" || (mode & 0o111) !== 0; -} - -function isCurrentUnixShimProbe(probe: StableShimPathProbe): boolean { - return probe.prefix.includes(UNIX_SHIM_REVISION_MARKER); -} - -function hasUsableBackingPath(file: ShimFileState): boolean { - return [existsSync(file.backupPath) ? file.backupPath : undefined, file.realPath] - .some(path => { - if (!path) return false; - const fingerprint = statFingerprint(path, true); - return fingerprint !== null && fingerprint.size > 0; - }); -} - /** * A PATH entry that reaches Windows through WSL drive interop * (`//...`; root defaults to /mnt, configurable via @@ -625,33 +232,6 @@ function backupPathFor(path: string): string { return ext ? `${path.slice(0, -ext.length)}.opencodex-real${ext}` : `${path}.opencodex-real`; } -/** - * True when a Codex binary lives inside a version manager's install tree. - * - * These trees are rewritten in place on upgrade, which destroys both the shim - * and the sibling .opencodex-real backup it restores from (#2412). The tempting - * repair — adopt the newly installed binary as a fresh original — is wrong - * twice: it records a provenance that never happened, and the next upgrade wipes - * it again, so the repair silently un-repairs on the version manager's schedule. - * - * Scope is the three managers named in the report. nvm/fnm/npm-prefix are - * deliberately excluded: a false positive here refuses a restore that would - * otherwise be correct. - */ -export function isVersionManagerOwnedCodexPath( - path: string, - platform: NodeJS.Platform = process.platform, -): boolean { - const normalized = (platform === "win32" - ? win32.normalize(path).replace(/\\/g, "/") - : posix.normalize(path)).toLowerCase(); - return normalized.includes("/mise/installs/") - || normalized.includes("/mise/shims/") - || normalized.includes("/.asdf/installs/") - || normalized.includes("/.asdf/shims/") - || normalized.includes("/.volta/"); -} - /** * Why auto-restore refused, in the operator's own terms. Auto-restore used to * return a bare `{ status: "ineligible" }`, and the CLI warns only when a @@ -671,159 +251,9 @@ function destroyedShimMessage(file: ShimFileState): string { return `${base} This Codex binary is owned by a version manager (mise/asdf/volta), so opencodex will not wrap it as a new original — the next upgrade would overwrite the shim and its backup again. Route through Codex instead with 'ocx start', and use 'ocx service install' for autostart.`; } -function shQuote(value: string): string { - return `'${value.replace(/'/g, "'\\''")}'`; -} - -// Provenance is required rather than defaulted: a default would let a caller pass an -// override binary and silently label it something else, which is precisely the -// path/marker disagreement this feature exists to prevent. -// -// The marker is scoped to the `ensure` invocation in every flavor below and is never -// exported into the shim's own environment. A shim wraps the real `codex`, so an -// exported marker would be inherited by Codex and everything it spawns — a shell that -// then ran a *different* Bun directly would carry a provenance describing a binary it -// is not executing. -export function buildUnixCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource, tokenFile = serviceApiTokenFilePath()): string { - const internalCommands = CODEX_INTERNAL_COMMANDS.join("|"); - const valueOptions = CODEX_GLOBAL_OPTIONS_WITH_VALUE.join("|"); - return `#!/usr/bin/env sh -# ${SHIM_MARKER} -# ${UNIX_SHIM_REVISION_MARKER} -if [ "\${OCX_SHIM_PROBE:-}" = "1" ]; then - if [ "\${OCX_SHIM_PROBE_ACTIVE:-}" = "1" ]; then - if [ -n "\${OCX_SHIM_PROBE_REENTRY_PATH:-}" ]; then - (umask 077; printf '%s\n' recursive > "$OCX_SHIM_PROBE_REENTRY_PATH") 2>/dev/null || true - fi - printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 - exit ${CODEX_SHIM_REENTRY_EXIT_CODE} - fi - OCX_SHIM_PROBE_ACTIVE=1 - export OCX_SHIM_PROBE_ACTIVE -fi -if [ "\${OCX_SHIM_ACTIVE_PID:-}" = "$$" ]; then - printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 - exit ${CODEX_SHIM_REENTRY_EXIT_CODE} -fi -case "\${OCX_SHIM_ACTIVE_DEPTH:-0}" in - 0) - OCX_SHIM_ACTIVE_DEPTH=1 - ;; - 1) - OCX_SHIM_ACTIVE_DEPTH=2 - ;; - *) - printf '%s\n' ${shQuote(CODEX_SHIM_REENTRY_DIAGNOSTIC)} >&2 - exit ${CODEX_SHIM_REENTRY_EXIT_CODE} - ;; -esac -# Dynamic launchers such as mise exec -- codex may resolve the command name -# back to this wrapper. An exec chain keeps the same PID. A legitimate nested -# Codex invocation may enter once with a new PID; repeated child-process -# redispatch reaches depth 2 and is rejected before it can form an infinite chain. -OCX_SHIM_ACTIVE_PID=$$ -export OCX_SHIM_ACTIVE_PID OCX_SHIM_ACTIVE_DEPTH -if [ -z "$OPENCODEX_API_AUTH_TOKEN" ] && [ -f ${shQuote(tokenFile)} ]; then - OPENCODEX_API_AUTH_TOKEN="$(cat ${shQuote(tokenFile)})" - export OPENCODEX_API_AUTH_TOKEN -fi -ocx_subcommand="" -ocx_skip_next=0 -for ocx_arg in "$@"; do - if [ "$ocx_skip_next" -eq 1 ]; then - ocx_skip_next=0 - continue - fi - case "$ocx_arg" in - --) - break - ;; - ${valueOptions}) - ocx_skip_next=1 - ;; - --help|-h|--version|-V) - ocx_subcommand="$ocx_arg" - break - ;; - -*) - ;; - *) - ocx_subcommand="$ocx_arg" - break - ;; - esac -done -case "$ocx_subcommand" in - ${internalCommands}|--help|-h|--version|-V) - ;; - *) - if [ -z "$OCX_SHIM_BYPASS" ]; then - ${BUN_RUNTIME_SOURCE_ENV}=${shQuote(bunRuntimeSource)} ${BUN_RUNTIME_PATH_ENV}=${shQuote(bunPath)} ${shQuote(bunPath)} ${shQuote(cliPath)} ensure >/dev/null 2>&1 || true - fi - ;; -esac -exec ${shQuote(realCodexPath)} "$@" -`; -} - -type UnixShimProbeCleanupPhase = "marker" | "reentry" | "group" | "stderr" | "group-id" | "termination" | "spawn" | "exception"; -interface UnixShimProbeCleanup { - kind: "cleanup"; - phase: UnixShimProbeCleanupPhase; - code: string; - status: number | null; - signal: string; -} -type UnixShimProbeResult = UnixShimProbeCleanup | "descendants" | "failed" | "recursive" | "timeout" | null; - -const SHIM_PROBE_ERROR_CODES = new Set([ - "EACCES", "EAGAIN", "EBADF", "ECANCELED", "EINTR", "EIO", "EMFILE", "ENFILE", - "ENOENT", "ENOEXEC", "ENOMEM", "ENOSPC", "EPERM", "EPIPE", "ESRCH", "ETIMEDOUT", "ETXTBSY", -]); -const SHIM_PROBE_SIGNALS = new Set([ - "SIGABRT", "SIGBUS", "SIGHUP", "SIGILL", "SIGINT", "SIGKILL", "SIGPIPE", "SIGQUIT", - "SIGSEGV", "SIGTERM", "SIGTRAP", "SIGXCPU", "SIGXFSZ", -]); - -/** Diagnostics cross a CLI boundary: never stringify arbitrary errors or metadata. */ -function shimProbeCleanup( - phase: UnixShimProbeCleanupPhase, error?: unknown, status?: unknown, signal?: unknown, -): UnixShimProbeCleanup { - let code = error === undefined ? "none" : "unknown"; - if (error !== null && typeof error === "object") { - try { - const value = Object.getOwnPropertyDescriptor(error, "code")?.value; - if (typeof value === "string" && SHIM_PROBE_ERROR_CODES.has(value)) code = value; - } catch { /* hostile accessors/proxies cannot turn diagnostics into an exception */ } - } - return { - kind: "cleanup", phase, code, - status: typeof status === "number" && Number.isInteger(status) && status >= 0 && status <= 255 ? status : null, - signal: typeof signal === "string" && SHIM_PROBE_SIGNALS.has(signal) ? signal : "none", - }; -} - -let codexShimProbeHookForTests: (() => void) | null = null; -let codexShimProbeShellForTests: string | null = null; let codexShimGuardedWriteHookForTests: (() => void) | null = null; let codexShimFreshWriteHookForTests: (() => void) | null = null; let codexShimRollbackRestoreHookForTests: ((target: ShimFileState) => void) | null = null; -let codexShimProbeObservationMs = CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS; - -/** Narrow deterministic seam for transaction rollback tests. */ -export function setCodexShimProbeHookForTests(hook: (() => void) | null): void { - codexShimProbeHookForTests = hook; -} - -/** Selects a POSIX shell only for cross-shell probe regression tests. */ -export function setCodexShimProbeShellForTests(path: string | null): void { - codexShimProbeShellForTests = path; -} - -/** Shortens the successful-launcher observation window only for focused tests. */ -export function setCodexShimProbeObservationMsForTests(value: number | null): void { - codexShimProbeObservationMs = value ?? CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS; -} /** Narrow deterministic seam for guarded partial-write rollback tests. */ export function setCodexShimGuardedWriteHookForTests(hook: (() => void) | null): void { @@ -849,136 +279,6 @@ export function setCodexShimRollbackRestoreHookForTests( codexShimRollbackRestoreHookForTests = hook; } -function readProbeMetadata(path: string, maxBytes: number): string | null { - try { - if (!existsSync(path)) return ""; - const stat = lstatSync(path); - if (!stat.isFile() || stat.size > maxBytes) return null; - return readFileSync(path, "utf8").trim(); - } catch { - return null; - } -} - -function probeUnixShimInstall(wrapperPath: string): UnixShimProbeResult { - if (process.platform === "win32") return null; - const probeDir = mkdtempSync(join(tmpdir(), "opencodex-shim-probe-")); - const markerPath = join(probeDir, "result"); - const reentryPath = join(probeDir, "reentry"); - const groupPath = join(probeDir, "group"); - const stderrPath = join(probeDir, "stderr"); - const env: NodeJS.ProcessEnv = { - ...process.env, - OCX_SHIM_BYPASS: "1", - OCX_SHIM_PROBE: "1", - OCX_SHIM_PROBE_REENTRY_PATH: reentryPath, - }; - delete env.OCX_SHIM_ACTIVE_PID; - delete env.OCX_SHIM_ACTIVE_DEPTH; - delete env.OCX_SHIM_PROBE_ACTIVE; - let groupId = 0; - let probeStatus: unknown; - let probeSignal: unknown; - try { - chmodSync(probeDir, 0o700); - const result = spawnSync(process.execPath, [ - "-e", - CODEX_SHIM_INSTALL_PROBE_SCRIPT, - markerPath, - reentryPath, - groupPath, - stderrPath, - codexShimProbeShellForTests ?? "/bin/sh", - wrapperPath, - String(CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS), - String(MAX_DIAGNOSTIC_VALUE_BYTES), - String(CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS), - String(codexShimProbeObservationMs), - ], { - encoding: "utf8", - env, - timeout: CODEX_SHIM_INSTALL_PROBE_TIMEOUT_MS + CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS, - killSignal: "SIGKILL", - }); - probeStatus = result.status; - probeSignal = result.signal; - const timedOut = (result.error as NodeJS.ErrnoException | undefined)?.code === "ETIMEDOUT"; - const marker = readProbeMetadata(markerPath, 64); - const reentryMarker = readProbeMetadata(reentryPath, 64); - const groupText = readProbeMetadata(groupPath, 64); - const launcherStderr = readProbeMetadata(stderrPath, MAX_DIAGNOSTIC_VALUE_BYTES); - groupId = groupText === null ? 0 : Number.parseInt(groupText, 10); - if (marker === null) return shimProbeCleanup("marker", result.error, probeStatus, probeSignal); - if (reentryMarker === null) return shimProbeCleanup("reentry", result.error, probeStatus, probeSignal); - if (groupText === null) return shimProbeCleanup("group", result.error, probeStatus, probeSignal); - if (launcherStderr === null) return shimProbeCleanup("stderr", result.error, probeStatus, probeSignal); - if (!Number.isInteger(groupId) || groupId <= 0) return shimProbeCleanup("group-id", result.error, probeStatus, probeSignal); - const groupSurvived = unixProcessGroupAlive(groupId); - if (timedOut || marker || reentryMarker || groupSurvived) { - try { - terminateUnixProcessGroup(groupId); - } catch (error) { - return shimProbeCleanup("termination", error, probeStatus, probeSignal); - } - } - if (result.error && !timedOut) return shimProbeCleanup("spawn", result.error, probeStatus, probeSignal); - if (timedOut || marker === "timeout") return "timeout"; - if (marker === "recursive" || reentryMarker === "recursive") return "recursive"; - if (reentryMarker !== "") return shimProbeCleanup("reentry", undefined, probeStatus, probeSignal); - if (marker === "descendants") return "descendants"; - if (groupSurvived) return "descendants"; - if (result.status === CODEX_SHIM_REENTRY_EXIT_CODE && launcherStderr.includes(CODEX_SHIM_REENTRY_DIAGNOSTIC)) { - return "recursive"; - } - if (result.status !== 0) return "failed"; - return null; - } catch (error) { - if (Number.isInteger(groupId) && groupId > 0) { - try { terminateUnixProcessGroup(groupId); } catch { /* cleanup classification below */ } - } - return shimProbeCleanup("exception", error, probeStatus, probeSignal); - } finally { - try { rmSync(probeDir, { recursive: true, force: true }); } catch { /* best-effort cleanup */ } - } -} - -function probeUnixShimFiles(files: readonly ShimFileState[]): UnixShimProbeResult { - if (process.platform === "win32") return null; - codexShimProbeHookForTests?.(); - return files - .filter(file => !file.preserveOnly) - .map(file => probeUnixShimInstall(file.wrapperPath)) - .find(result => result !== null) ?? null; -} - -function unixProcessGroupAlive(groupId: number): boolean { - try { - process.kill(-groupId, 0); - return true; - } catch (error) { - return (error as NodeJS.ErrnoException).code !== "ESRCH"; - } -} - -function terminateUnixProcessGroup(groupId: number): void { - let permissionError: unknown; - try { - process.kill(-groupId, "SIGKILL"); - } catch (error) { - const code = (error as NodeJS.ErrnoException).code; - if (code === "EPERM") permissionError = error; - else if (code !== "ESRCH") throw error; - } - // A concurrently exiting group can briefly reject a second signal. Only - // observed disappearance clears that uncertainty; never send another signal. - const deadline = Date.now() + CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS; - while (Date.now() < deadline && unixProcessGroupAlive(groupId)) Bun.sleepSync(10); - if (unixProcessGroupAlive(groupId)) { - if (permissionError) throw permissionError; - throw new Error(`Codex shim install probe process group ${groupId} did not terminate`); - } -} - interface FreshShimInstallJournalEntry { target: ShimFileState; movedOriginalFingerprint?: ShimPathFingerprint; @@ -1047,374 +347,6 @@ function rollbackFreshShimInstall(journal: readonly FreshShimInstallJournalEntry if (errors.length > 0) throw new AggregateError(errors, "Codex shim install validation rollback failed"); } -function windowsBatchValue(value: string): string { - return value - .replace(/%/g, "%%") - .replace(/\^/g, "^^") - .replace(/"/g, "") - .replace(/[\r\n]/g, ""); -} - -function windowsBatchSet(name: string, value: string): string { - // Paths are rewritten to %USERPROFILE%-style env indirection: cmd.exe parses .cmd - // files in the OEM codepage, so a literal non-ASCII profile prefix (Korean/Chinese - // usernames) written as UTF-8 turns to mojibake. The env token expands natively in - // the right codepage at parse time; no `chcp` here — this shim runs in the USER's - // console and must not leak a codepage change into it. - return `set "${name}=${windowsEnvIndirectBatchValue(value, windowsBatchValue)}"`; -} - -export function buildWindowsCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource): string { - const internalCommandChecks = CODEX_INTERNAL_COMMANDS.map(command => `if /I "%~1"=="${command}" goto run_codex`).join("\r\n"); - const valueOptionChecks = CODEX_GLOBAL_OPTIONS_WITH_VALUE.map(option => `if /I "%~1"=="${option}" goto skip_option_value`).join("\r\n"); - return `@echo off\r -rem ${SHIM_MARKER}\r -setlocal\r -${windowsBatchSet("OCX_REAL_CODEX", realCodexPath)}\r -${windowsBatchSet("OCX_BUN", bunPath)}\r -${windowsBatchSet("OCX_CLI", cliPath)}\r -${windowsBatchSet("OCX_API_TOKEN_FILE", serviceApiTokenFilePath())}\r -if "%OPENCODEX_API_AUTH_TOKEN%"=="" if exist "%OCX_API_TOKEN_FILE%" set /p OPENCODEX_API_AUTH_TOKEN=<"%OCX_API_TOKEN_FILE%"\r -if not "%OCX_SHIM_BYPASS%"=="" goto run_codex\r -goto scan_codex_args\r -:scan_codex_args\r -if "%~1"=="" goto ensure_ocx\r -if "%~1"=="--" goto ensure_ocx\r -${valueOptionChecks}\r -${internalCommandChecks}\r -if /I "%~1"=="--help" goto run_codex\r -if /I "%~1"=="-h" goto run_codex\r -if /I "%~1"=="--version" goto run_codex\r -if /I "%~1"=="-V" goto run_codex\r -set "OCX_SCAN_ARG=%~1"\r -if "%OCX_SCAN_ARG:~0,1%"=="-" goto shift_codex_arg\r -goto ensure_ocx\r -:skip_option_value\r -shift\r -if "%~1"=="" goto ensure_ocx\r -:shift_codex_arg\r -shift\r -goto scan_codex_args\r -:ensure_ocx\r -setlocal\r -${windowsBatchSet(BUN_RUNTIME_SOURCE_ENV, bunRuntimeSource)}\r -${windowsBatchSet(BUN_RUNTIME_PATH_ENV, bunPath)}\r -"%OCX_BUN%" "%OCX_CLI%" ensure >nul 2>nul\r -endlocal\r -:run_codex\r -"%OCX_REAL_CODEX%" %*\r -`; -} - -function psString(value: string): string { - return `'${value.replace(/'/g, "''")}'`; -} - -export function buildWindowsPowerShellCodexShim(realCodexPath: string, bunPath: string, cliPath: string, bunRuntimeSource: BunRuntimeSource): string { - const internalCommands = CODEX_INTERNAL_COMMANDS.map(command => psString(command)).join(", "); - const valueOptions = CODEX_GLOBAL_OPTIONS_WITH_VALUE.map(option => psString(option)).join(", "); - const tokenFile = serviceApiTokenFilePath(); - return `#!/usr/bin/env pwsh -# ${SHIM_MARKER} -$hadApiAuthToken = Test-Path Env:\\OPENCODEX_API_AUTH_TOKEN -$priorApiAuthToken = $env:OPENCODEX_API_AUTH_TOKEN -try { -if (-not $env:OPENCODEX_API_AUTH_TOKEN -and (Test-Path -LiteralPath ${psString(tokenFile)})) { - $env:OPENCODEX_API_AUTH_TOKEN = (Get-Content -Raw -LiteralPath ${psString(tokenFile)}).Trim() -} -$internalCommands = @(${internalCommands}) -$valueOptions = @(${valueOptions}) -$subcommand = "" -$skipNext = $false -foreach ($argValue in $args) { - $argText = [string]$argValue - if ($skipNext) { $skipNext = $false; continue } - if ($argText -eq "--") { break } - if ($valueOptions -contains $argText) { $skipNext = $true; continue } - if (@("--help", "-h", "--version", "-V") -contains $argText) { $subcommand = $argText; break } - if ($argText.StartsWith("-")) { continue } - $subcommand = $argText - break -} -$skipEnsure = $env:OCX_SHIM_BYPASS -or $internalCommands -contains $subcommand -or @("--help", "-h", "--version", "-V") -contains $subcommand -if (-not $skipEnsure) { - $priorRuntimeSource = $env:${BUN_RUNTIME_SOURCE_ENV} - $priorRuntimePath = $env:${BUN_RUNTIME_PATH_ENV} - $env:${BUN_RUNTIME_SOURCE_ENV} = ${psString(bunRuntimeSource)} - $env:${BUN_RUNTIME_PATH_ENV} = ${psString(bunPath)} - try { & ${psString(bunPath)} ${psString(cliPath)} ensure *> $null } - finally { - if ($null -eq $priorRuntimeSource) { Remove-Item Env:\\${BUN_RUNTIME_SOURCE_ENV} -ErrorAction SilentlyContinue } - else { $env:${BUN_RUNTIME_SOURCE_ENV} = $priorRuntimeSource } - if ($null -eq $priorRuntimePath) { Remove-Item Env:\\${BUN_RUNTIME_PATH_ENV} -ErrorAction SilentlyContinue } - else { $env:${BUN_RUNTIME_PATH_ENV} = $priorRuntimePath } - } -} -& ${psString(realCodexPath)} @args -$codexExitCode = $LASTEXITCODE -} finally { - if ($hadApiAuthToken) { $env:OPENCODEX_API_AUTH_TOKEN = $priorApiAuthToken } - else { Remove-Item Env:\\OPENCODEX_API_AUTH_TOKEN -ErrorAction SilentlyContinue } -} -exit $codexExitCode -`; -} - -interface ShimStateReadResult { - state: ShimState | null; - present: boolean; - warning?: string; -} - -function fileErrorCode(error: unknown): string | undefined { - return error && typeof error === "object" && "code" in error - ? String((error as { code?: unknown }).code) - : undefined; -} - -function readBoundedRegularFile(path: string, maxBytes: number): { bytes: Buffer; content: string } | { warning: string } | null { - let lexicalBefore: Stats; - try { - lexicalBefore = lstatSync(path); - if (lexicalBefore.isSymbolicLink() || !lexicalBefore.isFile()) { - return { warning: `Codex shim state is not a direct regular file at ${path}; auto-restore skipped.` }; - } - } catch (error) { - if (fileErrorCode(error) === "ENOENT") return null; - return { warning: `Codex shim state could not be inspected at ${path}.` }; - } - let fd: number; - try { - fd = openSync(path, "r"); - } catch (error) { - if (fileErrorCode(error) === "ENOENT") return null; - return { warning: `Codex shim state could not be opened as a regular file at ${path}.` }; - } - try { - const before = fstatSync(fd); - if (!before.isFile()) return { warning: `Codex shim state is not a regular file at ${path}; auto-restore skipped.` }; - if (before.size > maxBytes) { - return { warning: `Codex shim state exceeds the 1 MiB startup limit at ${path}; auto-restore skipped.` }; - } - const buffer = Buffer.allocUnsafe(before.size); - let offset = 0; - while (offset < buffer.length) { - const bytesRead = readSync(fd, buffer, offset, buffer.length - offset, offset); - if (bytesRead === 0) return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; - offset += bytesRead; - } - const extra = Buffer.allocUnsafe(1); - if (readSync(fd, extra, 0, 1, offset) !== 0) { - return { warning: `Codex shim state exceeds the 1 MiB startup limit at ${path}; auto-restore skipped.` }; - } - const after = fstatSync(fd); - let lexicalAfter: Stats; - try { - lexicalAfter = lstatSync(path); - } catch { - return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; - } - if (before.dev !== after.dev || before.ino !== after.ino || before.size !== after.size - || before.mtimeMs !== after.mtimeMs || before.ctimeMs !== after.ctimeMs - || lexicalBefore.dev !== before.dev || lexicalBefore.ino !== before.ino - || lexicalAfter.isSymbolicLink() || lexicalAfter.dev !== after.dev || lexicalAfter.ino !== after.ino) { - return { warning: `Codex shim state changed while being read at ${path}; auto-restore skipped.` }; - } - return { bytes: buffer, content: buffer.toString("utf8") }; - } finally { - closeSync(fd); - } -} - -function readStateResult(path = statePath()): ShimStateReadResult { - const bounded = readBoundedRegularFile(path, CODEX_SHIM_STATE_MAX_BYTES); - if (!bounded) return { state: null, present: false }; - if ("warning" in bounded) return { state: null, present: true, warning: bounded.warning }; - try { - const value = JSON.parse(bounded.content) as unknown; - if (!value || typeof value !== "object") return { state: null, present: true }; - const state = value as Record; - if (typeof state.platform !== "string") return { state: null, present: true }; - const validFile = (item: unknown): item is ShimFileState => { - if (!item || typeof item !== "object") return false; - const file = item as Record; - return typeof file.wrapperPath === "string" - && typeof file.originalPath === "string" - && typeof file.backupPath === "string" - && (file.realPath === undefined || typeof file.realPath === "string") - && (file.preserveOnly === undefined || typeof file.preserveOnly === "boolean"); - }; - if (state.wrappers !== undefined) { - if (!Array.isArray(state.wrappers) || state.wrappers.length === 0 || !state.wrappers.every(validFile)) return { state: null, present: true }; - } else if (!validFile(state)) { - return { state: null, present: true }; - } - return { state: state as unknown as ShimState, present: true }; - } catch { - return { state: null, present: true }; - } -} - -function readState(): ShimState | null { - return readStateResult().state; -} - -export function isLocalAbsoluteInspectionPath(path: string, platform: NodeJS.Platform): boolean { - if (platform !== "win32") return posix.isAbsolute(path); - const normalized = path.replace(/\//g, "\\"); - // UNC and device namespaces can initiate remote I/O while a nominally local - // inspection is resolving user-controlled paths. Root-relative paths are - // drive-context dependent, so require an explicit local drive as well. - return win32.isAbsolute(path) - && /^[a-z]:\\/i.test(normalized) - && !normalized.startsWith("\\\\"); -} - -function windowsShimInspectionIsDeferred(platform: NodeJS.Platform): boolean { - return platform === "win32"; -} - -/** Resolve one selected command through already-recorded shim state, without repair. */ -export function inspectCodexShimBackingForCommand( - selectedCommand: string, - platform: NodeJS.Platform = process.platform, - configDir: string = getConfigDir(), -): CodexShimBackingForCommand { - // Pathname prechecks cannot prevent a writable Windows ancestor from being - // replaced with a remote reparse point before the later state/fingerprint - // reads. Keep the exported read-only helper fail-closed until those reads are - // performed through a handle-bound Windows provenance layer. - if (windowsShimInspectionIsDeferred(platform)) { - return Object.freeze({ status: "unknown" as const, reason: "binding_unavailable" as const }); - } - if (!isLocalAbsoluteInspectionPath(configDir, platform)) { - return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); - } - const stateFile = join(configDir, "codex-shim.json"); - try { - const stateEntry = lstatSync(stateFile); - if (stateEntry.isSymbolicLink()) { - return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); - } - } catch (error) { - if (fileErrorCode(error) !== "ENOENT") { - return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); - } - } - const result = readStateResult(stateFile); - if (!result.state) { - return result.present - ? Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }) - : Object.freeze({ status: "not-tracked" as const }); - } - const pathApi = platform === "win32" ? win32 : posix; - const samePath = (left: string, right: string): boolean => { - const normalizedLeft = pathApi.resolve(left); - const normalizedRight = pathApi.resolve(right); - return platform === "win32" - ? normalizedLeft.toLowerCase() === normalizedRight.toLowerCase() - : normalizedLeft === normalizedRight; - }; - const files = stateFiles(result.state); - if (files.some(file => !file.wrapperPath || !file.originalPath || !file.backupPath - || ![file.wrapperPath, file.originalPath, file.backupPath, file.realPath] - .filter((path): path is string => typeof path === "string") - .every(path => isLocalAbsoluteInspectionPath(path, platform)))) { - return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); - } - const wrapperKeys = files.map(file => platform === "win32" - ? pathApi.resolve(file.wrapperPath).toLowerCase() - : pathApi.resolve(file.wrapperPath)); - if (new Set(wrapperKeys).size !== wrapperKeys.length) { - return Object.freeze({ status: "unknown" as const, reason: "state_invalid" as const }); - } - const selectedFingerprint = shimPathFingerprint(selectedCommand); - if (!selectedFingerprint) { - return Object.freeze({ status: "unknown" as const, reason: "binding_unavailable" as const }); - } - const selectedIdentity = selectedFingerprint.target ?? selectedFingerprint; - const sameEffectiveIdentity = (fingerprint: ShimPathFingerprint | null): boolean => { - if (!fingerprint) return false; - const identity = fingerprint.target ?? fingerprint; - return identity.dev === selectedIdentity.dev && identity.ino === selectedIdentity.ino; - }; - const matches = files.flatMap(file => { - const backingPath = file.realPath ?? file.backupPath; - const roles: Array<"wrapper" | "backing"> = []; - if (samePath(file.wrapperPath, selectedCommand) - || sameEffectiveIdentity(shimPathFingerprint(file.wrapperPath))) { - roles.push("wrapper"); - } - if (samePath(backingPath, selectedCommand) - || sameEffectiveIdentity(shimPathFingerprint(backingPath))) { - roles.push("backing"); - } - return roles.map(selectedRole => ({ file, backingPath, selectedRole })); - }); - if (matches.length === 0) return Object.freeze({ status: "not-tracked" as const }); - if (result.state.platform !== platform) { - return Object.freeze({ status: "unknown" as const, reason: "platform_mismatch" as const }); - } - if (matches.length !== 1) { - return Object.freeze({ status: "unknown" as const, reason: "ambiguous_match" as const }); - } - const { file, backingPath, selectedRole } = matches[0]!; - if (file.preserveOnly === true) { - return Object.freeze({ status: "unknown" as const, reason: "preserve_only" as const }); - } - const backing = statFingerprint(backingPath, true); - if (!backing || backing.size <= 0 || samePath(backingPath, file.wrapperPath)) { - return Object.freeze({ status: "unknown" as const, reason: "backing_missing" as const }); - } - const wrapperProbe = stableShimPathProbe(file.wrapperPath); - if (!wrapperProbe || !isHealthyShimProbe(wrapperProbe, result.state.platform)) { - return Object.freeze({ - status: "unknown" as const, - reason: isVersionManagerOwnedCodexPath(file.wrapperPath) - ? "version_manager_refused" as const - : "wrapper_unhealthy" as const, - }); - } - const wrapperIdentity = wrapperProbe.fingerprint.target ?? wrapperProbe.fingerprint; - if (backing.dev === wrapperIdentity.dev && backing.ino === wrapperIdentity.ino) { - return Object.freeze({ status: "unknown" as const, reason: "backing_mismatch" as const }); - } - const wrapperExt = extname(file.wrapperPath).toLowerCase(); - const invokesBacking = platform !== "win32" - ? wrapperProbe.prefix.includes(`exec ${shQuote(backingPath)} "$@"`) - : wrapperExt === ".cmd" || wrapperExt === ".bat" - ? wrapperProbe.prefix.includes(windowsBatchSet("OCX_REAL_CODEX", backingPath)) - && wrapperProbe.prefix.includes('"%OCX_REAL_CODEX%" %*') - : wrapperExt === ".ps1" - ? wrapperProbe.prefix.includes(`& ${psString(backingPath)} @args`) - : wrapperProbe.prefix.includes(`exec ${shQuote(gitBashPath(backingPath))} "$@"`); - if (!invokesBacking) { - return Object.freeze({ status: "unknown" as const, reason: "backing_mismatch" as const }); - } - return Object.freeze({ - status: "matched" as const, - selectedRole, - backingPath, - backingKind: file.realPath !== undefined ? "real" as const : "backup" as const, - }); -} - -function statePath(): string { - return join(getConfigDir(), "codex-shim.json"); -} - -function writeState(state: ShimState): void { - const path = statePath(); - recordOwnedConfigPath(getConfigDir(), path); - if (!existsSync(getConfigDir())) mkdirSync(getConfigDir(), { recursive: true }); - writeFileSync(path, JSON.stringify(state, null, 2) + "\n", "utf8"); -} - -/** Git-Bash accepts `C:/...` but not backslashed paths inside sh scripts. */ -function gitBashPath(path: string): string { - return path.replace(/\\/g, "/"); -} - /** * Write the wrapper and return the identity of the inode this call created, or * `undefined` where the platform still writes the destination in place. @@ -1519,12 +451,6 @@ function wrapperInodeIsOurs( return written !== undefined; } -function stateFiles(state: ShimState): ShimFileState[] { - return state.wrappers?.length - ? state.wrappers - : [{ wrapperPath: state.wrapperPath, originalPath: state.originalPath, backupPath: state.backupPath }]; -} - function primaryState(files: ShimFileState[]): ShimState { const first = files[0]!; return { platform: process.platform, ...first, wrappers: files }; @@ -1611,152 +537,6 @@ interface GuardedRefreshJournalEntry { let guardedRefreshTransactionId = 0; -interface ShimRestoreLock { - release(): void; -} - -interface ShimRestoreLockRecord { - version: 1; - token: string; - pid: number; - createdAt: number; -} - -interface ShimRestoreLockSnapshot { - record: ShimRestoreLockRecord; - ownerPath: string; - lockIdentity: Pick; - fingerprint: ShimPathFingerprint; -} - -function restoreLockPath(): string { - return join(getConfigDir(), "codex-shim.autorestore.lock"); -} - -function sameFileIdentity(left: Pick, right: Pick): boolean { - return left.dev === right.dev && left.ino === right.ino; -} - -function readShimRestoreLockSnapshot(path: string): ShimRestoreLockSnapshot | null { - let lockIdentity: Stats; - let entries: string[]; - try { - lockIdentity = lstatSync(path); - if (!lockIdentity.isDirectory()) return null; - entries = readdirSync(path); - } catch { - return null; - } - if (entries.length !== 1 || !entries[0].endsWith(".json")) return null; - const ownerPath = join(path, entries[0]); - const probe = stableShimPathProbe(ownerPath); - if (!probe || probe.fingerprint.kind !== "file" || probe.fingerprint.size > 4096) return null; - try { - const value = JSON.parse(probe.prefix) as Partial; - if (value.version !== 1 || typeof value.token !== "string" || value.token.length === 0 - || typeof value.pid !== "number" || !Number.isSafeInteger(value.pid) || value.pid <= 0 - || typeof value.createdAt !== "number" || !Number.isFinite(value.createdAt)) return null; - if (entries[0] !== `${value.token}.json`) return null; - const currentLockIdentity = lstatSync(path); - if (!currentLockIdentity.isDirectory() || !sameFileIdentity(lockIdentity, currentLockIdentity)) return null; - return { - record: value as ShimRestoreLockRecord, - ownerPath, - lockIdentity, - fingerprint: probe.fingerprint, - }; - } catch { - return null; - } -} - -function sameShimRestoreLock(left: ShimRestoreLockSnapshot, right: ShimRestoreLockSnapshot): boolean { - return left.record.token === right.record.token - && sameFileIdentity(left.lockIdentity, right.lockIdentity) - && sameFingerprint(left.fingerprint, right.fingerprint); -} - -function reclaimStaleRestoreLock(path: string, beforeDelete?: () => void): boolean { - const observed = readShimRestoreLockSnapshot(path); - if (!observed) return false; - const createdAt = Math.max(observed.record.createdAt, observed.fingerprint.mtimeMs); - if (Date.now() - createdAt <= CODEX_SHIM_RESTORE_LOCK_STALE_MS) return false; - if (isProcessAlive(observed.record.pid)) return false; - const current = readShimRestoreLockSnapshot(path); - if (!current || !sameShimRestoreLock(observed, current)) return false; - beforeDelete?.(); - try { - // The token is part of the owner filename. Even if the lock directory is - // replaced after the comparison, this unlink cannot target a successor's - // differently named owner record. - unlinkSync(observed.ownerPath); - rmdirSync(path); - return true; - } catch { - return false; - } -} - -function tryAcquireShimRestoreLock(beforeStaleDelete?: () => void): ShimRestoreLock | null { - const dir = getConfigDir(); - if (!existsSync(dir)) mkdirSync(dir, { recursive: true, mode: 0o700 }); - const path = restoreLockPath(); - for (let attempt = 0; attempt < 2; attempt += 1) { - let fd: number | null = null; - let identity: Stats | null = null; - let createdDirectory = false; - const record: ShimRestoreLockRecord = { - version: 1, - token: `${process.pid}-${Date.now()}-${randomUUID()}`, - pid: process.pid, - createdAt: Date.now(), - }; - const ownerPath = join(path, `${record.token}.json`); - try { - mkdirSync(path, { mode: 0o700 }); - createdDirectory = true; - fd = openSync(ownerPath, "wx", 0o600); - identity = fstatSync(fd); - writeFileSync(fd, `${JSON.stringify(record)}\n`, "utf8"); - identity = fstatSync(fd); - let released = false; - return { - release(): void { - if (released) return; - released = true; - try { closeSync(fd!); } catch { /* stale recovery handles an uncertain lock */ } - try { - const current = readShimRestoreLockSnapshot(path); - if (identity && current && current.record.token === record.token - && sameFileIdentity(identity, current.fingerprint)) { - unlinkSync(ownerPath); - rmdirSync(path); - } - } catch { /* stale recovery handles release failures */ } - }, - }; - } catch (error) { - if (fd !== null) { - try { closeSync(fd); } catch { /* best-effort close before ownership cleanup */ } - try { - const current = readShimRestoreLockSnapshot(path); - if (identity && current && current.record.token === record.token - && sameFileIdentity(identity, current.fingerprint)) { - unlinkSync(ownerPath); - rmdirSync(path); - } - } catch { /* leave an uncertain lock for stale recovery */ } - } else if (createdDirectory) { - try { rmdirSync(path); } catch { /* another owner exists or cleanup is uncertain */ } - } - if (fileErrorCode(error) !== "EEXIST") throw error; - if (attempt === 0 && reclaimStaleRestoreLock(path, beforeStaleDelete)) continue; - return null; - } - } - return null; -} - function planGuardedRefreshTransaction( files: readonly ShimFileState[], expectedReplacements: ReadonlyMap, diff --git a/src/providers/quota.ts b/src/providers/quota.ts index 386e921391..e02447ffb0 100644 --- a/src/providers/quota.ts +++ b/src/providers/quota.ts @@ -1,3078 +1,102 @@ -import { createHash } from "node:crypto"; -import { - effectiveCodexAuthAccountId, - fetchMainAccountInfoSnapshot, - listCodexAuthAccountsSnapshot, -} from "../codex/auth-api"; -import { withoutRetiredCodexQuota, type StoredAccountQuota } from "../codex/quota"; -import { isMainAccountIdentityGenerationLive } from "../codex/main-account-cache"; -import { MAIN_CODEX_ACCOUNT_ID } from "../codex/main-account"; -import { codexPlanKey } from "../codex/plan"; -import { resolveEnvValue } from "../config"; -import { resolveProviderApiKey } from "./key-store"; -import { getValidAccessToken, getValidAccessTokenForAccount } from "../oauth"; -import { getAccountCredential, getAccountSet, getCredential } from "../oauth/store"; -import { antigravityUserAgent } from "../adapters/client-fingerprint"; -import { isCanonicalOllamaCloudUrl } from "../adapters/ollama-native-url"; -import { DestinationDnsResolutionError } from "../lib/destination-policy"; -import { PinnedHttpError } from "../lib/pinned-http"; -import { ProviderOutboundPolicyError, providerOutboundPost, providerRedirectError, type ProviderOutboundDependencies } from "../lib/provider-outbound"; -import { apiKeyPoolEntryId } from "./api-keys"; -import { fetchMuseKeyQuotaSnapshot } from "./muse-key-quota"; -import { XAI_GROK_CLIENT_VERSION, XAI_GROK_COMPATIBILITY } from "./xai-transport"; -import { getProviderRegistryEntry, providerCodexAccountMode, registryEntryForProviderDestination } from "./registry"; -import type { OcxConfig, OcxProviderConfig } from "../types"; -import { isCanonicalOpenAiForwardProvider, OPENAI_CODEX_PROVIDER_ID } from "./openai-tiers"; -import { - captureConfigGeneration, - sweepExpiredOnWrite, - type GenerationContext, -} from "../lib/state-store-sweeper"; -import { - ACCOUNT_QUOTA_TTL_MS, - asRecord, - CACHE_TTL_MS, - normalizePercent, - normalizeResetAt, - QUOTA_JSON_READ_FAILURE, - readQuotaJson, - REQUEST_TIMEOUT_MS, - toFiniteNumber, -} from "./quota-wire"; -import { - clearCachedProviderQuotas, - providerQuotaRoutingBinding, - replaceCachedProviderQuotas, - type ProviderQuotaRoutingEvidence, -} from "./quota-routing-cache"; -import { - aggregateCodexPoolCapacity, - CODEX_CAPACITY_MAX_QUOTA_AGE_MS, - type CodexCapacityAggregation, - type CodexCapacityQuota, -} from "./codex-capacity"; -import type { - AccountQuotaMode, - QuotaFailureCode, - ProviderQuota, - ProviderQuotaCreditsUsd, - ProviderQuotaWindow, - ProviderRoutingQuota, -} from "./quota-types"; -import { - clearKiroAccountUsageState, - commitKiroAccountUsageState, - fetchKiroUsageSnapshot, - type KiroUsageSnapshot, - kiroUsageContextForAccount, - reconcileKiroAccountUsageState, -} from "./kiro-usage"; -import { - cancelPendingAccountQuotaPersist, - readPersistedAccountQuotas, - schedulePersistAccountQuotas, -} from "./account-quota-disk"; -import { clearProviderApiKeyQuotaCache, mapQuotaRoster, readProviderApiKeyQuotas, type ProviderApiKeyQuota } from "./quota-key-accounts"; - -export type { ProviderQuota, ProviderQuotaCreditsUsd, ProviderQuotaWindow } from "./quota-types"; - -/** Match oauth/index REFRESH_SKEW_MS — use stored access without refresh when still fresh. */ -const ACCOUNT_TOKEN_SKEW_MS = 60_000; -/** Successful provider quota payloads are small; reject oversized or stalled JSON before parsing. */ -export { QUOTA_RESPONSE_MAX_BYTES } from "./quota-wire"; -const KIMI_CODE_BASE_URL = "https://api.kimi.com/coding/v1"; -const KIMI_CODE_USAGE_URL = `${KIMI_CODE_BASE_URL}/usages`; -const COMMAND_CODE_BASE_URL = "https://api.commandcode.ai"; -const COMMAND_CODE_WHOAMI_URL = `${COMMAND_CODE_BASE_URL}/alpha/whoami`; -const COMMAND_CODE_CREDITS_URL = `${COMMAND_CODE_BASE_URL}/alpha/billing/credits`; -const COMMAND_CODE_SUBSCRIPTIONS_URL = `${COMMAND_CODE_BASE_URL}/alpha/billing/subscriptions`; -const COMMAND_CODE_USAGE_URL = `${COMMAND_CODE_BASE_URL}/alpha/usage/summary`; -const A6API_BASE_URL = "https://api.a6api.com"; -const OPENCODE_GO_BASE_URL = "https://opencode.ai/zen/go/v1"; -const OPENCODE_GO_USAGE_URL = `${OPENCODE_GO_BASE_URL}/usage`; -const OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"; -const DEEPSEEK_BASE_URL = "https://api.deepseek.com"; -const CLINE_BASE_URL = "https://api.cline.bot"; -const OLLAMA_CLOUD_BASE_URL = "https://ollama.com"; -const OLLAMA_CLOUD_USAGE_URL = `${OLLAMA_CLOUD_BASE_URL}/api/usage`; -const ZAI_BASE_URL = "https://api.z.ai"; -const ZAI_CN_BASE_URL = "https://open.bigmodel.cn"; -const MINIMAX_REMAINS_URL = "https://www.minimax.io/v1/token_plan/remains"; -const MOONSHOT_BASE_URL = "https://api.moonshot.ai/v1"; -const VENICE_BASE_URL = "https://api.venice.ai/api/v1"; -const SYNTHETIC_BASE_URL = "https://api.synthetic.new/v2"; -const DEEPINFRA_BASE_URL = "https://api.deepinfra.com"; -const NEURALWATT_BASE_URL = "https://api.neuralwatt.com/v1"; -const XAI_BILLING_URL = "https://cli-chat-proxy.grok.com/v1/billing"; -const XAI_CREDITS_URL = `${XAI_BILLING_URL}?format=credits`; -/** Keep a failed probe's previous row at most this long before dropping it. */ -const LAST_GOOD_MAX_AGE_MS = CODEX_CAPACITY_MAX_QUOTA_AGE_MS; -const nativeMainReportGenerations = new WeakMap(); -const accountReportCurrent = new WeakMap boolean>(); -const routingEvidence = new WeakMap(); -let providerQuotaBeforePublishForTests: (() => void | Promise) | null = null; - -/** Test-only seam for identity/config invalidation after probes but before publication. */ -export function setProviderQuotaBeforePublishForTests( - hook: (() => void | Promise) | null, -): void { - providerQuotaBeforePublishForTests = hook; -} -const TERMINAL_QUOTA_FAILURE = Symbol("terminal-quota-failure"); -/** - * The probe succeeded and the upstream authoritatively reported NO model-quota windows. - * - * Distinct from `null`, which means "this probe told us nothing" and deliberately preserves - * the last-good row for up to 30 minutes. Collapsing the two would let a stale report outlive - * the authoritative answer that replaced it: a GLM plan whose payload carries only MCP - * `TIME_LIMIT` rows has no model windows, and the dashboard and quota-aware routing must stop - * showing the previous token windows rather than keep them for another half hour. - * - * Suppression is shared with `TERMINAL_QUOTA_FAILURE`; only the reason differs. - */ -const AUTHORITATIVE_EMPTY_QUOTA = Symbol("authoritative-empty-quota"); -type ProviderQuotaProbeResult = - | ProviderQuotaReport - | null - | typeof TERMINAL_QUOTA_FAILURE - | typeof AUTHORITATIVE_EMPTY_QUOTA; - -export interface ProviderQuotaReport { - provider: string; - label: string; - source: string; - quota: ProviderQuota; - updatedAt: number; - /** Added by the management response projection, never stored on a cached report. */ - routingQuota?: ProviderRoutingQuota; - reverseEngineered?: boolean; - /** - * The row was OBSERVED in-band on a streaming turn rather than probed. - * - * Age means something different for these. A probed provider re-reads on its own TTL, - * so a row older than the last-good bound means the probe is failing and showing it - * would misrepresent a live number. A passive provider publishes no endpoint at all - * (`hasPassiveAccountQuota`), so its last observation is not a stale reading of - * something fresher — it is the only measurement that exists, and dropping it leaves - * the operator with nothing. Consumers that enforce a freshness bound must exempt - * these and state the observation age instead. - */ - observed?: boolean; - aggregation?: CodexCapacityAggregation; -} - -export interface ProviderQuotaResponse { - generatedAt: number; - reports: ProviderQuotaReport[]; -} - -let cache: { key: string; ts: number; response: ProviderQuotaResponse } | null = null; -const inflight = new Map }>(); -/** Bumped on cache clear and on force-refresh start; stale-epoch probes lose commit authority. */ -let invalidationEpoch = 0; - -/** Invalidate the report cache (e.g. after switching a provider's active account). */ -export function clearProviderQuotaCache(): void { - cache = null; - clearCachedProviderQuotas(); - clearProviderApiKeyQuotaCache(); - invalidationEpoch += 1; -} - -function cacheKey(config: OcxConfig): string { - const providers = Object.entries(config.providers) - .map(([name, provider]) => { - const resolvedKey = typeof provider.apiKey === "string" - ? resolveProviderApiKey(provider.apiKey)?.trim() - : undefined; - const activeKeyId = resolvedKey ? apiKeyPoolEntryId(resolvedKey) : "none"; - return `${name}:${provider.adapter}:${provider.authMode ?? "key"}:${providerCodexAccountMode(name, provider) ?? "none"}:${provider.disabled === true ? "off" : "on"}:${provider.baseUrl}:${activeKeyId}`; - }) - .sort() - .join("|"); - return `${config.defaultProvider}|${providers}`; -} - -type CodexAuthAccountsSnapshotPromise = ReturnType; - -function hasCodexPoolProvider(config: OcxConfig): boolean { - return Object.entries(config.providers).some(([name, provider]) => ( - provider.disabled !== true - && isBuiltInChatGptForwardProvider(name, provider) - && providerCodexAccountMode(name, provider) !== "direct" - )); -} - -function quotaSignatureValue(quota: CodexCapacityQuota | null): unknown { - if (!quota) return null; - return { - fiveHourPercent: quota.fiveHourPercent, - fiveHourResetAt: quota.fiveHourResetAt, - weeklyPercent: quota.weeklyPercent, - weeklyResetAt: quota.weeklyResetAt, - monthlyPercent: quota.monthlyPercent, - monthlyResetAt: quota.monthlyResetAt, - updatedAt: quota.updatedAt, - customWindows: [...(quota.customWindows ?? [])] - .map(window => ({ label: window.label, percent: window.percent, resetAt: window.resetAt })) - .sort((a, b) => a.label.localeCompare(b.label)), - }; -} - -function providerQuotaFromCodexQuota( - quota: StoredAccountQuota | Omit | null | undefined, -): CodexCapacityQuota | null { - if (!quota) return null; - // Direct snapshots bypass account DTOs; sanitize here as well as at ingestion. - quota = withoutRetiredCodexQuota(quota); - if (!quota) return null; - const projected: CodexCapacityQuota = { - ...(quota.shortPercent !== undefined ? { fiveHourPercent: quota.shortPercent } : {}), - ...(quota.shortResetAt !== undefined ? { fiveHourResetAt: quota.shortResetAt } : {}), - ...(quota.weeklyPercent !== undefined ? { weeklyPercent: quota.weeklyPercent } : {}), - ...(quota.weeklyResetAt !== undefined ? { weeklyResetAt: quota.weeklyResetAt } : {}), - ...(quota.monthlyPercent !== undefined ? { monthlyPercent: quota.monthlyPercent } : {}), - ...(quota.monthlyResetAt !== undefined ? { monthlyResetAt: quota.monthlyResetAt } : {}), - ...(quota.customWindows !== undefined ? { customWindows: quota.customWindows } : {}), - updatedAt: "updatedAt" in quota ? quota.updatedAt : Date.now(), - }; - return hasQuotaRows(projected) ? projected : null; -} - -/** Hash only presentation-relevant state; account ids and email addresses never enter the key. */ -function cacheKeyWithAggregationState( - config: OcxConfig, - prefetchedSnapshot?: CodexAuthAccountsSnapshotPromise, -): string | Promise { - const base = cacheKey(config); - if (!hasCodexPoolProvider(config)) return base; - return (async () => { - try { - const activeId = effectiveCodexAuthAccountId(config); - const snapshot = await (prefetchedSnapshot ?? listCodexAuthAccountsSnapshot(config, false)); - const rows = snapshot.accounts.map(account => ({ - isMain: account.isMain, - active: account.id === activeId, - plan: codexPlanKey(account.plan) ?? null, - paused: account.paused, - needsReauth: account.needsReauth === true, - quota: quotaSignatureValue(providerQuotaFromCodexQuota(account.quota)), - })); - const canonicalRows = rows.map(row => JSON.stringify(row)).sort(); - const digest = createHash("sha256").update(JSON.stringify(canonicalRows)).digest("hex").slice(0, 24); - return `${base}|codex-pool:${digest}`; - } catch { - return `${base}|codex-pool:unavailable`; - } - })(); -} - -function publicCapacityWindow(window: import("./codex-capacity").CodexCapacityWindowAggregation) { - const { totalWeight: _totalWeight, consumedWeight: _consumedWeight, remainingWeight: _remainingWeight, ...safe } = window; - return safe; -} - -/** Management API metadata intentionally omits configured/weighted unit counts. */ -function publicCapacityAggregation( - aggregation: CodexCapacityAggregation, - presentation: NonNullable, -): CodexCapacityAggregation { - const safeCurrentAccount = presentation === "coverage-only" && aggregation.currentAccount - ? { ...aggregation.currentAccount, quota: null } - : aggregation.currentAccount; - return { - ...aggregation, - presentation, - ...(safeCurrentAccount ? { currentAccount: safeCurrentAccount } : {}), - ...(aggregation.fiveHour ? { fiveHour: publicCapacityWindow(aggregation.fiveHour) } : {}), - ...(aggregation.weekly ? { weekly: publicCapacityWindow(aggregation.weekly) } : {}), - ...(aggregation.monthly ? { monthly: publicCapacityWindow(aggregation.monthly) } : {}), - ...(aggregation.customWindows ? { - customWindows: aggregation.customWindows.map(window => ({ - label: window.label, - ...publicCapacityWindow(window), - })), - } : {}), - }; -} - -function hasQuotaRows(quota: ProviderQuota | null | undefined): quota is ProviderQuota { - if (!quota) return false; - return typeof quota.fiveHourPercent === "number" - || typeof quota.weeklyPercent === "number" - || typeof quota.monthlyPercent === "number" - || quota.creditsUsd?.unlimited === true - || typeof quota.creditsUsd?.percent === "number" - || !!quota.customWindows?.some(window => typeof window.percent === "number"); -} - -function providerLabel(providerId: string): string { - return getProviderRegistryEntry(providerId)?.label ?? providerId; -} - -/** Test-only access to the quota reader's deadline and cancellation contract. */ -export async function readProviderQuotaJsonForTests(response: Response, timeoutMs: number): Promise { - const result = await readQuotaJson(response, timeoutMs); - return result === QUOTA_JSON_READ_FAILURE ? null : result; -} - -function isBuiltInChatGptForwardProvider(name: string, provider: OcxProviderConfig): boolean { - return name === OPENAI_CODEX_PROVIDER_ID && isCanonicalOpenAiForwardProvider(provider); -} - -function isCanonicalA6apiBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === A6API_BASE_URL || normalized === `${A6API_BASE_URL}/v1`; -} - -function isCanonicalOpenCodeGoBaseUrl(baseUrl: string): boolean { - return normalizedBaseUrl(baseUrl) === OPENCODE_GO_BASE_URL; -} - -function isCanonicalOpenRouterBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === OPENROUTER_BASE_URL; -} - -function isCanonicalDeepSeekBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === DEEPSEEK_BASE_URL || normalized === `${DEEPSEEK_BASE_URL}/v1`; -} - -function isCanonicalClineBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === CLINE_BASE_URL || normalized === `${CLINE_BASE_URL}/api/v1`; -} - -function isCanonicalOllamaCloudBaseUrl(baseUrl?: string): boolean { - if (!baseUrl) return false; - try { - return isCanonicalOllamaCloudUrl(baseUrl); - } catch { - return false; - } -} - -function zaiQuotaMonitorHost(baseUrl: string): string | null { - // Admission and destination selection must share one mapping: admitting a new - // international wire must never fall through to the CN host/authentication scheme. - switch (normalizedBaseUrl(baseUrl)) { - case ZAI_BASE_URL: - case `${ZAI_BASE_URL}/api/coding/paas/v4`: - case `${ZAI_BASE_URL}/api/anthropic`: - case `${ZAI_BASE_URL}/api/v1`: - return ZAI_BASE_URL; - case ZAI_CN_BASE_URL: - case `${ZAI_CN_BASE_URL}/api/coding/paas/v4`: - case `${ZAI_CN_BASE_URL}/api/v1`: - return ZAI_CN_BASE_URL; - default: - return null; - } -} - -function isCanonicalZaiBaseUrl(baseUrl: string): boolean { - return zaiQuotaMonitorHost(baseUrl) !== null; -} - -function isCanonicalMinimaxBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === "https://api.minimax.io/v1" || normalized === "https://api.minimaxi.com/v1"; -} - -function isCanonicalMoonshotBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === MOONSHOT_BASE_URL || normalized === "https://api.moonshot.cn/v1"; -} - -function isCanonicalVeniceBaseUrl(baseUrl: string): boolean { - return normalizedBaseUrl(baseUrl) === VENICE_BASE_URL; -} - -function isCanonicalSyntheticBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === SYNTHETIC_BASE_URL || normalized === "https://api.synthetic.new/openai/v1"; -} - -function isCanonicalDeepInfraBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - return normalized === DEEPINFRA_BASE_URL || normalized === `${DEEPINFRA_BASE_URL}/v1/openai`; -} - -function isCanonicalNeuralwattBaseUrl(baseUrl: string): boolean { - return normalizedBaseUrl(baseUrl) === NEURALWATT_BASE_URL; -} - -function a6apiPayload(value: unknown): Record | null { - const body = asRecord(value); - return asRecord(body?.data) ?? body; -} - -function firstFinite(record: Record | null, names: string[]): number | undefined { - if (!record) return undefined; - for (const name of names) { - const value = toFiniteNumber(record[name]); - if (value !== undefined) return value; - } - return undefined; -} - -async function fetchA6apiQuota(provider: string, config: OcxProviderConfig): Promise { - // Never send a configured API key to a lookalike host or through a redirect. - if (!isCanonicalA6apiBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const headers = { Accept: "application/json", Authorization: `Bearer ${apiKey}` } as const; - const [subscriptionResponse, tokenResponse] = await Promise.all([ - fetch(`${A6API_BASE_URL}/dashboard/billing/subscription`, { - headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }), - fetch(`${A6API_BASE_URL}/api/usage/token/`, { - headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }), - ]); - if (!subscriptionResponse.ok || !tokenResponse.ok) { - const statuses = [subscriptionResponse.status, tokenResponse.status]; - // 408/429 are transient (timeout/throttle), not invalid-account signals: keep the - // last-good row like 5xx/network failures. 401/403 (bad key) and 404 (contract change) - // stay terminal. - return statuses.some(status => status >= 400 && status < 500 && status !== 429 && status !== 408) - ? TERMINAL_QUOTA_FAILURE - : null; - } - const [subscriptionBody, tokenBody] = await Promise.all([ - readQuotaJson(subscriptionResponse), - readQuotaJson(tokenResponse), - ]); - if (subscriptionBody === QUOTA_JSON_READ_FAILURE || tokenBody === QUOTA_JSON_READ_FAILURE) return null; - const subscription = a6apiPayload(subscriptionBody); - const token = a6apiPayload(tokenBody); - const unlimited = token?.unlimited_quota === true - || token?.unlimited_quota === 1 - || token?.unlimited_quota === "true"; - const normalizedExpiry = normalizeResetAt(token?.expires_at); - const expiry = normalizedExpiry && normalizedExpiry > 0 - ? { expiresAt: normalizedExpiry } - : {}; - if (unlimited) { - // Every row is an API-credit constraint on inference, so the display quota is also - // the routing projection. Passing it explicitly is the opt-in. - const quota: ProviderQuota = { - creditsUsd: { - used: 0, - limit: 0, - remaining: 0, - percent: 0, - unlimited: true, - ...expiry, - }, - customWindows: [{ label: "Unlimited API credits", percent: 0 }], - updatedAt: Date.now(), - }; - return keyReport(provider, "a6api:billing", quota, config, apiKey, quota); - } - const limitUsd = firstFinite(subscription, ["hard_limit_usd"]); - const grantedUnits = firstFinite(token, ["total_granted"]); - const usedUnits = firstFinite(token, ["total_used"]); - const availableUnits = firstFinite(token, ["total_available"]); - const reconciledUnits = usedUnits !== undefined && availableUnits !== undefined - ? usedUnits + availableUnits - : undefined; - const reconciliationTolerance = grantedUnits !== undefined - ? Math.abs(grantedUnits) * 1e-9 - : 0; - if (limitUsd === undefined || grantedUnits === undefined || usedUnits === undefined - || availableUnits === undefined || limitUsd <= 0 || grantedUnits <= 0 - || usedUnits < 0 || availableUnits < 0 - || reconciledUnits === undefined - || Math.abs(reconciledUnits - grantedUnits) > reconciliationTolerance) return TERMINAL_QUOTA_FAILURE; - const usdPerUnit = limitUsd / grantedUnits; - const usedUsd = usedUnits * usdPerUnit; - const remainingUsd = Math.max(0, availableUnits * usdPerUnit); - const percent = normalizePercent((usedUsd / limitUsd) * 100); - if (percent === undefined) return TERMINAL_QUOTA_FAILURE; - const label = `API credits ($${remainingUsd.toFixed(2)} of $${limitUsd.toFixed(2)} remaining)`; - const quota: ProviderQuota = { - creditsUsd: { - used: usedUsd, - limit: limitUsd, - remaining: remainingUsd, - percent, - ...expiry, - }, - customWindows: [{ label, percent }], - updatedAt: Date.now(), - }; - // The credit balance funds inference itself, so display and routing scope agree. - return keyReport(provider, "a6api:billing", quota, config, apiKey, quota); -} - -function parseOpenCodeGoUsageWindow(value: unknown): { percent: number; resetAt?: number } | null { - const row = asRecord(value); - if (!row) return null; - const percent = normalizePercent(row.percent); - if (percent === undefined) return null; - const resetAt = normalizeResetAt(row.resetsAt); - return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; -} - -async function fetchOpenCodeGoQuota(provider: string, config: OcxProviderConfig): Promise { - // Never send a configured API key when the provider destination is not the built-in Go endpoint. - if (!isCanonicalOpenCodeGoBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(OPENCODE_GO_USAGE_URL, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const usage = asRecord(body?.usage); - if (!usage) return null; - const rolling = parseOpenCodeGoUsageWindow(usage.rolling); - const weekly = parseOpenCodeGoUsageWindow(usage.weekly); - const monthly = parseOpenCodeGoUsageWindow(usage.monthly); - const quota: ProviderQuota = { - ...(rolling ? { - fiveHourPercent: rolling.percent, - ...(rolling.resetAt !== undefined ? { fiveHourResetAt: rolling.resetAt } : {}), - } : {}), - ...(weekly ? { - weeklyPercent: weekly.percent, - ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), - } : {}), - ...(monthly ? { - monthlyPercent: monthly.percent, - ...(monthly.resetAt !== undefined ? { monthlyResetAt: monthly.resetAt } : {}), - } : {}), - updatedAt: Date.now(), - }; - return keyReport(provider, "opencode-go:usage", quota, config, apiKey, quota); -} - -/** - * OpenRouter `GET /api/v1/key` — the key's own credit balance and optional - * per-key spending cap. `limit` is the configured cap (absent = uncapped); - * `usage` is lifetime spend; `limit_remaining` is what is left of the cap. - * When no cap is set there is no hard limit to meter against, so no bar is - * produced — the provider falls back to its documented reference. - */ -async function fetchOpenRouterQuota(provider: string, config: OcxProviderConfig): Promise { - // Never send a configured API key to a lookalike host or through a redirect. - if (!isCanonicalOpenRouterBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${OPENROUTER_BASE_URL}/key`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - if (!data) return null; - const limit = toFiniteNumber(data.limit); - const limitRemaining = toFiniteNumber(data.limit_remaining); - const usage = toFiniteNumber(data.usage); - // A successful no-cap response is a DELIBERATE change, not a transient - // failure: the old capped row must be dropped, not preserved as last-good. - if (limit === undefined || limit <= 0) return TERMINAL_QUOTA_FAILURE; - // Prefer the authoritative remaining-cap value when present: `usage` is - // lifetime accumulated spend and overstates a reset or re-capped key. - const used = limitRemaining !== undefined - ? Math.max(0, limit - limitRemaining) - : usage !== undefined && usage >= 0 ? usage : undefined; - if (used === undefined) return null; - const percent = normalizePercent((used / limit) * 100); - if (percent === undefined) return null; - const remaining = Math.max(0, limit - used); - const label = `API credits ($${remaining.toFixed(2)} of $${limit.toFixed(2)} remaining)`; - // The per-key spending cap stops every request this credential can make, so the - // whole report is inference-wide routing evidence. - const quota: ProviderQuota = { - customWindows: [{ label, percent }], - updatedAt: Date.now(), - }; - return keyReport(provider, "openrouter:key-info", quota, config, apiKey, quota); -} - -/** - * DeepSeek `GET /user/balance` — the account's granted + topped-up credit - * balance. The payload places `total_balance` / `granted_balance` inside - * entries of `balance_infos` (one row per currency); the row for the account's - * currency is selected by preference. `granted_balance` is a CURRENT balance - * component, not the original grant ceiling, so no consumed percentage is - * fabricated — the balance is reported as a balance-only window. - */ -async function fetchDeepSeekQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalDeepSeekBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${DEEPSEEK_BASE_URL}/user/balance`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - // The payload nests balances under `balance_infos` rows keyed by currency; - // prefer a USD row, then CNY, then the first row that parses. - const infos = Array.isArray(body?.balance_infos) ? body.balance_infos as unknown[] : null; - const rows = infos - ? infos.map((raw): Record | null => asRecord(raw)).filter((r): r is Record => r !== null) - : []; - const pick = (currency: string): Record | null => - rows.find(row => String(row.currency ?? "").toUpperCase() === currency) ?? null; - const preferred = pick("USD") ?? pick("CNY") ?? rows[0] ?? null; - if (!preferred) return null; - const totalBalance = toFiniteNumber(preferred.total_balance); - const grantedBalance = toFiniteNumber(preferred.granted_balance); - const toppedUp = toFiniteNumber(preferred.topped_up_balance); - const balance = totalBalance ?? grantedBalance ?? toppedUp; - if (balance === undefined || balance < 0) return null; - const label = grantedBalance !== undefined && grantedBalance > 0 - ? `API balance ($${balance.toFixed(2)} total, $${grantedBalance.toFixed(2)} granted)` - : `API balance ($${balance.toFixed(2)})`; - return report(provider, "deepseek:balance", { - customWindows: [{ label, percent: 0 }], - updatedAt: Date.now(), - }); -} - -/** - * ClinePass `GET /api/v1/users/me/plan/usage-limits` — the subscription's - * rolling five-hour, weekly, and monthly utilization, matching the existing - * ProviderQuota windows directly. The endpoint 404s (or returns a null plan) - * for accounts without an active ClinePass, which is a no-report, not an error. - */ -async function fetchClineQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalClineBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${CLINE_BASE_URL}/api/v1/users/me/plan/usage-limits`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - // 404 = no active plan; a plain "no plan" is a no-report, everything else - // 4xx (except 408/429) is a credential/contract problem. - if (response.status === 404) return null; - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - const limits = Array.isArray(data?.limits) ? data.limits : null; - if (!limits) return null; - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - for (const raw of limits) { - const row = asRecord(raw); - if (!row) continue; - const percent = normalizePercent(row.percentUsed); - if (percent === undefined) continue; - const resetAt = normalizeResetAt(row.resetsAt); - if (row.type === "five_hour") { - quota.fiveHourPercent = percent; - if (resetAt !== undefined) quota.fiveHourResetAt = resetAt; - windows += 1; - } else if (row.type === "weekly") { - quota.weeklyPercent = percent; - if (resetAt !== undefined) quota.weeklyResetAt = resetAt; - windows += 1; - } else if (row.type === "monthly") { - quota.monthlyPercent = percent; - if (resetAt !== undefined) quota.monthlyResetAt = resetAt; - windows += 1; - } - } - return windows > 0 ? keyReport(provider, "cline:plan-usage-limits", quota, config, apiKey, quota) : null; -} - -/** - * Ollama Cloud `GET https://ollama.com/api/usage` — returns account usage. - * Legacy plans report rolling 5-hour `limits.session.usage` and 7-day - * `limits.weekly.usage`. Migrated monthly-credit plans report - * `limits.monthly.usage`. `usage` values are normalized fractions (0..1). - */ -function parseOllamaPercent(usageValue: unknown): number | undefined { - const usage = toFiniteNumber(usageValue); - if (usage === undefined || usage < 0) return undefined; - const percent = Math.round(usage * 10000) / 100; - return normalizePercent(percent); -} - -export function parseOllamaCloudQuota(body: Record | null): ProviderQuota | null { - if (!body) return null; - const limits = asRecord(body.limits); - if (!limits) return null; - - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - - const session = asRecord(limits.session); - if (session) { - const percent = parseOllamaPercent(session.usage); - if (percent !== undefined) { - quota.fiveHourPercent = percent; - windows += 1; - } - } - - const weekly = asRecord(limits.weekly); - if (weekly) { - const percent = parseOllamaPercent(weekly.usage); - if (percent !== undefined) { - quota.weeklyPercent = percent; - windows += 1; - } - } - - const monthly = asRecord(limits.monthly); - if (monthly) { - const percent = parseOllamaPercent(monthly.usage); - if (percent !== undefined) { - quota.monthlyPercent = percent; - windows += 1; - } - } - - return windows > 0 ? quota : null; -} - -async function fetchOllamaCloudQuota(provider: string, config: OcxProviderConfig): Promise { - const effectiveBaseUrl = config.baseUrl ?? getProviderRegistryEntry(provider)?.baseUrl ?? ""; - if (!isCanonicalOllamaCloudBaseUrl(effectiveBaseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(OLLAMA_CLOUD_USAGE_URL, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - if (response.status === 404) return null; - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const quota = parseOllamaCloudQuota(body); - return quota ? keyReport(provider, "ollama-cloud:usage", quota, config, apiKey, quota) : null; -} - -/** - * Z.AI GLM Coding Plan `GET /api/monitor/usage/quota/limit` — the coding-plan - * limits arrive as a `limits` array of `TOKENS_LIMIT` (newer plans call the - * same rows `CREDIT_LIMIT`) and `TIME_LIMIT` rows. `TOKENS_LIMIT`/`CREDIT_LIMIT` - * rows carry the window length as `unit`/`number`: unit 3 is hours (number 5 → - * the rolling five-hour window), unit 6 is weeks (number 1 → the weekly - * window). Every row's `percentage` is the consumed share (falling - * back to `currentValue`/`usage` when absent) and `nextResetTime` (unix ms) - * the window reset. - * - * `TIME_LIMIT` rows are deliberately ignored (issue #1168). They are the shared - * monthly MCP *call* allowance for Web Search / Web Reader / Zread — not a - * model-token budget — and `ProviderQuota.monthlyPercent` is consumed as a - * model-capacity signal: `headroomOf()` in `src/oauth/account-quota-rank.ts` - * takes the MAX across every window, so a user who spent their MCP search - * allowance would be ranked as having no model capacity left, and the dashboard - * would draw a full monthly bar for a plan whose model tokens are untouched. - * A payload carrying only `TIME_LIMIT` rows therefore reports no quota at all, - * which is the honest answer rather than a fabricated one. - */ -export function parseZaiQuotaLimits(data: Record | null): ProviderQuota | null { - const limits = Array.isArray(data?.limits) ? data.limits as unknown[] : null; - if (!limits) return null; - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - for (const raw of limits) { - const row = asRecord(raw); - if (!row) continue; - // Gate on row type before deriving a percentage: an MCP row must not even - // contribute a parsed value to a model-quota report. - if (row.type !== "TOKENS_LIMIT" && row.type !== "CREDIT_LIMIT") continue; - const resetAt = normalizeResetAt(row.nextResetTime); - let percent = normalizePercent(row.percentage); - if (percent === undefined) { - const used = toFiniteNumber(row.currentValue); - const total = toFiniteNumber(row.usage); - if (used !== undefined && total !== undefined && total > 0) { - percent = normalizePercent((used / total) * 100); - } - } - if (percent === undefined) continue; - const unit = toFiniteNumber(row.unit); - const number = toFiniteNumber(row.number); - if (unit === 3 && number === 5) { - quota.fiveHourPercent = percent; - if (resetAt !== undefined) quota.fiveHourResetAt = resetAt; - windows += 1; - } else if (unit === 6 && number === 1) { - quota.weeklyPercent = percent; - if (resetAt !== undefined) quota.weeklyResetAt = resetAt; - windows += 1; - } - } - return windows > 0 ? quota : null; -} - -/** - * Legacy Z.AI payload shape: percent fields with window identifiers directly on - * the data object (optionally nested under `quota`). Kept as a fallback so - * older responses keep rendering when the `limits` array is absent. - */ -function parseZaiQuotaLegacyFields(data: Record | null): ProviderQuota | null { - if (!data) return null; - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - const percentAt = (key: string): number | undefined => { - const value = normalizePercent(data[key]); - if (value !== undefined) return value; - const nested = asRecord(data.quota); - return nested ? normalizePercent(nested[key]) : undefined; - }; - const fiveHour = percentAt("fiveHourPercent") ?? percentAt("fiveHourUsage") ?? percentAt("fiveHourUsed"); - const weekly = percentAt("weeklyPercent") ?? percentAt("weeklyUsage") ?? percentAt("weeklyUsed"); - const monthly = percentAt("monthlyPercent") ?? percentAt("mcpPercent") ?? percentAt("monthlyMCPUsage"); - if (fiveHour !== undefined) { - quota.fiveHourPercent = fiveHour; - windows += 1; - } - if (weekly !== undefined) { - quota.weeklyPercent = weekly; - windows += 1; - } - if (monthly !== undefined) { - quota.monthlyPercent = monthly; - windows += 1; - } - return windows > 0 ? quota : null; -} - -/** - * Fetches the Z.AI GLM Coding Plan quota — on whichever region the provider - * points at (api.z.ai or open.bigmodel.cn). The `limits` array shape is - * preferred; older field-name payloads fall back to the legacy parser. - * - * Authentication differs by host (issue #1168). `api.z.ai` takes the API key as - * a Bearer token per Z.AI's API reference; `open.bigmodel.cn` expects the key - * directly in `Authorization` with no scheme prefix and answers a Bearer header - * with an auth error, which is why BigModel Coding Plan quota never rendered. - * The host is already canonicalized by `isCanonicalZaiBaseUrl` above and - * `redirect: "error"` stays set, so the bare key cannot travel to a lookalike - * host or follow a redirect off-origin. - */ -async function fetchZaiQuota(provider: string, config: OcxProviderConfig): Promise { - const monitorHost = zaiQuotaMonitorHost(config.baseUrl); - if (!monitorHost) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const authorization = monitorHost === ZAI_CN_BASE_URL ? apiKey : `Bearer ${apiKey}`; - const response = await fetch(`${monitorHost}/api/monitor/usage/quota/limit`, { - headers: { Accept: "application/json", Authorization: authorization }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - if (!body || body.success === false) return null; - const data = asRecord(body.data) ?? body; - if (Array.isArray(data?.limits)) { - const quota = parseZaiQuotaLimits(data); - // A well-formed `limits[]` we fully understood is authoritative even when it yields no - // model window — for example a plan reporting only the monthly MCP `TIME_LIMIT` row. - // Returning `null` here would preserve the previous token windows for up to 30 minutes - // and keep quota-aware routing acting on a report the provider has already superseded. - return quota - ? keyReport(provider, "zai:quota-limit", quota, config, apiKey, quota) - : AUTHORITATIVE_EMPTY_QUOTA; - } - const legacy = parseZaiQuotaLegacyFields(data); - if (!legacy) return null; - // The legacy monthly figure also carries MCP usage; it is display evidence, not - // proof that model inference is unavailable. Modern TOKEN_LIMIT rows above are scoped. - const inferenceQuota = { ...legacy }; - delete inferenceQuota.monthlyPercent; - delete inferenceQuota.monthlyResetAt; - return keyReport(provider, "zai:quota-limit", legacy, config, apiKey, inferenceQuota); -} - -/** - * MiniMax Token Plan `GET /v1/token_plan/remains` — the subscription's - * remaining quota as a countdown-time value (ms). The endpoint does not expose - * the plan's total duration, so no percentage is fabricated from a presumed - * window: the remaining time is reported as a duration-only window. When the - * API supplies a total (`total_time` / `plan_duration_ms`), a consumed share - * is derived from it. Region selects the host: `minimax` → www.minimax.io, - * `minimax-cn` → api.minimaxi.com. - */ -async function fetchMinimaxQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalMinimaxBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const cnHost = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.minimaxi.com"); - const remainsUrl = cnHost ? "https://api.minimaxi.com/v1/token_plan/remains" : MINIMAX_REMAINS_URL; - const response = await fetch(remainsUrl, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - if (!body || body.success === false) return null; - const data = asRecord(body.data) ?? body; - const remainsMs = toFiniteNumber(data.remains_time ?? data.remainsTime); - if (remainsMs === undefined || remainsMs < 0) return null; - const hours = Math.floor(remainsMs / 3_600_000); - const label = `Token Plan remaining (${hours}h)`; - // Only derive a consumed share when the API actually reports the plan total; - // a presumed window (e.g. 30 days) would fabricate utilization. A valid - // response that omits the total after a prior refresh had it is a DELIBERATE - // contract change — the old row must be dropped (terminal), not preserved as - // a transient last-good. - const totalMs = toFiniteNumber(data.total_time ?? data.plan_duration_ms ?? data.total_duration_ms); - if (totalMs === undefined || totalMs <= 0) return TERMINAL_QUOTA_FAILURE; - const consumed = Math.max(0, totalMs - remainsMs); - const percent = normalizePercent((consumed / totalMs) * 100); - if (percent === undefined) return null; - return report(provider, "minimax:token-plan-remains", { - customWindows: [{ label, percent }], - updatedAt: Date.now(), - }); -} - -/** - * Moonshot/Kimi `GET /v1/users/me/balance` — the account's available balance - * (voucher + cash). Renders a single balance window against the sum of - * voucher + cash when positive (there is no per-window rate limit to meter). - */ -async function fetchMoonshotQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalMoonshotBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const host = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.moonshot.cn") ? "https://api.moonshot.cn/v1" : MOONSHOT_BASE_URL; - const response = await fetch(`${host}/users/me/balance`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - if (!data) return null; - const available = toFiniteNumber(data.available_balance); - const voucher = toFiniteNumber(data.voucher_balance); - const cash = toFiniteNumber(data.cash_balance); - if (available === undefined || available < 0) return null; - // Moonshot exposes no per-window quota ceiling, only a balance — report it - // as a balance-only window (percent 0) rather than a fabricated utilization. - // Currency is host-scoped: China platform (api.moonshot.cn) bills in CNY; - // the international platform (api.moonshot.ai) bills in USD. Do not force - // either side into the other unit — the number is correct, only the unit - // must match the host. - const isChinaHost = host.startsWith("https://api.moonshot.cn"); - const money = (n: number) => isChinaHost ? `¥${n.toFixed(2)}` : `$${n.toFixed(2)}`; - const unit = isChinaHost ? "CNY" : "USD"; - const label = voucher !== undefined && cash !== undefined - ? `Balance (${money(available)} ${unit} available, ${money(voucher)} voucher)` - : `Balance (${money(available)} ${unit} available)`; - return report(provider, "moonshot:balance", { - customWindows: [{ label, percent: 0 }], - updatedAt: Date.now(), - }); -} - -/** - * Venice `GET /api/v1/billing/balance` — DIEM (native credits) or USD balance. - * Shows the remaining balance; epoch allocation progress when present. - */ -async function fetchVeniceQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalVeniceBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${VENICE_BASE_URL}/billing/balance`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - if (!data) return null; - const diemBalance = toFiniteNumber(data.balance); - const usdBalance = toFiniteNumber(data.balance_usd); - const epochUsed = toFiniteNumber(data.diem_epoch_used); - const epochAllocated = toFiniteNumber(data.diem_epoch_allocated); - if (diemBalance === undefined && usdBalance === undefined) return null; - const label = diemBalance !== undefined - ? `DIEM balance (${Math.round(diemBalance)})` - : `USD balance ($${usdBalance?.toFixed(2) ?? "?"})`; - if (epochAllocated !== undefined && epochAllocated > 0 && epochUsed !== undefined) { - const percent = normalizePercent((epochUsed / epochAllocated) * 100); - if (percent === undefined) return null; - return report(provider, "venice:billing-balance", { - customWindows: [{ label, percent }], - updatedAt: Date.now(), - }); - } - return report(provider, "venice:billing-balance", { - customWindows: [{ label, percent: 0 }], - updatedAt: Date.now(), - }); -} - -/** - * Synthetic `GET /v2/quotas` — the known quota lanes (rolling 5-hour, - * weekly token, search-hourly) mapped onto the quota windows. - */ -async function fetchSyntheticQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalSyntheticBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${SYNTHETIC_BASE_URL}/quotas`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - const percentAt = (key: string): number | undefined => { - const value = normalizePercent(data?.[key]); - if (value !== undefined) return value; - const nested = asRecord(data?.quota) ?? asRecord(data?.quotas); - return nested ? normalizePercent(nested[key]) : undefined; - }; - const fiveHour = percentAt("rollingFiveHourLimit"); - const weekly = percentAt("weeklyTokenLimit"); - if (fiveHour !== undefined) { - quota.fiveHourPercent = fiveHour; - windows += 1; - } - if (weekly !== undefined) { - quota.weeklyPercent = weekly; - windows += 1; - } - const search = asRecord(data?.search); - const searchHourly = search ? normalizePercent(search.hourly) : undefined; - if (searchHourly !== undefined) { - quota.customWindows = [...(quota.customWindows ?? []), { label: "Search hourly", percent: searchHourly }]; - windows += 1; - } - const inferenceQuota = { ...quota }; - delete inferenceQuota.customWindows; // search.hourly does not constrain model inference. - return windows > 0 ? keyReport(provider, "synthetic:quotas", quota, config, apiKey, inferenceQuota) : null; -} - -/** - * DeepInfra `GET /payment/checklist?compute_owed=true` — prepaid balance, - * recent spend, spending limit, and suspension state. Renders a balance - * window (prepaid funds are a negative `stripe_balance` → positive available). - */ -async function fetchDeepInfraQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalDeepInfraBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${DEEPINFRA_BASE_URL}/payment/checklist?compute_owed=true`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - if (!data) return null; - const stripeBalance = toFiniteNumber(data.stripe_balance); - const spendLimit = toFiniteNumber(data.spending_limit); - const total = toFiniteNumber(data.total_amount_due); - if (stripeBalance === undefined) return null; - // Prepaid funds are negative; a positive value is money owed. - const available = stripeBalance < 0 ? -stripeBalance : 0; - if (spendLimit !== undefined && spendLimit > 0) { - const spent = total !== undefined && total > 0 ? total : Math.max(0, spendLimit - available); - const percent = normalizePercent((spent / spendLimit) * 100); - if (percent === undefined) return null; - return report(provider, "deepinfra:billing-checklist", { - customWindows: [{ label: `Billing cycle spend ($${spent.toFixed(2)} of $${spendLimit.toFixed(2)})`, percent }], - updatedAt: Date.now(), - }); - } - return report(provider, "deepinfra:billing-checklist", { - customWindows: [{ label: `Prepaid balance ($${available.toFixed(2)})`, percent: 0 }], - updatedAt: Date.now(), - }); -} - -/** - * Neuralwatt `GET /v1/quota` — subscription kWh usage (primary window) and - * prepaid USD credit balance (secondary). - */ -async function fetchNeuralwattQuota(provider: string, config: OcxProviderConfig): Promise { - if (!isCanonicalNeuralwattBaseUrl(config.baseUrl)) return null; - const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); - if (!apiKey) return null; - const response = await fetch(`${NEURALWATT_BASE_URL}/quota`, { - headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const body = asRecord(await readQuotaJson(response)); - const data = asRecord(body?.data) ?? body; - const quota: ProviderQuota = { updatedAt: Date.now() }; - let windows = 0; - const subscription = asRecord(data?.subscription); - const kwhUsed = subscription ? toFiniteNumber(subscription.kwh_used) : undefined; - const kwhIncluded = subscription ? toFiniteNumber(subscription.kwh_included) : undefined; - if (kwhUsed !== undefined && kwhIncluded !== undefined && kwhIncluded > 0) { - const percent = normalizePercent((kwhUsed / kwhIncluded) * 100); - if (percent !== undefined) { - quota.fiveHourPercent = percent; - const periodEnd = subscription ? normalizeResetAt(subscription.current_period_end) : undefined; - if (periodEnd !== undefined) quota.fiveHourResetAt = periodEnd; - windows += 1; - } - } - const balance = asRecord(data?.balance); - const totalCredits = balance ? toFiniteNumber(balance.total_credits_usd) : undefined; - const remainingCredits = balance ? toFiniteNumber(balance.credits_remaining_usd) : undefined; - if (totalCredits !== undefined && totalCredits > 0 && remainingCredits !== undefined) { - // Utilization is CONSUMED credits, not the remaining share. - const used = Math.max(0, totalCredits - remainingCredits); - const percent = normalizePercent((used / totalCredits) * 100); - if (percent !== undefined) { - quota.customWindows = [...(quota.customWindows ?? []), { label: "Prepaid credits", percent }]; - windows += 1; - } - } - return windows > 0 ? report(provider, "neuralwatt:quota", quota) : null; -} - -function report( - provider: string, - source: string, - quota: ProviderQuota, - aggregation?: CodexCapacityAggregation, -): ProviderQuotaReport | null { - if (!hasQuotaRows(quota)) return null; - return { - provider, - label: providerLabel(provider), - source, - quota, - updatedAt: quota.updatedAt, - ...(aggregation ? { aggregation } : {}), - }; -} - -/** - * Publish a credential-bound report, and routing evidence only when the producer - * hands over its inference-only projection. - * - * The projection is deliberately not defaulted to the display quota. A producer must - * decide that its rows really do constrain inference on the probed credential; omitting - * the argument leaves the report display-only, so a new producer cannot inherit - * provider-veto authority merely by calling this helper. Ownership alone is not the - * scope decision: providerQuotaRoutingBinding resolving is necessary, never sufficient. - */ -function keyReport( - provider: string, - source: string, - quota: ProviderQuota, - config: OcxProviderConfig, - probedCredential: string, - inferenceQuota?: ProviderQuota, -): ProviderQuotaReport | null { - const result = report(provider, source, quota); - if (!result || !inferenceQuota) return result; - const binding = providerQuotaRoutingBinding(provider, config, probedCredential); - if (binding) routingEvidence.set(result, { quota: inferenceQuota, binding }); - return result; -} - -function tagNativeMainReport( - value: ProviderQuotaReport | null, - generation: number, -): ProviderQuotaReport | null { - if (value) nativeMainReportGenerations.set(value, generation); - return value; -} - -/** - * Test-only seam: publish exactly as a credential-bound producer does, and hand back the - * routing evidence the publication actually attached. - * - * Live producers all pass a projection today, so no probe fixture can prove the OTHER half - * of the contract: that omitting it stays display-only. Routing an omitted argument through - * the real helper keeps that provable, and a re-introduced `= quota` default would be - * observed here (a defaulted parameter also fires for an explicitly undefined argument). - */ -export function publishKeyReportForTests( - provider: string, - source: string, - quota: ProviderQuota, - config: OcxProviderConfig, - probedCredential: string, - inferenceQuota?: ProviderQuota, -): { report: ProviderQuotaReport | null; routing: ProviderQuotaRoutingEvidence | undefined } { - const result = keyReport(provider, source, quota, config, probedCredential, inferenceQuota); - return { report: result, routing: result ? routingEvidence.get(result) : undefined }; -} - -function isProviderQuotaReportCurrent(value: ProviderQuotaReport): boolean { - const generation = nativeMainReportGenerations.get(value); - return (generation === undefined || isMainAccountIdentityGenerationLive(generation)) - && (accountReportCurrent.get(value)?.() ?? true); -} - -async function fetchChatGptForwardQuota( - config: OcxConfig, - provider: string, - providerConfig: OcxProviderConfig, - forceRefresh: boolean, - prefetchedSnapshot?: CodexAuthAccountsSnapshotPromise, -): Promise { - if (providerCodexAccountMode(provider, providerConfig) === "direct") { - const snapshot = await fetchMainAccountInfoSnapshot(forceRefresh); - const quota = providerQuotaFromCodexQuota(snapshot.info.quota); - if (quota) quota.updatedAt = Date.now(); - return quota - ? tagNativeMainReport(report(provider, "chatgpt:wham", quota), snapshot.mainIdentityGeneration) - : null; - } - const snapshot = await (prefetchedSnapshot ?? listCodexAuthAccountsSnapshot(config, forceRefresh)); - const accounts = snapshot.accounts; - const activeId = effectiveCodexAuthAccountId(config); - const capacityAccounts = accounts.map(account => ({ - ...account, - active: account.id === activeId, - quota: providerQuotaFromCodexQuota(account.quota), - })); - const active = capacityAccounts.find(account => account.active) - ?? capacityAccounts.find(account => account.id === MAIN_CODEX_ACCOUNT_ID) - ?? capacityAccounts[0]; - const now = Date.now(); - const capacity = aggregateCodexPoolCapacity(capacityAccounts, now); - if (capacity.aggregation && capacity.quota) { - return tagNativeMainReport( - report( - provider, - "chatgpt:wham", - capacity.quota as ProviderQuota, - publicCapacityAggregation(capacity.aggregation, "aggregate"), - ), - snapshot.mainIdentityGeneration, - ); - } - const activeUsable = !!active && !active.paused && active.needsReauth !== true; - const quota = activeUsable && active?.quota - ? { ...active.quota, updatedAt: active.quota.updatedAt ?? Date.now() } as CodexCapacityQuota - : null; - const quotaFresh = !!quota - && Number.isFinite(quota.updatedAt) - && now - quota.updatedAt < CODEX_CAPACITY_MAX_QUOTA_AGE_MS; - if (quota && quotaFresh) { - const fallback = report( - provider, - "chatgpt:wham", - quota as ProviderQuota, - capacity.aggregation - ? publicCapacityAggregation(capacity.aggregation, "effective-account-fallback") - : undefined, - ); - return tagNativeMainReport(fallback, snapshot.mainIdentityGeneration); - } - if (capacity.aggregation) { - const updatedAt = Date.now(); - return tagNativeMainReport( - { - provider, - label: providerLabel(provider), - source: "chatgpt:wham", - quota: { updatedAt }, - updatedAt, - aggregation: publicCapacityAggregation(capacity.aggregation, "coverage-only"), - }, - snapshot.mainIdentityGeneration, - ); - } - return null; -} - -function centsValue(value: unknown): number | undefined { - const rec = asRecord(value); - return rec ? toFiniteNumber(rec.val) : undefined; -} - -/** Decode JWT payload `sub` for xAI weekly credits when the stored credential lacks accountId. */ -function xaiUserIdFromAccessToken(accessToken: string): string | undefined { - const parts = accessToken.split("."); - if (parts.length < 2 || !parts[1]) return undefined; - try { - const payload = JSON.parse(Buffer.from(parts[1], "base64url").toString("utf8")) as { sub?: unknown }; - return typeof payload.sub === "string" && payload.sub.trim() ? payload.sub.trim() : undefined; - } catch { - return undefined; - } -} - -/** - * Grok Build weekly credits envelope: - * `{ config: { creditUsagePercent?, currentPeriod: { type: "USAGE_PERIOD_TYPE_WEEKLY", end } } }`. - * Omitted percent is treated as 0 (proto3 default). - */ -export function parseXaiCreditsResponse(value: unknown): { percent: number; resetAt?: number } | null { - const body = asRecord(value); - const config = asRecord(body?.config); - if (!config) return null; - const period = asRecord(config.currentPeriod); - if (!period || period.type !== "USAGE_PERIOD_TYPE_WEEKLY") return null; - let percent = 0; - if (config.creditUsagePercent !== undefined) { - const normalized = normalizePercent(config.creditUsagePercent); - if (normalized === undefined) return null; - percent = normalized; - } - const resetAt = normalizeResetAt(period.end); - return { - percent, - ...(resetAt !== undefined ? { resetAt } : {}), - }; -} - -async function fetchXaiWeeklyCredits(accessToken: string, userId: string): Promise { - try { - const response = await fetch(XAI_CREDITS_URL, { - redirect: "error", - headers: { - Accept: "application/json", - Authorization: `Bearer ${accessToken}`, - [XAI_GROK_COMPATIBILITY.headers.tokenAuth]: "xai-grok-cli", - [XAI_GROK_COMPATIBILITY.headers.authenticateResponse]: "authenticate-response", - "x-userid": userId, - [XAI_GROK_COMPATIBILITY.headers.clientVersion]: XAI_GROK_CLIENT_VERSION, - }, - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - const parsed = parseXaiCreditsResponse(await readQuotaJson(response)); - if (!parsed) return null; - return { - weeklyPercent: parsed.percent, - ...(parsed.resetAt !== undefined ? { weeklyResetAt: parsed.resetAt } : {}), - updatedAt: Date.now(), - }; - } catch { - return null; - } -} - -async function fetchXaiQuota(provider: string, context: { accessToken: string; upstreamAccountId?: string }): Promise { - const { accessToken } = context; - - // Prefer the SuperGrok weekly credits window that actually gates prompting (#1283). - const userId = context.upstreamAccountId?.trim() || xaiUserIdFromAccessToken(accessToken); - if (userId) { - const weekly = await fetchXaiWeeklyCredits(accessToken, userId); - if (weekly) return report(provider, "xai:grok-billing-credits", weekly); - } - - // Legacy monthly dollar pool — retained when weekly is unavailable. - try { - const response = await fetch(XAI_BILLING_URL, { - redirect: "error", - headers: { Accept: "application/json", Authorization: `Bearer ${accessToken}` }, - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - const body = asRecord(await readQuotaJson(response)); - const config = asRecord(body?.config); - if (!config) return null; - const limitCents = centsValue(config.monthlyLimit); - const usedCents = centsValue(config.used); - if (limitCents === undefined || usedCents === undefined || limitCents <= 0) return null; - const percent = normalizePercent((usedCents / limitCents) * 100); - if (percent === undefined) return null; - return report(provider, "xai:grok-billing", { - monthlyPercent: percent, - monthlyResetAt: normalizeResetAt(config.billingPeriodEnd), - updatedAt: Date.now(), - }); - } catch { - return null; - } -} - -function parseClaudeBucket(value: unknown): { percent?: number; resetAt?: number } | null { - const rec = asRecord(value); - if (!rec) return null; - const percent = normalizePercent(rec.utilization); - const resetAt = normalizeResetAt(rec.resets_at); - if (percent === undefined && resetAt === undefined) return null; - return { percent, resetAt }; -} - -function parseClaudeLimit(value: unknown): { label: string; percent: number; resetAt?: number } | null { - const rec = asRecord(value); - if (!rec) return null; - const percent = normalizePercent(rec.percent); - if (percent === undefined) return null; - const scope = asRecord(rec.scope); - const model = asRecord(scope?.model); - const rawLabel = String(model?.display_name ?? "").trim(); - if (!rawLabel) return null; - const lowerLabel = rawLabel.toLowerCase(); - const label = lowerLabel.includes("fable") ? "Fable" - : lowerLabel.includes("opus") ? "Opus" - : lowerLabel.includes("sonnet") ? "Sonnet" - : rawLabel; - const resetAt = normalizeResetAt(rec.resets_at); - return { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }; -} - -/** Claude's OAuth usage endpoint, probed with ONE account's own bearer token. */ -const anthropicUsageInflight = new Map>(); - -/** - * Anthropic per-credential usage. - * - * This endpoint reports quota only. Its body carries `five_hour`, `seven_day`, the - * model-scoped weekly buckets (`seven_day_fable`/`_opus`/`_sonnet`) and a `limits` array, - * and **no subscription or tier field** — nor does the OAuth token response, which yields only - * `account.uuid` and `account.email_address` (`src/oauth/anthropic.ts`). That is why - * `OAuthAccountSummary.plan` is `null` for Anthropic rather than populated here (#3777); it is - * a missing upstream field, not an unfinished mapping. - * - * A tier must not be inferred from what is here. Percentages are normalized per account, so a - * Max x5 seat at 50% is byte-identical to a Max x20 seat at 50%, and the presence of a - * model-scoped window tracks entitlement rather than seat size. Populate `plan` only when - * upstream returns the tier itself. - */ -async function fetchAnthropicUsageQuota(accessToken: string): Promise { - const joinable = anthropicUsageInflight.get(accessToken); - if (joinable) return joinable; - - const probe = (async (): Promise => { - const response = await fetch("https://api.anthropic.com/api/oauth/usage", { - headers: { - Accept: "application/json, text/plain, */*", - "Content-Type": "application/json", - "User-Agent": "claude-cli/2.1.63 (external, cli)", - "anthropic-beta": "claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14,context-management-2025-06-27,prompt-caching-scope-2026-01-05", - Authorization: `Bearer ${accessToken}`, - }, - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - const body = asRecord(await readQuotaJson(response)); - if (!body) return null; - const fiveHour = parseClaudeBucket(body.five_hour); - const sevenDay = parseClaudeBucket(body.seven_day); - const fable = parseClaudeBucket(body.seven_day_fable); - const opus = parseClaudeBucket(body.seven_day_opus); - const sonnet = parseClaudeBucket(body.seven_day_sonnet); - const customWindows: ProviderQuotaWindow[] = []; - if (fable?.percent !== undefined) customWindows.push({ label: "Fable", percent: fable.percent, ...(fable.resetAt !== undefined ? { resetAt: fable.resetAt } : {}) }); - if (opus?.percent !== undefined) customWindows.push({ label: "Opus", percent: opus.percent, ...(opus.resetAt !== undefined ? { resetAt: opus.resetAt } : {}) }); - if (sonnet?.percent !== undefined) customWindows.push({ label: "Sonnet", percent: sonnet.percent, ...(sonnet.resetAt !== undefined ? { resetAt: sonnet.resetAt } : {}) }); - const knownLabels = new Set(customWindows.map(window => window.label.toLowerCase())); - const limits = Array.isArray(body.limits) ? body.limits : []; - for (const rawLimit of limits) { - const limitRecord = asRecord(rawLimit); - // `session` and `weekly_all` mirror the canonical five-hour and weekly - // buckets above; only model-scoped weekly limits add a third window. - if (String(limitRecord?.kind ?? "").trim().toLowerCase() !== "weekly_scoped") continue; - const limit = parseClaudeLimit(rawLimit); - if (!limit || knownLabels.has(limit.label.toLowerCase())) continue; - knownLabels.add(limit.label.toLowerCase()); - customWindows.push(limit); - } - const quota: ProviderQuota = { - // Claude's 5-hour window is a first-class rate limit, same as the Codex login 5h/weekly - // rows: report it in the canonical fields so the dashboard renders it with the standard - // "5-hour limit" label and ordering instead of as a generic extra window. - ...(fiveHour?.percent !== undefined ? { fiveHourPercent: fiveHour.percent } : {}), - ...(fiveHour?.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), - ...(sevenDay?.percent !== undefined ? { weeklyPercent: sevenDay.percent } : {}), - ...(sevenDay?.resetAt !== undefined ? { weeklyResetAt: sevenDay.resetAt } : {}), - ...(customWindows.length > 0 ? { customWindows } : {}), - updatedAt: Date.now(), - }; - // Empty / schema-changed payloads must not cache as "success with no bars". - return hasQuotaRows(quota) ? quota : null; - })().finally(() => { - if (anthropicUsageInflight.get(accessToken) === probe) anthropicUsageInflight.delete(accessToken); - }); - anthropicUsageInflight.set(accessToken, probe); - return probe; -} - -async function fetchAnthropicQuota(provider: string): Promise { - // Capture the account we intend to probe before awaiting — a mid-flight active - // switch must not seed the wrong account's cache with this response. - const probedAccountId = getAccountSet("anthropic")?.activeAccountId; - const probedAccountKey = probedAccountId ? accountCacheKey("anthropic", probedAccountId) : null; - const writerGeneration = captureConfigGeneration(); - let accessToken: string; - try { - accessToken = await getValidAccessToken("anthropic"); - } catch { - return null; - } - const quota = await fetchAnthropicUsageQuota(accessToken); - if (!quota) return null; - // Share the active-account probe with the per-account cache so Providers-page - // loads do not double-hit Anthropic's rate-limited usage endpoint. - if (probedAccountId && probedAccountKey) { - const stillOwnsToken = getAccountCredential("anthropic", probedAccountId)?.access === accessToken; - if (stillOwnsToken && mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { - accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota }); - } - } - return report(provider, "anthropic:oauth-usage", quota); -} - -/** - * Provider-level Kiro row: the active account's usage, shown on the Providers page. - * - * The per-account cache is seeded from the same probe so opening that page does not read - * the active account twice, and the account id is captured before the await so a - * concurrent account switch cannot file this answer under the wrong account. - */ -async function fetchKiroQuota(provider: string): Promise { - const probedAccountId = getAccountSet("kiro")?.activeAccountId; - if (!probedAccountId) return null; - const probedAccountKey = accountCacheKey("kiro", probedAccountId); - const writerGeneration = captureConfigGeneration(); - let snapshot: KiroUsageSnapshot | null; - try { - snapshot = await fetchKiroUsageSnapshot(await kiroUsageContextForAccount(probedAccountId)); - } catch { - return null; - } - if (!snapshot) return null; - if (mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { - accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota: snapshot.quota }); - commitKiroAccountUsageState(probedAccountKey, snapshot); - } - return report(provider, "kiro:usage-limits", snapshot.quota); -} - -/** - * Provider-level row probed from the key endpoint, for an account that CAN be probed. - * - * Written through the same account cache the passive path reads, so the measurement - * survives a restart and the per-account rows at oauth-account-routes.ts:313 pick it up - * with no mode change. Deliberately does not flip providerOAuthAccountQuotaMode: that - * mode selects readPassiveProviderAccountQuotas, and the probed per-account path it would - * switch to is gated on supportsPerAccountQuota, which has no meta-muse reader, so the - * GUI account list would go from showing observations to showing nothing. - */ -async function fetchMuseKeyQuota(provider: string): Promise { - const probedAccountId = getAccountSet(provider)?.activeAccountId; - if (!probedAccountId) return null; - const oauthAccessToken = getAccountCredential(provider, probedAccountId)?.muse?.oauthAccessToken; - // An imported or pasted credential has no account token and never will: it is - // capability, not provider id, that decides whether a probe is possible. - if (!oauthAccessToken) return null; - const probedAccountKey = accountCacheKey(provider, probedAccountId); - const writerGeneration = captureConfigGeneration(); - const quota = await fetchMuseKeyQuotaSnapshot(probedAccountId, oauthAccessToken); - if (!quota) return null; - if (mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { - // Hydrate before writing, for the same reason recordPassiveAccountQuota does: - // persistAccountQuotaCache serializes the whole in-memory map. - hydrateAccountQuotaCache(); - accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota }); - persistAccountQuotaCache(); - } - return report(provider, `${provider}:key-endpoint`, quota); -} -/** - * Provider-level row for a passive provider: the ACTIVE account's last observed - * subscription windows, the same shape `fetchAnthropicQuota` and `fetchKiroQuota` - * return. - * - * Cache-only. A dashboard load or `ocx account refresh` must never spend an inference - * turn, so `forceRefresh` does not exist on this path — there is nothing to refresh. - * `report.updatedAt` is the observation time, which is what both GUI surfaces render - * as the relative age of the row. - */ -async function fetchPassiveProviderQuota(provider: string): Promise { - const activeId = getAccountSet(provider)?.activeAccountId; - if (!activeId) return null; - // Idempotent; without it a proxy restart shows nothing until the next streaming turn - // even though the last observation is on disk. - hydrateAccountQuotaCache(); - const entry = accountQuotaCache.get(accountCacheKey(provider, activeId)); - if (!entry?.quota) return null; - const built = report(provider, `${provider}:subscription-observation`, entry.quota); - // Tagged here rather than inside report(), which every probed path shares. - return built ? { ...built, observed: true } : null; -} - -// --------------------------------------------------------------------------- -// Per-account quota (multiauth) -// --------------------------------------------------------------------------- - -/** - * Anthropic and Kiro both report usage per CREDENTIAL, so every logged-in account can be - * probed with its own bearer token — the active-account selection and the local usage log - * are irrelevant here. Mirrors the Codex pool behaviour - * (codex/auth-api.ts:fetchPoolAccountQuota), including a per-account TTL so N accounts cost - * at most N upstream calls per window. `ACCOUNT_QUOTA_TTL_MS` lives in `quota-wire.ts` - * because the Kiro exhaustion reader applies the same staleness bound. - */ -type AccountQuotaCacheEntry = { - ts: number; - quota: ProviderQuota | null; - /** Last probe failed (429 / network / expired login); still may hold last-good quota. */ - unavailable?: true; - quotaFailure?: QuotaFailureCode; - quotaFailureIsCurrent?: () => boolean; - /** Private new-reader identity; never persisted or serialized. */ - identity?: string; - isCurrent?: () => boolean; -}; -/** Expired measurements become unknown; missing reset evidence never implies a fresh allowance. */ -function normalizeAnthropicQuota(quota: ProviderQuota | null | undefined, now: number): ProviderQuota | null { - if (!quota) return null; - const validReset = (resetAt: unknown): resetAt is number => typeof resetAt === "number" - && Number.isFinite(resetAt) && resetAt > 0 && Number.isFinite(new Date(resetAt).getTime()); - let result = quota; - for (const [percent, reset] of [ - ["fiveHourPercent", "fiveHourResetAt"], - ["weeklyPercent", "weeklyResetAt"], - ["monthlyPercent", "monthlyResetAt"], - ] as const) { - const resetAt = quota[reset]; - if (resetAt === undefined) continue; - const valid = validReset(resetAt); - if (valid && resetAt > now) continue; - if (result === quota) result = { ...quota }; - if (valid) delete result[percent]; - delete result[reset]; - } - // Persisted rows validate only the outer quota object, so custom data may be malformed. - if (quota.customWindows !== undefined) { - const windows = Array.isArray(quota.customWindows) ? quota.customWindows : []; - const retained: ProviderQuotaWindow[] = []; - let changed = !Array.isArray(quota.customWindows); - for (const window of windows) { - if (!window || typeof window !== "object" || typeof window.label !== "string" || !window.label.trim() - || typeof window.percent !== "number" || !Number.isFinite(window.percent) - || window.percent < 0 || window.percent > 100) { - changed = true; - continue; - } - if (validReset(window.resetAt) && window.resetAt <= now) { - changed = true; - continue; - } - if (window.resetAt !== undefined && !validReset(window.resetAt)) { - const normalized = { ...window }; - delete normalized.resetAt; - retained.push(normalized); - changed = true; - } else { - retained.push(window); - } - } - if (changed) { - if (result === quota) result = { ...quota }; - if (retained.length) result.customWindows = retained; - else delete result.customWindows; - } - } - return hasQuotaRows(result) ? result : null; -} - -const accountQuotaCache = new Map(); -let explicitAccountEpoch = 0; - -/** - * Seed the cache from the last run, once. - * - * Without this a restart forgets every measurement, so the pool opens its next turn with - * no idea which account has room — the exact blindness pre-dispatch selection exists to - * remove. A hydrated row is still subject to the ordinary TTL, so it orders the first - * request and is replaced by a live probe immediately after. - */ -let diskHydrated = false; -function hydrateAccountQuotaCache(): void { - if (diskHydrated) return; - diskHydrated = true; - for (const [key, quota] of readPersistedAccountQuotas()) { - // Disk stores observation time, not the Anthropic usage probe's clock. - if (!accountQuotaCache.has(key)) { - const anthropic = key.startsWith("anthropic\u0000"); - accountQuotaCache.set(key, { - ts: anthropic ? 0 : quota.updatedAt, - quota: anthropic ? normalizeAnthropicQuota(quota, Date.now()) : quota, - }); - } - } -} - -function persistAccountQuotaCache(): void { - schedulePersistAccountQuotas(function* () { - const now = Date.now(); - for (const [key, entry] of accountQuotaCache) { - const quota = key.startsWith("anthropic\u0000") ? normalizeAnthropicQuota(entry.quota, now) : entry.quota; - if (quota) yield [key, quota] as [string, ProviderQuota]; - } - }); -} -const accountQuotaInflight = new Map>(); -let lastReconciledGeneration = 0; -let liveAccountQuotaKeys = new Set(); -let liveProviderQuotaKeys = new Set(); - -function mayCommitAccountQuotaKey(key: string, writerGeneration: number): boolean { - return writerGeneration >= lastReconciledGeneration || liveAccountQuotaKeys.has(key); -} - -function mayCommitProviderQuotaKey(key: string, writerGeneration: number): boolean { - return writerGeneration >= lastReconciledGeneration || liveProviderQuotaKeys.has(key); -} - -export interface ProviderAccountQuota { - accountId: string; - quota: ProviderQuota | null; - /** Set when the probe could not reach upstream (expired login, 429, network). */ - unavailable?: true; - quotaFailure?: QuotaFailureCode; - quotaFailureIsCurrent?: () => boolean; - isCurrent?: () => boolean; -} - -/** Providers whose per-account quota can be probed. Extend as other OAuth APIs are covered. */ -export function supportsPerAccountQuota(provider: string): boolean { - return provider === "anthropic" || provider === "kiro" || provider === "google-antigravity" - || explicitAccountReader(provider); -} - -function explicitAccountReader(provider: string): boolean { - return provider === "xai" || provider === "cursor" || provider === "kimi" || provider === "command-code"; -} - -export function providerOAuthAccountQuotaMode(provider: string): AccountQuotaMode { - return hasPassiveAccountQuota(provider) ? "passive" : supportsPerAccountQuota(provider) ? "probe" : "unsupported"; -} - -function accountCacheKey(provider: string, accountId: string): string { - return `${provider}\u0000${accountId}`; -} - -/** - * Synchronous last-good per-account quota read for routing. Never probes the network. - * Returns null when nothing is cached (or the cached row has no bars). - */ -export function getCachedProviderAccountQuota(provider: string, accountId: string): ProviderQuota | null { - const entry = accountQuotaCache.get(accountCacheKey(provider, accountId)); - if (entry?.isCurrent && !entry.isCurrent()) return null; - return provider === "anthropic" ? normalizeAnthropicQuota(entry?.quota, Date.now()) : entry?.quota ?? null; -} - -/** Test-only: seed or clear the per-account quota cache without probing upstream. */ -export function setCachedProviderAccountQuotaForTests( - provider: string, - accountId: string, - quota: ProviderQuota | null, -): void { - const key = accountCacheKey(provider, accountId); - if (quota === null) { - accountQuotaCache.delete(key); - return; - } - accountQuotaCache.set(key, { ts: Date.now(), quota }); -} - -/** Unified headers report utilization fractions and epoch-second reset times. */ -function anthropicHeaderResetAt(value: string | null): number | undefined { - const seconds = toFiniteNumber(value); - if (seconds === undefined || seconds <= 0) return undefined; - const timestamp = seconds * 1000; - return Number.isFinite(new Date(timestamp).getTime()) ? timestamp : undefined; -} - -export function parseAnthropicRateLimitHeaders(headers: Headers): ProviderQuota | null { - const fiveHourPercent = normalizeUtilizationFraction(headers.get("anthropic-ratelimit-unified-5h-utilization")); - const weeklyPercent = normalizeUtilizationFraction(headers.get("anthropic-ratelimit-unified-7d-utilization")); - if (fiveHourPercent === undefined && weeklyPercent === undefined) return null; - const fiveHourResetAt = anthropicHeaderResetAt(headers.get("anthropic-ratelimit-unified-5h-reset")); - const weeklyResetAt = anthropicHeaderResetAt(headers.get("anthropic-ratelimit-unified-7d-reset")); - return { - ...(fiveHourPercent !== undefined ? { fiveHourPercent } : {}), - ...(fiveHourPercent !== undefined && fiveHourResetAt !== undefined ? { fiveHourResetAt } : {}), - ...(weeklyPercent !== undefined ? { weeklyPercent } : {}), - ...(weeklyPercent !== undefined && weeklyResetAt !== undefined ? { weeklyResetAt } : {}), - updatedAt: Date.now(), - }; -} - -/** Reject unknown scales; round fraction conversion for persisted/displayed percentages. */ -function normalizeUtilizationFraction(value: string | null): number | undefined { - const numeric = toFiniteNumber(value); - if (numeric === undefined || numeric < 0 || numeric > 1) return undefined; - return Math.round(numeric * 10_000) / 100; -} - -/** - * Merge serving-account observations without advancing the usage probe's clock or - * erasing model-specific windows. The caller owns credential attribution; this guard - * prevents a retired account key from being revived by an older config generation. - */ -export function recordAnthropicAccountQuotaFromHeaders( - accountId: string, - headers: Headers, - writerGeneration: number, -): void { - if (!accountId) return; - const observed = parseAnthropicRateLimitHeaders(headers); - if (!observed) return; - const key = accountCacheKey("anthropic", accountId); - if (!mayCommitAccountQuotaKey(key, writerGeneration)) return; - // Hydrate before writing, for the same reason `recordPassiveAccountQuota` does: this write - // arrives unprompted from the request path, and `persistAccountQuotaCache` serializes the - // whole map. Landing before any reader has hydrated would persist this single row and erase - // every other provider's saved row. - hydrateAccountQuotaCache(); - const previous = accountQuotaCache.get(key); - accountQuotaCache.set(key, { - ...previous, - // Headers do not prove that the last usage probe succeeded. - ts: previous?.ts ?? 0, - quota: normalizeAnthropicQuota({ - ...normalizeAnthropicQuota(previous?.quota, observed.updatedAt), ...observed, - }, observed.updatedAt), - }); - persistAccountQuotaCache(); -} - -/** - * Providers whose per-account quota is OBSERVED in-band, never probed. - * - * Deliberately separate from `supportsPerAccountQuota` rather than folded into it. That - * predicate gates explicit upstream readers. Meta publishes no quota endpoint, so it - * remains a cache-only observation even when every probe reader is account-scoped. - */ -export function hasPassiveAccountQuota(provider: string): boolean { - return provider === "meta-muse"; -} - -/** - * Record a quota observed in-band on a streaming turn. - * - * The CALLER captures `writerGeneration` when it resolves the serving credential, not - * this function at write time. A streaming turn is a long await, and a generation - * captured immediately before the write cannot see a config or account change that - * happened EARLIER in the same turn — which is exactly the case the fence exists for. - */ -export function recordPassiveAccountQuota( - provider: string, - accountId: string, - quota: ProviderQuota, - writerGeneration: number, -): void { - if (!hasPassiveAccountQuota(provider) || !accountId) return; - const key = accountCacheKey(provider, accountId); - if (!mayCommitAccountQuotaKey(key, writerGeneration)) return; - // Hydrate BEFORE writing, not only on the read path. `persistAccountQuotaCache` - // serializes the whole in-memory map, so a passive write that lands before anything - // has read the cache would persist this one row and erase every other provider's - // saved row -- and `diskHydrated` would then stop any later reader from recovering - // them. A probe writer cannot hit this because its own read hydrates first; an - // observation arrives unprompted, so it must hydrate itself. - hydrateAccountQuotaCache(); - accountQuotaCache.set(key, { ts: Date.now(), quota }); - // Persisted so a restart keeps the last observation: with no probe to re-establish it, - // a forgotten row stays forgotten until the user happens to run another streaming turn. - persistAccountQuotaCache(); - // sweepExpiredOnWrite is deliberately NOT called. Existing probe writers call it - // because they run on a poll; this runs on the request path, where a state sweep does - // not belong. Passive rows are still reclaimed by generation reconciliation - // (reconcileProviderAccountQuotaRows) and by the disk reader's age bound. -} - -/** - * Cache-only per-account rows for a passive provider. Never probes, never refreshes. - * - * An account with no observation is OMITTED rather than returned with `quota: null` and - * `unavailable`: that pair means "a probe was attempted and failed", and no probe was - * ever attempted here. A user who has not yet run a streaming turn simply has no - * measurement, which is not an error state. - */ -export function readPassiveProviderAccountQuotas(provider: string): ProviderAccountQuota[] { - if (!hasPassiveAccountQuota(provider)) return []; - // Idempotent, and otherwise only reached from probe paths a passive provider never - // enters — without it a restart shows nothing until the next streaming turn, even - // though the row is sitting on disk. - hydrateAccountQuotaCache(); - const set = getAccountSet(provider); - if (!set) return []; - const rows: ProviderAccountQuota[] = []; - for (const account of set.accounts) { - const entry = accountQuotaCache.get(accountCacheKey(provider, account.id)); - if (entry?.quota) rows.push({ accountId: account.id, quota: entry.quota }); - } - return rows; -} - -export function sweepExpiredProviderAccountQuotaRows(now = Date.now()): number { - let removed = 0; - for (const [key, entry] of accountQuotaCache) { - // Anthropic observations extend retention, never the usage probe's eligibility clock. - const retainedAt = key.startsWith("anthropic\u0000") - ? Math.max(entry.ts, entry.quota?.updatedAt ?? 0) - : entry.ts; - if (retainedAt + ACCOUNT_QUOTA_TTL_MS > now) continue; - accountQuotaCache.delete(key); - removed += 1; - } - return removed; -} - -export function reconcileProviderAccountQuotaRows(context: GenerationContext): number { - if (context.generation <= lastReconciledGeneration) return 0; - let removed = 0; - for (const key of accountQuotaCache.keys()) { - if (context.oauthAccountKeys.has(key)) continue; - accountQuotaCache.delete(key); - removed += 1; - } - // Kiro exhaustion rows are keyed identically, so they retire with their quota row; a - // verdict outliving its account would hand the replacement a cooldown it never earned. - removed += reconcileKiroAccountUsageState(context.oauthAccountKeys); - if (cache) { - const reports = cache.response.reports.filter(report => context.providerNames.has(report.provider)); - removed += cache.response.reports.length - reports.length; - cache = { ...cache, response: { ...cache.response, reports } }; - replaceCachedProviderQuotas(reports, routingEvidence); - } - liveAccountQuotaKeys = new Set(context.oauthAccountKeys); - liveProviderQuotaKeys = new Set(context.providerNames); - lastReconciledGeneration = context.generation; - return removed; -} - -/** Test-only reset so a direct reconcile call in one file cannot leak across files. */ -export function resetProviderQuotaReconcileStateForTests(): void { - lastReconciledGeneration = 0; - liveAccountQuotaKeys = new Set(); - liveProviderQuotaKeys = new Set(); -} - -/** Drop cached per-account rows (all, or just one provider's). */ -export function clearAccountQuotaCache(provider?: string): void { - explicitAccountEpoch += 1; - if (!provider) { - accountQuotaCache.clear(); - accountQuotaInflight.clear(); - clearKiroAccountUsageState(); - // A cleared cache must not be re-seeded from the file it was just cleared of, and any - // pending write of the old rows is abandoned. - diskHydrated = false; - cancelPendingAccountQuotaPersist(); - return; - } - const prefix = `${provider}\u0000`; - for (const key of [...accountQuotaCache.keys()]) { - if (key.startsWith(prefix)) accountQuotaCache.delete(key); - } - clearKiroAccountUsageState(prefix); - // Drop in-flight probes too so a late resolve cannot repopulate after logout/remove. - for (const key of [...accountQuotaInflight.keys()]) { - if (key.startsWith(prefix)) accountQuotaInflight.delete(key); - } - persistAccountQuotaCache(); -} - -/** - * Resolve a bearer for quota probing without silently adopting a newer global - * Claude CLI credential into a background multiauth slot. - * - * - Fresh stored access → use as-is (no refresh). - * - Active account with expired access → normal refresh path. - * - Background `local-cli` with expired access → fail closed (unavailable): - * `getValidAccessTokenForAccount` can persist a mismatched Claude CLI identity. - * - Background ordinary OAuth (`source !== "local-cli"`) → safe to refresh; - * Anthropic's lock only adopts disk credentials for `local-cli` rows. - */ -async function getTokenForAccountQuotaProbe(provider: string, accountId: string): Promise { - const stored = getAccountCredential(provider, accountId); - if (!stored) throw new Error("account credential missing"); - if (stored.expires > Date.now() + ACCOUNT_TOKEN_SKEW_MS) return stored.access; - const activeId = getAccountSet(provider)?.activeAccountId; - if (activeId !== accountId && stored.source === "local-cli") { - throw new Error("background local-cli token expired; skip CLI-adopting refresh for quota probe"); - } - return getValidAccessTokenForAccount(provider, accountId); -} - -function explicitQuotaConfig(provider: string, configured?: OcxProviderConfig): OcxProviderConfig | undefined { - if (configured) return configured; - const entry = getProviderRegistryEntry(provider); - return entry ? { adapter: entry.adapter, baseUrl: entry.baseUrl, authMode: "oauth" } : undefined; -} - -function explicitQuotaIdentity(provider: string, accountId: string, configured?: OcxProviderConfig): string | undefined { - const credential = getAccountCredential(provider, accountId); - const target = explicitQuotaConfig(provider, configured); - if (!credential || !target) return undefined; - return quotaCredentialIdentity(provider, accountId, credential, target); -} - -function quotaCredentialIdentity(provider: string, accountId: string, credential: NonNullable>, target: OcxProviderConfig): string { - return createHash("sha256").update(JSON.stringify([ - provider, accountId, credential.access, credential.refresh, credential.expires, - credential.accountId, credential.projectId, credential.source, - target.adapter, target.baseUrl, target.authMode, target.disabled === true, - ])).digest("hex"); -} - -function explicitQuotaDestination(provider: string, config: OcxProviderConfig): boolean { - if (config.disabled === true || config.authMode !== "oauth") return false; - if (provider === "kimi") return isCanonicalKimiCodeBaseUrl(config.baseUrl); - if (provider === "command-code") return isCanonicalCommandCodeBaseUrl(config.baseUrl); - // These readers use fixed canonical billing origins, never config.baseUrl. - return provider === "xai" || provider === "cursor"; -} - -async function readExplicitAccountQuota(provider: string, accountId: string, configured?: OcxProviderConfig): Promise<{ - result: ProviderQuotaProbeResult; - identity: string | undefined; - isCurrent: () => boolean; -} | null> { - const target = explicitQuotaConfig(provider, configured); - if (!target || !explicitQuotaDestination(provider, target)) return null; - const config = { ...target }; - const epoch = explicitAccountEpoch; - const accessToken = await getTokenForAccountQuotaProbe(provider, accountId); - const credential = getAccountCredential(provider, accountId); - if (!credential || credential.access !== accessToken) return null; - // Pair the post-renewal credential with the destination captured before renewal. - const identity = explicitQuotaIdentity(provider, accountId, config); - const isCurrent = () => epoch === explicitAccountEpoch - && identity === explicitQuotaIdentity(provider, accountId, configured); - if (!isCurrent()) return null; - let result: ProviderQuotaProbeResult; - switch (provider) { - case "xai": result = await fetchXaiQuota(provider, { accessToken, upstreamAccountId: credential.accountId }); break; - case "cursor": result = await fetchCursorQuota(provider, accessToken); break; - case "kimi": result = await fetchKimiQuota(provider, config, accessToken); break; - case "command-code": result = await fetchCommandCodeQuota(provider, config, accessToken); break; - default: return null; - } - return { result, identity, isCurrent }; -} - -async function fetchExplicitAccountQuota(provider: string, accountId: string, force: boolean, configured?: OcxProviderConfig): Promise { - const key = accountCacheKey(provider, accountId); - const identity = explicitQuotaIdentity(provider, accountId, configured); - const previous = accountQuotaCache.get(key); - const cached = identity && previous?.identity === identity && previous.isCurrent?.() ? previous : undefined; - if (!force && cached && Date.now() - cached.ts < ACCOUNT_QUOTA_TTL_MS - && (!cached.quota || Date.now() - cached.quota.updatedAt < LAST_GOOD_MAX_AGE_MS)) return cached; - const flightKey = `${key}\u0000${identity ?? "missing"}`; - const running = accountQuotaInflight.get(flightKey); - if (running) return running; - const epoch = explicitAccountEpoch; - const lastGood = cached?.quota && Date.now() - cached.quota.updatedAt < LAST_GOOD_MAX_AGE_MS ? cached.quota : null; - const flight = (async (): Promise => { - let read: Awaited> = null; - try { read = await readExplicitAccountQuota(provider, accountId, configured); } catch { /* unavailable */ } - const isCurrent = read?.isCurrent ?? (() => epoch === explicitAccountEpoch && !!identity - && identity === explicitQuotaIdentity(provider, accountId, configured)); - const result = read?.result; - const current = epoch === explicitAccountEpoch && isCurrent(); - const quota = current && result && typeof result !== "symbol" ? result.quota : null; - const empty = result === AUTHORITATIVE_EMPTY_QUOTA; - const entry: AccountQuotaCacheEntry = { - ts: Date.now(), - quota: quota ?? (current && result !== TERMINAL_QUOTA_FAILURE && !empty - && lastGood && Date.now() - lastGood.updatedAt < LAST_GOOD_MAX_AGE_MS ? lastGood : null), - ...(!current || (!quota && !empty) ? { unavailable: true as const } : {}), - identity: read?.identity ?? identity, - isCurrent: () => epoch === explicitAccountEpoch && isCurrent(), - }; - if (entry.isCurrent?.()) accountQuotaCache.set(key, entry); - return entry; - })().finally(() => { if (accountQuotaInflight.get(flightKey) === flight) accountQuotaInflight.delete(flightKey); }); - accountQuotaInflight.set(flightKey, flight); - return flight; -} - -async function fetchExplicitCurrentQuota(provider: string, config: OcxProviderConfig, liveConfig: OcxConfig): Promise { - const id = getAccountSet(provider)?.activeAccountId; - if (!id) return null; - const read = await readExplicitAccountQuota(provider, id, config); - if (!read) return null; - const isCurrent = () => liveConfig.providers[provider] === config - && read.isCurrent() && getAccountSet(provider)?.activeAccountId === id; - if (!isCurrent()) return TERMINAL_QUOTA_FAILURE; - if (read.result && typeof read.result !== "symbol") accountReportCurrent.set(read.result, isCurrent); - return read.result; -} - -function antigravityQuotaDiagnosticIdentity(accountId: string, credential = getAccountCredential("google-antigravity", accountId)): string | undefined { - return credential ? quotaCredentialIdentity("google-antigravity", accountId, credential, { - adapter: "google", baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE, authMode: "oauth", - }) : undefined; -} - -async function fetchAccountQuota( - provider: string, - accountId: string, - forceRefresh: boolean, - providerConfig?: OcxProviderConfig, -): Promise { - if (!supportsPerAccountQuota(provider)) return { ts: Date.now(), quota: null, unavailable: true }; - if (explicitAccountReader(provider)) return fetchExplicitAccountQuota(provider, accountId, forceRefresh, providerConfig); - if (provider === "anthropic") hydrateAccountQuotaCache(); - const key = accountCacheKey(provider, accountId); - const writerGeneration = captureConfigGeneration(); - const cached = accountQuotaCache.get(key); - if (!forceRefresh && cached && Date.now() - cached.ts < ACCOUNT_QUOTA_TTL_MS) { - if (provider === "google-antigravity" && cached.quotaFailure && cached.quotaFailureIsCurrent?.() !== true) return { ...cached, quotaFailure: undefined }; - return provider === "anthropic" ? { ...cached, quota: normalizeAnthropicQuota(cached.quota, Date.now()) } : cached; - } - const joinable = accountQuotaInflight.get(key); - if (joinable) return joinable; - - const epoch = explicitAccountEpoch; - const probe = (async (): Promise => { - let diagnosticIdentity: string | undefined; - let quotaFailure: QuotaFailureCode | undefined; - const quotaFailureIsCurrent = () => { - try { return epoch === explicitAccountEpoch && diagnosticIdentity !== undefined && diagnosticIdentity === antigravityQuotaDiagnosticIdentity(accountId); } - catch { return false; } - }; - const diagnosticFields = () => quotaFailure && quotaFailureIsCurrent() ? { quotaFailure, quotaFailureIsCurrent } : {}; - try { - if (provider === "google-antigravity") diagnosticIdentity = antigravityQuotaDiagnosticIdentity(accountId); - let quota: ProviderQuota | null; - let kiroSnapshot: KiroUsageSnapshot | null = null; - if (provider === "kiro") { - // Kiro resolves the bearer and its routing metadata from ONE account-scoped - // snapshot. It deliberately does not use getTokenForAccountQuotaProbe: that - // helper refuses to refresh a background `local-cli` slot because Anthropic's - // lock can adopt a mismatched Claude CLI identity, but Kiro marks every - // CLI-imported credential `local-cli`, so the same rule would blank the quota of - // every inactive pool account the moment its token expired. - kiroSnapshot = await fetchKiroUsageSnapshot(await kiroUsageContextForAccount(accountId)); - quota = kiroSnapshot?.quota ?? null; - } else { - const token = await getTokenForAccountQuotaProbe(provider, accountId); - if (provider === "google-antigravity") { - // Per-account Gem/Cla windows (#1082). The project id is part of the stored - // credential; without it the probe cannot be made, and that is "unavailable", - // never 0%. - const credential = getAccountCredential(provider, accountId); - diagnosticIdentity = credential?.access === token ? antigravityQuotaDiagnosticIdentity(accountId, credential) : undefined; - if (!diagnosticIdentity || !credential?.projectId) throw new Error("antigravity account unavailable"); - const result = await probeAntigravityUsageQuota(token, credential.projectId); - quota = result.kind === "available" ? result.quota : null; - if (result.kind === "unavailable") quotaFailure = result.failure; - } else if (provider === "anthropic") { - quota = await fetchAnthropicUsageQuota(token); - } else { - return { ts: Date.now(), quota: null, unavailable: true }; - } - } - if (!quota) { - // Preserve last-good bars and mark unavailable; advance TTL so failures - // negative-cache instead of re-probing on every GUI poll. - const entry: AccountQuotaCacheEntry = { - ts: Date.now(), - // Settle once for all joiners against observations committed during the probe. - quota: provider === "anthropic" - ? normalizeAnthropicQuota(accountQuotaCache.get(key)?.quota, Date.now()) : cached?.quota ?? null, - unavailable: true, - ...diagnosticFields(), - }; - if (mayCommitAccountQuotaKey(key, writerGeneration)) { - accountQuotaCache.set(key, entry); - if (provider === "kiro") commitKiroAccountUsageState(key, null); - sweepExpiredOnWrite(entry.ts); - } - return entry; - } - const entry: AccountQuotaCacheEntry = { - ts: Date.now(), quota: provider === "anthropic" ? normalizeAnthropicQuota(quota, Date.now()) : quota, - }; - if (mayCommitAccountQuotaKey(key, writerGeneration)) { - accountQuotaCache.set(key, entry); - // Exhaustion state rides the SAME commit guard as the quota row: a probe from a - // superseded config generation must not publish either half. - if (provider === "kiro") commitKiroAccountUsageState(key, kiroSnapshot); - sweepExpiredOnWrite(entry.ts); - } - return entry; - } catch { - if (provider === "google-antigravity") quotaFailure = "account_unavailable"; - const entry: AccountQuotaCacheEntry = { - ts: Date.now(), - quota: provider === "anthropic" - ? normalizeAnthropicQuota(accountQuotaCache.get(key)?.quota, Date.now()) : cached?.quota ?? null, - unavailable: true, - ...diagnosticFields(), - }; - if (mayCommitAccountQuotaKey(key, writerGeneration)) { - accountQuotaCache.set(key, entry); - sweepExpiredOnWrite(entry.ts); - } - return entry; - } - })().finally(() => { - if (accountQuotaInflight.get(key) === probe) accountQuotaInflight.delete(key); - }); - accountQuotaInflight.set(key, probe); - return probe; -} - -/** - * Per-account quota rows for a provider's logged-in accounts. Probes run in parallel; a - * single failing account never blocks the others. - */ -export async function fetchProviderAccountQuotas( - provider: string, - forceRefresh = false, - providerConfig?: OcxProviderConfig, -): Promise { - if (!supportsPerAccountQuota(provider)) return []; - const set = getAccountSet(provider); - if (!set) return []; - return mapQuotaRoster(set.accounts, async account => { - const entry = await fetchAccountQuota(provider, account.id, forceRefresh, providerConfig); - const result: ProviderAccountQuota = { - accountId: account.id, - quota: provider === "anthropic" ? normalizeAnthropicQuota(entry.quota, Date.now()) : entry.quota, - ...(entry.unavailable ? { unavailable: true as const } : {}), - ...(entry.unavailable && entry.quotaFailure && entry.quotaFailureIsCurrent?.() === true ? { quotaFailure: entry.quotaFailure } : {}), - }; - if (entry.quotaFailureIsCurrent) Object.defineProperty(result, "quotaFailureIsCurrent", { value: entry.quotaFailureIsCurrent }); - if (!explicitAccountReader(provider)) return result; - const identity = entry.identity; - Object.defineProperty(result, "isCurrent", { value: () => { - if (entry.isCurrent) return entry.isCurrent(); - const credential = getAccountCredential(provider, account.id); - return !!credential && (!identity || explicitQuotaIdentity(provider, account.id, providerConfig) === identity); - } }); - return result; - }); -} - -function normalizedBaseUrl(value: string): string | null { - try { - const url = new URL(value); - if (url.username || url.password || url.search || url.hash) return null; - return `${url.origin.toLowerCase()}${url.pathname.replace(/\/+$/, "")}`; - } catch { - return null; - } -} - -function quotaResetAt(row: Record): number | undefined { - return normalizeResetAt(row.resetTime ?? row.resetAt ?? row.reset_time ?? row.reset_at); -} - -function isCanonicalKimiCodeBaseUrl(baseUrl: string): boolean { - return normalizedBaseUrl(baseUrl) === KIMI_CODE_BASE_URL; -} - -function isCanonicalCommandCodeBaseUrl(baseUrl: string): boolean { - const normalized = normalizedBaseUrl(baseUrl); - // OAuth preset points at the API root; the Provider-API preset at /provider/v1. - return normalized === COMMAND_CODE_BASE_URL || normalized === `${COMMAND_CODE_BASE_URL}/provider/v1`; -} - -/** Prefer the nested `data` shell when the outer object is only an envelope. */ -function unwrapKimiQuotaPayload(value: unknown): Record | null { - const body = asRecord(value); - if (!body) return null; - const nested = asRecord(body.data); - if (!nested) return body; - // A null/non-usable outer field is a placeholder, not data — an envelope like - // { usage: null, data: { usage: {...} } } must still unwrap to the nested payload. - const usable = (field: unknown): boolean => field !== undefined && field !== null; - const outerHasUsage = usable(body.usage) || usable(body.limits) || usable(body.totalQuota); - const nestedHasUsage = usable(nested.usage) || usable(nested.limits) || usable(nested.totalQuota); - return !outerHasUsage && nestedHasUsage ? nested : body; -} - -function kimiLimitLabel(item: Record, detail: Record): string { - return [item.name, item.title, item.scope, detail.name, detail.title] - .filter((value): value is string => typeof value === "string") - .join(" ") - .toLowerCase(); -} - -function parseKimiQuotaRow(value: unknown, resetFallback?: Record): { percent: number; resetAt?: number } | null { - const row = asRecord(value); - if (!row) return null; - const resetAt = quotaResetAt(row) ?? (resetFallback ? quotaResetAt(resetFallback) : undefined); - const limit = toFiniteNumber(row.limit); - if (limit !== undefined && limit > 0) { - let used = toFiniteNumber(row.used); - if (used === undefined) { - const remaining = toFiniteNumber(row.remaining); - if (remaining !== undefined) used = limit - remaining; - } - if (used !== undefined) { - const percent = normalizePercent((used / limit) * 100); - if (percent !== undefined) return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; - } - } - // Some payloads expose utilisation directly when limit/used arithmetic is absent. - const direct = normalizePercent(row.utilization ?? row.percent ?? row.usedPercent ?? row.used_percent); - return direct === undefined ? null : { percent: direct, ...(resetAt !== undefined ? { resetAt } : {}) }; -} - -function isKimiFiveHourLimit(item: Record, detail: Record, window: Record): boolean { - const duration = toFiniteNumber(window.duration ?? item.duration ?? detail.duration); - const unit = String(window.timeUnit ?? item.timeUnit ?? detail.timeUnit ?? "").toUpperCase(); - if ((unit.includes("MINUTE") && duration === 300) || (unit.includes("HOUR") && duration === 5)) return true; - return /(^|\b)5\s*(?:h|hour)/.test(kimiLimitLabel(item, detail)); -} - -function isKimiWeeklyLimit(item: Record, detail: Record, window: Record): boolean { - const duration = toFiniteNumber(window.duration ?? item.duration ?? detail.duration); - const unit = String(window.timeUnit ?? item.timeUnit ?? detail.timeUnit ?? "").toUpperCase(); - if ((unit.includes("DAY") && duration === 7) || (unit.includes("HOUR") && duration === 168)) return true; - return /weekly|7\s*(?:d|day)/.test(kimiLimitLabel(item, detail)); -} - -function parseKimiQuotaPayload(value: unknown): ProviderQuota | null { - const body = unwrapKimiQuotaPayload(value); - if (!body) return null; - let weekly = parseKimiQuotaRow(body.usage); - const total = parseKimiQuotaRow(body.totalQuota); - let fiveHour: { percent: number; resetAt?: number } | null = null; - if (Array.isArray(body.limits)) { - for (const rawItem of body.limits) { - const item = asRecord(rawItem); - if (!item) continue; - const detail = asRecord(item.detail) ?? item; - const window = asRecord(item.window) ?? {}; - if (!fiveHour && isKimiFiveHourLimit(item, detail, window)) { - fiveHour = parseKimiQuotaRow(detail, window); - } - if (!weekly && isKimiWeeklyLimit(item, detail, window)) { - weekly = parseKimiQuotaRow(detail, window); - } - if (fiveHour && weekly) break; - } - } - const quota: ProviderQuota = { - ...(fiveHour ? { - fiveHourPercent: fiveHour.percent, - ...(fiveHour.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), - } : {}), - ...(weekly ? { - weeklyPercent: weekly.percent, - ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), - } : {}), - ...(total ? { customWindows: [{ label: "Total subscription credits", percent: total.percent, ...(total.resetAt !== undefined ? { resetAt: total.resetAt } : {}) }] } : {}), - updatedAt: Date.now(), - }; - return hasQuotaRows(quota) ? quota : null; -} - -async function resolveKimiQuotaBearer(config: OcxProviderConfig, accountId?: string): Promise { - if (config.authMode === "oauth") { - try { - return accountId ? await getTokenForAccountQuotaProbe("kimi", accountId) : null; - } catch { - return null; - } - } - // ACTIVE key only: silently walking apiKeyPool when the primary env reference is - // unresolved would render a quota bar for a DIFFERENT account than the one routing - // requests — a wrong meter is worse than no meter. - const primary = resolveProviderApiKey(config.apiKey)?.trim(); - return primary || null; -} - -async function fetchKimiQuota(provider: string, config: OcxProviderConfig, accessToken: string): Promise { - // Never release credentials to a user-edited or lookalike provider host. - if (!isCanonicalKimiCodeBaseUrl(config.baseUrl)) return null; - if (!accessToken) return null; - const response = await fetch(KIMI_CODE_USAGE_URL, { - headers: { Accept: "application/json", Authorization: `Bearer ${accessToken}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - const quota = parseKimiQuotaPayload(await readQuotaJson(response)); - return quota ? keyReport(provider, "kimi:usages", quota, config, accessToken, quota) : null; -} - -/** - * Command Code rolling window: `{ cap, used, resetAt }` off /alpha/billing/credits, - * normalized to a percent with an optional reset timestamp. - */ -function parseCommandCodeWindow(value: unknown): { percent: number; resetAt?: number } | null { - const row = asRecord(value); - if (!row) return null; - const cap = toFiniteNumber(row.cap); - const used = toFiniteNumber(row.used); - if (cap === undefined || used === undefined || cap <= 0 || used < 0) return null; - const percent = normalizePercent((used / cap) * 100); - if (percent === undefined) return null; - const resetAt = quotaResetAt(row); - return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; -} - -/** Soft-fail GET returning a parsed record, or null when unavailable. */ -async function fetchCommandCodeJson(url: string, bearer: string): Promise | null> { - try { - const response = await fetch(url, { - headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - return asRecord(await readQuotaJson(response)); - } catch { - return null; - } -} - -/** - * Soft-fail period spend (used) against the remaining credit pools → creditsUsd. - * Period scoping: `since=` keeps spend aligned with the - * pools' billing cycle, and `currentPeriodEnd` becomes expiresAt. - */ -async function fetchCommandCodeSpend( - bearer: string, - credits: Record | null, - orgQuery: string, -): Promise { - if (!credits) return undefined; - const subscriptionBody = await fetchCommandCodeJson(`${COMMAND_CODE_SUBSCRIPTIONS_URL}${orgQuery}`, bearer); - const subscription = asRecord(subscriptionBody?.data) ?? subscriptionBody; - const periodStart = typeof subscription?.currentPeriodStart === "string" ? subscription.currentPeriodStart.trim() : ""; - // Unscoped /usage/summary is lifetime spend; mixing it with current-cycle - // remaining pools produces a wrong percent. Omit creditsUsd until a period exists. - if (!periodStart) return undefined; - const sinceQuery = `${orgQuery ? "&" : "?"}since=${encodeURIComponent(periodStart)}`; - const expiresAt = normalizeResetAt(subscription?.currentPeriodEnd); - const summaryBody = await fetchCommandCodeJson(`${COMMAND_CODE_USAGE_URL}${orgQuery}${sinceQuery}`, bearer); - const summary = asRecord(summaryBody?.data) ?? summaryBody; - const used = toFiniteNumber(summary?.totalCost) ?? toFiniteNumber(summary?.totalMonthlyCredits); - if (used === undefined || used < 0) return undefined; - const pools = [credits.monthlyCredits, credits.purchasedCredits, credits.freeCredits] - .map(value => toFiniteNumber(value)) - .filter((value): value is number => value !== undefined); - // Field presence is what separates a real balance from absent data: an exhausted - // all-zero account still reports remaining=0, while no remaining-credit field at - // all means there is nothing to meter. - if (pools.length === 0) return undefined; - const remaining = pools.reduce((sum, value) => sum + Math.max(0, value ?? 0), 0); - const limit = used + remaining; - const percent = normalizePercent(limit > 0 ? (used / limit) * 100 : 0); - // Purchased credits roll over past the subscription period end, so an expiry is - // only truthful when the aggregate contains no non-expiring purchased pool. - const purchased = toFiniteNumber(credits.purchasedCredits) ?? 0; - return percent === undefined - ? undefined - : { - used, - limit, - remaining, - percent, - ...(expiresAt !== undefined && purchased <= 0 ? { expiresAt } : {}), - }; -} - -/** OAuth access token or ACTIVE Provider-API key for the Command Code quota probe. */ -async function resolveCommandCodeQuotaBearer(config: OcxProviderConfig, accountId?: string): Promise { - if (config.authMode === "oauth") { - try { - return accountId ? await getTokenForAccountQuotaProbe("command-code", accountId) : null; - } catch { - return null; - } - } - // ACTIVE key only: a quota bar for a different account than the one routing - // requests is a wrong meter, not a helpful one. - return resolveProviderApiKey(config.apiKey)?.trim() || null; -} - -/** - * Command Code `GET /alpha/billing/credits` — the same Bearer surface the CLI's - * usage view uses (windowLimits.fiveHour / windowLimits.weekly), plus soft - * whoami (team orgId scoping) and subscription-scoped spend for creditsUsd. - */ -async function fetchCommandCodeQuota(provider: string, config: OcxProviderConfig, bearer: string): Promise { - // Never release credentials to a user-edited or lookalike provider host. - if (!isCanonicalCommandCodeBaseUrl(config.baseUrl)) return null; - if (!bearer) return null; - const whoamiBody = await fetchCommandCodeJson(COMMAND_CODE_WHOAMI_URL, bearer); - const whoami = asRecord(whoamiBody?.data) ?? whoamiBody; - const org = asRecord(whoami?.org); - const orgId = typeof org?.id === "string" && org.id.trim() ? org.id.trim() : null; - const orgQuery = orgId ? `?orgId=${encodeURIComponent(orgId)}` : ""; - const response = await fetch(`${COMMAND_CODE_CREDITS_URL}${orgQuery}`, { - headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) { - return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 - ? TERMINAL_QUOTA_FAILURE - : null; - } - const raw = asRecord(await readQuotaJson(response)); - const body = asRecord(raw?.data) ?? raw; - const credits = asRecord(body?.credits); - const limits = asRecord(body?.windowLimits); - if (!credits && !limits) return null; - const fiveHour = parseCommandCodeWindow(limits?.fiveHour); - const weekly = parseCommandCodeWindow(limits?.weekly); - const creditsUsd = await fetchCommandCodeSpend(bearer, credits, orgQuery); - const quota: ProviderQuota = { - ...(fiveHour ? { - fiveHourPercent: fiveHour.percent, - ...(fiveHour.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), - } : {}), - ...(weekly ? { - weeklyPercent: weekly.percent, - ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), - } : {}), - ...(creditsUsd ? { creditsUsd } : {}), - updatedAt: Date.now(), - }; - // Rolling windows and the credit balance both gate inference on this bearer. - return keyReport(provider, "command-code:credits", quota, config, bearer, quota); -} - -/** Cursor included usage via api2.cursor.sh (Bearer from OAuth) — unofficial, may change. */ -async function fetchCursorQuota(provider: string, accessToken: string): Promise { - - const authHeaders = { - Accept: "application/json", - Authorization: `Bearer ${accessToken}`, - "User-Agent": "opencodex-quota", - } as const; - - // Prefer dashboard period usage (Pro/Team/Ultra spend allowance in USD cents). - // Field names follow Cursor's Connect RPC shape (limit/remaining/includedSpend), not usedCents. - try { - const periodRes = await fetch("https://api2.cursor.sh/aiserver.v1.DashboardService/GetCurrentPeriodUsage", { - method: "POST", - redirect: "error", - headers: { - ...authHeaders, - "Content-Type": "application/json", - "Connect-Protocol-Version": "1", - }, - body: "{}", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (periodRes.ok) { - const body = asRecord(await readQuotaJson(periodRes)); - const planUsage = asRecord(body?.planUsage); - if (planUsage) { - const resetAt = normalizeResetAt(body?.billingCycleEnd ?? planUsage.billingCycleEnd ?? body?.periodEnd); - - // Primary meter: overall included allowance (Cursor Settings → Usage total %). - // autoPercentUsed / apiPercentUsed are secondary pools and must not replace the total. - const limit = toFiniteNumber(planUsage.limit ?? planUsage.limitCents ?? planUsage.totalLimitCents); - const remaining = toFiniteNumber(planUsage.remaining ?? planUsage.remainingCents); - const includedSpend = toFiniteNumber(planUsage.includedSpend ?? planUsage.usedCents ?? planUsage.used); - const totalSpend = toFiniteNumber(planUsage.totalSpend); - let used: number | undefined; - if (includedSpend !== undefined) used = includedSpend; - else if (limit !== undefined && remaining !== undefined) used = Math.max(0, limit - remaining); - else if (totalSpend !== undefined) used = totalSpend; - const totalPercent = normalizePercent(planUsage.totalPercentUsed ?? planUsage.percentUsed) - ?? (limit !== undefined && limit > 0 && used !== undefined - ? normalizePercent((used / limit) * 100) - : undefined); - - const autoPercent = normalizePercent(planUsage.autoPercentUsed); - const apiPercent = normalizePercent(planUsage.apiPercentUsed); - const customWindows: ProviderQuotaWindow[] = []; - if (autoPercent !== undefined) { - customWindows.push({ - label: "First-party models", - percent: autoPercent, - ...(resetAt !== undefined ? { resetAt } : {}), - }); - } - if (apiPercent !== undefined) { - customWindows.push({ - label: "API usage", - percent: apiPercent, - ...(resetAt !== undefined ? { resetAt } : {}), - }); - } - - if (totalPercent !== undefined || customWindows.length > 0) { - const built = report(provider, "cursor:period-usage", { - ...(totalPercent !== undefined ? { - monthlyPercent: totalPercent, - ...(resetAt !== undefined ? { monthlyResetAt: resetAt } : {}), - } : {}), - ...(customWindows.length > 0 ? { customWindows } : {}), - updatedAt: Date.now(), - }); - if (built) return { ...built, reverseEngineered: true }; - } - } - } - } catch { - /* fall through */ - } - - // /api/usage/summary — same host, sometimes richer than /auth/usage for Team plans. - try { - const summaryRes = await fetch("https://api2.cursor.sh/api/usage/summary", { - headers: authHeaders, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (summaryRes.ok) { - const body = asRecord(await readQuotaJson(summaryRes)); - const individual = asRecord(body?.individualUsage); - const plan = asRecord(individual?.plan); - if (plan) { - const used = toFiniteNumber(plan.used); - const limit = toFiniteNumber(plan.limit); - const percent = normalizePercent(plan.totalPercentUsed) - ?? (used !== undefined && limit !== undefined && limit > 0 - ? normalizePercent((used / limit) * 100) - : undefined); - if (percent !== undefined) { - const built = report(provider, "cursor:usage-summary", { - monthlyPercent: percent, - monthlyResetAt: normalizeResetAt(body?.billingCycleEnd), - updatedAt: Date.now(), - }); - if (built) return { ...built, reverseEngineered: true }; - } - } - } - } catch { - /* fall through to /auth/usage */ - } - - const response = await fetch("https://api2.cursor.sh/auth/usage", { - headers: authHeaders, - redirect: "error", - signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }); - if (!response.ok) return null; - const body = asRecord(await readQuotaJson(response)); - if (!body) return null; - - // Prefer the gpt-4 bucket (historical "fast requests"); else first model with used+limit. - let used: number | undefined; - let limit: number | undefined; - const gpt4 = asRecord(body["gpt-4"]); - if (gpt4) { - used = toFiniteNumber(gpt4.numRequests ?? gpt4.used); - limit = toFiniteNumber(gpt4.maxRequestUsage ?? gpt4.limit ?? gpt4.maxRequests); - } - if (used === undefined || limit === undefined || limit <= 0) { - for (const [key, value] of Object.entries(body)) { - if (key === "startOfMonth" || key === "billingCycleStart") continue; - const bucket = asRecord(value); - if (!bucket) continue; - const bucketUsed = toFiniteNumber(bucket.numRequests ?? bucket.used); - const bucketLimit = toFiniteNumber(bucket.maxRequestUsage ?? bucket.limit ?? bucket.maxRequests); - if (bucketUsed !== undefined && bucketLimit !== undefined && bucketLimit > 0) { - used = bucketUsed; - limit = bucketLimit; - break; - } - } - } - if (used === undefined || limit === undefined || limit <= 0) return null; - const percent = normalizePercent((used / limit) * 100); - if (percent === undefined) return null; - const startOfMonth = normalizeResetAt(body.startOfMonth ?? body.billingCycleStart); - // Next reset = same day next month, computed in UTC to avoid timezone-shifted rollover. - const monthlyResetAt = startOfMonth !== undefined - ? (() => { - const start = new Date(startOfMonth); - return Date.UTC(start.getUTCFullYear(), start.getUTCMonth() + 1, start.getUTCDate()); - })() - : undefined; - const built = report(provider, "cursor:auth-usage", { - monthlyPercent: percent, - ...(monthlyResetAt !== undefined ? { monthlyResetAt } : {}), - updatedAt: Date.now(), - }); - return built ? { ...built, reverseEngineered: true } : null; -} - -function quotaInfoEntries(modelInfo: Record): Record[] { - const entries: Record[] = []; - const add = (value: unknown, tier?: string) => { - const rec = asRecord(value); - if (!rec) return; - entries.push(tier ? { ...rec, tier } : rec); - }; - const addArray = (value: unknown) => { - if (!Array.isArray(value)) return; - for (const entry of value) add(entry); - }; - - if (Array.isArray(modelInfo.quotaInfo)) addArray(modelInfo.quotaInfo); - else add(modelInfo.quotaInfo); - addArray(modelInfo.quotaInfos); - - const byTier = asRecord(modelInfo.quotaInfoByTier); - if (byTier) { - for (const [tier, value] of Object.entries(byTier)) { - if (Array.isArray(value)) { - for (const entry of value) add(entry, tier); - } else { - add(value, tier); - } - } - } - return entries; -} - -function classifyAntigravityFamily(modelId: string, modelInfo: Record, quotaInfo: Record): "Gem" | "Cla" | null { - const displayName = typeof modelInfo.displayName === "string" ? modelInfo.displayName : ""; - const tier = typeof quotaInfo.tier === "string" ? quotaInfo.tier : ""; - const haystack = `${modelId} ${displayName} ${tier}`.toLowerCase(); - if (haystack.includes("gemini")) return "Gem"; - if (haystack.includes("claude") || haystack.includes("opus") || haystack.includes("sonnet") || haystack.includes("gpt-oss") || haystack.includes("gpt_oss")) return "Cla"; - return null; -} - -function antigravityUsedPercent(quotaInfo: Record): number | undefined { - const target = asRecord(quotaInfo.remaining) ?? quotaInfo; - const remaining = normalizePercent(toFiniteNumber(target.remainingFraction) !== undefined - ? toFiniteNumber(target.remainingFraction)! * 100 - : toFiniteNumber(target.remainingPercentage) !== undefined - ? toFiniteNumber(target.remainingPercentage)! * 100 - : undefined); - if (remaining === undefined) return undefined; - return normalizePercent(100 - remaining); -} - -/** Gem/Cla windows from a `fetchAvailableModels` body; shared by the provider and account probes. */ -function antigravityWindowsFromModels(body: Record | null): ProviderQuotaWindow[] { - const models = asRecord(body?.models); - if (!models) return []; - - const windows = new Map(); - for (const [modelId, rawModelInfo] of Object.entries(models)) { - const modelInfo = asRecord(rawModelInfo); - if (!modelInfo) continue; - for (const quotaInfo of quotaInfoEntries(modelInfo)) { - const label = classifyAntigravityFamily(modelId, modelInfo, quotaInfo); - if (!label || windows.has(label)) continue; - const percent = antigravityUsedPercent(quotaInfo); - if (percent === undefined) continue; - windows.set(label, { - label, - percent, - ...(normalizeResetAt(quotaInfo.resetTime) !== undefined ? { resetAt: normalizeResetAt(quotaInfo.resetTime) } : {}), - }); - } - } - - const customWindows = ["Gem", "Cla"].flatMap(label => { - const window = windows.get(label); - return window ? [window] : []; - }); - return customWindows; -} - -/** - * Parse Google Antigravity quota from `v1internal:retrieveUserQuotaSummary`. - * Groups contain Gemini models and Claude/3P models, each with 5h and weekly limit buckets. - */ -function parseAntigravityQuotaSummary(body: Record | null): ProviderQuota | null { - const groups = Array.isArray(body?.groups) ? (body.groups as unknown[]) : []; - if (groups.length === 0) return null; - - const customWindowsMap = new Map(); - - for (const rawGroup of groups) { - const group = asRecord(rawGroup); - if (!group) continue; - const groupName = `${typeof group.displayName === "string" ? group.displayName : ""} ${typeof group.description === "string" ? group.description : ""}`.toLowerCase(); - const isGemini = groupName.includes("gemini"); - const isClaude = groupName.includes("claude") || groupName.includes("3p") || groupName.includes("gpt"); - - const buckets = Array.isArray(group.buckets) ? (group.buckets as unknown[]) : []; - for (const rawBucket of buckets) { - const bucket = asRecord(rawBucket); - if (!bucket) continue; - const windowStr = `${typeof bucket.window === "string" ? bucket.window : ""} ${typeof bucket.bucketId === "string" ? bucket.bucketId : ""} ${typeof bucket.displayName === "string" ? bucket.displayName : ""}`.toLowerCase(); - const percent = antigravityUsedPercent(bucket); - if (percent === undefined) continue; - const resetAt = normalizeResetAt(bucket.resetTime); - - const isWeekly = windowStr.includes("week"); - const is5h = windowStr.includes("5h") || windowStr.includes("five"); - - if (isGemini) { - const label = is5h ? "Gem" : isWeekly ? "Gem (Weekly)" : ""; - if (label && !customWindowsMap.has(label)) { - customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); - } - } else if (isClaude) { - const label = is5h ? "Cla" : isWeekly ? "Cla (Weekly)" : ""; - if (label && !customWindowsMap.has(label)) { - customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); - } - } else { - const baseLabel = typeof group.displayName === "string" ? group.displayName : "Other"; - const label = isWeekly ? `${baseLabel} (Weekly)` : baseLabel; - if (!customWindowsMap.has(label)) { - customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); - } - } - } - } - - const PREFERRED_ORDER = ["Gem", "Gem (Weekly)", "Cla", "Cla (Weekly)"]; - const customWindows = Array.from(customWindowsMap.values()).sort((a, b) => { - const ia = PREFERRED_ORDER.indexOf(a.label); - const ib = PREFERRED_ORDER.indexOf(b.label); - if (ia !== -1 && ib !== -1) return ia - ib; - if (ia !== -1) return -1; - if (ib !== -1) return 1; - return a.label.localeCompare(b.label); - }); - - if (customWindows.length === 0) { - return null; - } - - return { - customWindows, - updatedAt: Date.now(), - }; -} - -const ANTIGRAVITY_ACCOUNT_QUOTA_BASE = "https://daily-cloudcode-pa.googleapis.com"; -const ANTIGRAVITY_QUOTA_SUMMARY_URL = `${ANTIGRAVITY_ACCOUNT_QUOTA_BASE}/v1internal:retrieveUserQuotaSummary`; -const ANTIGRAVITY_QUOTA_MODELS_URL = `${ANTIGRAVITY_ACCOUNT_QUOTA_BASE}/v1internal:fetchAvailableModels`; - -/** Only these fixed accounting destinations may use transparent Fake-IP DNS. */ -export function isCanonicalAntigravityQuotaUrl(name: string, url: string): boolean { - return name === "google-antigravity" - && (url === ANTIGRAVITY_QUOTA_SUMMARY_URL || url === ANTIGRAVITY_QUOTA_MODELS_URL); -} - -let antigravityOutboundDependencies: ProviderOutboundDependencies = { - isCanonicalUrl: isCanonicalAntigravityQuotaUrl, -}; - -/** Test seam: inject resolver/pinned transport for provider and per-account probes. */ -export function setAntigravityAccountQuotaTransportForTests(dependencies: ProviderOutboundDependencies | null): void { - antigravityOutboundDependencies = { ...dependencies, isCanonicalUrl: isCanonicalAntigravityQuotaUrl }; -} - -/** - * Per-account Antigravity quota (#1082). Always probes Google's own Cloud Code Assist host - * through the pinned provider-outbound transport: a configured `baseUrl` is a routing choice - * for requests, not a second source of Google's accounting for a stored credential, and fixing - * the destination keeps the `provider\0accountId` cache identity exact across config changes. - * A redirect or non-2xx yields null (unavailable), never a partial row. - */ -type AntigravityQuotaProbeResult = - | { kind: "available"; quota: ProviderQuota; source: "google-antigravity:retrieveUserQuotaSummary" | "google-antigravity:fetchAvailableModels" } - | { kind: "unavailable"; failure: QuotaFailureCode; legacy: { kind: "null" } | { kind: "throw"; error: unknown } }; - -function quotaTransportFailure(error: unknown): QuotaFailureCode { - if (error instanceof ProviderOutboundPolicyError) return "destination_blocked"; - if (error instanceof DestinationDnsResolutionError) return "dns_failed"; - if (error instanceof PinnedHttpError) return error.code === "output_byte_limit" ? "response_unusable" : "timeout"; - if (error instanceof DOMException && error.name === "TimeoutError") return "timeout"; - return "transport_error"; -} - -function quotaHttpFailure(status: number): QuotaFailureCode { - if (status >= 300 && status < 400) return "redirect_blocked"; - if (status === 401 || status === 403) return "access_denied"; - if (status === 429) return "rate_limited"; - return "upstream_error"; -} - -function unavailableAntigravityQuota(failure: QuotaFailureCode): AntigravityQuotaProbeResult { - return { kind: "unavailable", failure, legacy: { kind: "null" } }; -} - -/** - * Prefer a summary network-policy diagnosis over a vaguer fallback. A blocked - * destination is an actionable local-network fact, while "upstream_error" tells - * the operator to go look at Google. A successful models probe still clears - * the first failure completely. - */ -function antigravityUnavailableFailure( - summaryFailure: QuotaFailureCode | undefined, - fallbackFailure: QuotaFailureCode, -): QuotaFailureCode { - if ( - (summaryFailure === "destination_blocked" || summaryFailure === "dns_failed") - && fallbackFailure !== "destination_blocked" - && fallbackFailure !== "dns_failed" - ) { - return summaryFailure; - } - return fallbackFailure; -} - -async function probeAntigravityUsageQuota(accessToken: string, projectId: string): Promise { - const fetchQuota = (url: string) => providerOutboundPost("google-antigravity", { baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE }, url, { - headers: { - Accept: "application/json", "Content-Type": "application/json", - "User-Agent": antigravityUserAgent(), Authorization: `Bearer ${accessToken}`, - }, - body: JSON.stringify({ project: projectId }), signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), - }, antigravityOutboundDependencies); - let summaryFailure: QuotaFailureCode | undefined; - try { - const response = await fetchQuota(ANTIGRAVITY_QUOTA_SUMMARY_URL); - if (await providerRedirectError(response, ANTIGRAVITY_QUOTA_SUMMARY_URL)) return unavailableAntigravityQuota("redirect_blocked"); - if (response.status === 401 || response.status === 403) return unavailableAntigravityQuota("access_denied"); - if (response.ok) { - const quota = parseAntigravityQuotaSummary(asRecord(await readQuotaJson(response))); - if (quota) return { kind: "available", quota, source: "google-antigravity:retrieveUserQuotaSummary" }; - } - } catch (error) { - // Existing behavior: summary transport/parse failure may recover through the models probe. - summaryFailure = quotaTransportFailure(error); - } - try { - const response = await fetchQuota(ANTIGRAVITY_QUOTA_MODELS_URL); - if (await providerRedirectError(response, ANTIGRAVITY_QUOTA_MODELS_URL)) { - return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, "redirect_blocked")); - } - if (!response.ok) { - return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, quotaHttpFailure(response.status))); - } - const customWindows = antigravityWindowsFromModels(asRecord(await readQuotaJson(response))); - if (!customWindows.length) { - return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, "response_unusable")); - } - return { kind: "available", quota: { customWindows, updatedAt: Date.now() }, source: "google-antigravity:fetchAvailableModels" }; - } catch (error) { - // The public compatibility wrapper still rejects this exact fallback error; it never enters a DTO. - return { - kind: "unavailable", - failure: antigravityUnavailableFailure(summaryFailure, quotaTransportFailure(error)), - legacy: { kind: "throw", error }, - }; - } -} - -export async function fetchAntigravityUsageQuota(accessToken: string, projectId: string): Promise { - const result = await probeAntigravityUsageQuota(accessToken, projectId); - if (result.kind === "available") return result.quota; - if (result.legacy.kind === "throw") throw result.legacy.error; - return null; -} - -async function fetchAntigravityQuota(provider: string): Promise { - const credential = getCredential("google-antigravity"); - if (!credential?.projectId) return null; - let accessToken: string; - try { accessToken = await getValidAccessToken("google-antigravity"); } catch { return null; } - const result = await probeAntigravityUsageQuota(accessToken, credential.projectId); - if (result.kind === "available") return report(provider, result.source, result.quota); - if (result.legacy.kind === "throw") throw result.legacy.error; - return null; -} -type KeyQuotaReader = (name: string, provider: OcxProviderConfig) => Promise; - -/** Same selector drives cheap capabilities and uncached reads; never resolves credentials. */ -function keyQuotaReaderForProvider(name: string, provider: OcxProviderConfig): KeyQuotaReader | null { - if (provider.disabled === true || (provider.authMode ?? "key") !== "key") return null; - if (isCanonicalKimiCodeBaseUrl(provider.baseUrl)) { - return async (id, config) => { - const bearer = await resolveKimiQuotaBearer(config); - return bearer ? fetchKimiQuota(id, config, bearer) : null; - }; - } - if (name === "commandcode" && isCanonicalCommandCodeBaseUrl(provider.baseUrl)) { - return async (id, config) => { - const bearer = await resolveCommandCodeQuotaBearer(config); - return bearer ? fetchCommandCodeQuota(id, config, bearer) : null; - }; - } - if (registryEntryForProviderDestination(provider)?.id === "opencode-go") return fetchOpenCodeGoQuota; - if (isCanonicalA6apiBaseUrl(provider.baseUrl)) return fetchA6apiQuota; - if (name === "openrouter" && isCanonicalOpenRouterBaseUrl(provider.baseUrl)) return fetchOpenRouterQuota; - if (name === "deepseek" && isCanonicalDeepSeekBaseUrl(provider.baseUrl)) return fetchDeepSeekQuota; - if (name === "cline-pass" && isCanonicalClineBaseUrl(provider.baseUrl)) return fetchClineQuota; - if (isCanonicalOllamaCloudBaseUrl(provider.baseUrl ?? getProviderRegistryEntry(name)?.baseUrl)) return fetchOllamaCloudQuota; - // #4201: the Responses preset is the same domestic GLM Coding Plan subscription on the OpenAI - // Responses wire, so it reads the same monitor endpoint. Eligibility stays a name list AND the - // canonical-URL guard: the guard is what keeps BigModel's bare-key Authorization from reaching a - // lookalike host, so a same-named custom destination still dispatches nothing. - if (["zai", "glm", "glm-cn", "zhipu-bigmodel-coding", "zhipu-bigmodel-responses"].includes(name) && isCanonicalZaiBaseUrl(provider.baseUrl)) return fetchZaiQuota; - if (["minimax", "minimax-cn"].includes(name) && isCanonicalMinimaxBaseUrl(provider.baseUrl)) return fetchMinimaxQuota; - if (name === "moonshot" && isCanonicalMoonshotBaseUrl(provider.baseUrl)) return fetchMoonshotQuota; - if (name === "venice" && isCanonicalVeniceBaseUrl(provider.baseUrl)) return fetchVeniceQuota; - if (name === "synthetic" && isCanonicalSyntheticBaseUrl(provider.baseUrl)) return fetchSyntheticQuota; - if (name === "deepinfra" && isCanonicalDeepInfraBaseUrl(provider.baseUrl)) return fetchDeepInfraQuota; - if (name === "neuralwatt" && isCanonicalNeuralwattBaseUrl(provider.baseUrl)) return fetchNeuralwattQuota; - return null; -} +import { listCodexAuthAccountsSnapshot } from "../codex/auth-api"; +import { resolveEnvValue } from "../config"; +import { getAccountCredential, getAccountSet } from "../oauth/store"; +import { apiKeyPoolEntryId } from "./api-keys"; +import { captureConfigGeneration, sweepExpiredOnWrite } from "../lib/state-store-sweeper"; +import { ACCOUNT_QUOTA_TTL_MS, CACHE_TTL_MS } from "./quota-wire"; +import { replaceCachedProviderQuotas } from "./quota-routing-cache"; +import { + commitKiroAccountUsageState, + fetchKiroUsageSnapshot, + type KiroUsageSnapshot, + kiroUsageContextForAccount, +} from "./kiro-usage"; +import { mapQuotaRoster, readProviderApiKeyQuotas, type ProviderApiKeyQuota } from "./quota-key-accounts"; +import type { OcxConfig, OcxProviderConfig } from "../types"; +import type { ProviderQuota, QuotaFailureCode } from "./quota-types"; +import { + accountReportCurrent, + AUTHORITATIVE_EMPTY_QUOTA, + bumpProviderQuotaInvalidationEpoch, + cacheKeyWithAggregationState, + getProviderQuotaReportCache, + hasCodexPoolProvider, + inflight, + invalidationEpoch, + isBuiltInChatGptForwardProvider, + isProviderQuotaReportCurrent, + LAST_GOOD_MAX_AGE_MS, + providerQuotaBeforePublishForTests, + routingEvidence, + setProviderQuotaReportCache, + TERMINAL_QUOTA_FAILURE, + type CodexAuthAccountsSnapshotPromise, + type ProviderQuotaProbeResult, + type ProviderQuotaReport, + type ProviderQuotaResponse, +} from "./quota/report-cache"; +import { + accountCacheKey, + accountQuotaCache, + accountQuotaInflight, + explicitAccountEpoch, + explicitAccountReader, + explicitQuotaConfig, + explicitQuotaDestination, + explicitQuotaIdentity, + getTokenForAccountQuotaProbe, + hasPassiveAccountQuota, + hydrateAccountQuotaCache, + mayCommitAccountQuotaKey, + mayCommitProviderQuotaKey, + normalizeAnthropicQuota, + supportsPerAccountQuota, + type AccountQuotaCacheEntry, + type ProviderAccountQuota, +} from "./quota/account-cache"; +import { + fetchAnthropicQuota, + fetchAnthropicUsageQuota, + fetchChatGptForwardQuota, + fetchCursorQuota, + fetchKiroQuota, + fetchMuseKeyQuota, + fetchPassiveProviderQuota, + fetchXaiQuota, +} from "./quota/vendor-probes-oauth"; +import { fetchCommandCodeQuota, fetchKimiQuota, keyQuotaReaderForProvider } from "./quota/vendor-probes-key"; +import { antigravityQuotaDiagnosticIdentity, fetchAntigravityQuota, probeAntigravityUsageQuota } from "./quota/antigravity"; -export function providerApiKeyQuotaMode(name: string, provider: OcxProviderConfig): AccountQuotaMode { - return keyQuotaReaderForProvider(name, provider) ? "probe" : "unsupported"; -} +export type { ProviderQuota, ProviderQuotaCreditsUsd, ProviderQuotaWindow } from "./quota-types"; +export { QUOTA_RESPONSE_MAX_BYTES } from "./quota-wire"; +export { + clearProviderQuotaCache, + publishKeyReportForTests, + readProviderQuotaJsonForTests, + setProviderQuotaBeforePublishForTests, + type ProviderQuotaReport, + type ProviderQuotaResponse, +} from "./quota/report-cache"; +export { + clearAccountQuotaCache, + getCachedProviderAccountQuota, + hasPassiveAccountQuota, + parseAnthropicRateLimitHeaders, + providerOAuthAccountQuotaMode, + readPassiveProviderAccountQuotas, + recordAnthropicAccountQuotaFromHeaders, + recordPassiveAccountQuota, + reconcileProviderAccountQuotaRows, + resetProviderQuotaReconcileStateForTests, + setCachedProviderAccountQuotaForTests, + supportsPerAccountQuota, + sweepExpiredProviderAccountQuotaRows, + type ProviderAccountQuota, +} from "./quota/account-cache"; +export { fetchAntigravityUsageQuota, isCanonicalAntigravityQuotaUrl, setAntigravityAccountQuotaTransportForTests } from "./quota/antigravity"; +export { parseOllamaCloudQuota, parseZaiQuotaLimits, providerApiKeyQuotaMode } from "./quota/vendor-probes-key"; +export { parseXaiCreditsResponse } from "./quota/vendor-probes-oauth"; export async function fetchProviderApiKeyQuotas(config: OcxConfig, name: string, forceRefresh = false): Promise { const provider = config.providers[name]; @@ -3220,19 +244,21 @@ export async function fetchProviderQuotaReports(config: OcxConfig, forceRefresh // by construction and never becomes fresher on its own. Without the exemption a single // configured passive provider makes this predicate permanently false, so every dashboard // poll re-probes every OTHER provider upstream instead of serving the 5-minute cache. - const cacheFresh = cache && cache.key === key && now - cache.ts < CACHE_TTL_MS - && cache.response.reports.every(item => + const currentCache = getProviderQuotaReportCache(); + const cacheFresh = currentCache && currentCache.key === key && now - currentCache.ts < CACHE_TTL_MS + && currentCache.response.reports.every(item => (item.observed === true || now - item.updatedAt < LAST_GOOD_MAX_AGE_MS) && isProviderQuotaReportCurrent(item)); - if (!forceRefresh && cacheFresh) return cache!.response; + if (!forceRefresh && cacheFresh) return currentCache!.response; const joinable = inflight.get(key); if (!forceRefresh && joinable && joinable.epoch === invalidationEpoch) return joinable.promise; // A forced probe takes commit authority: older in-flight probes must not overwrite its result. - if (forceRefresh) invalidationEpoch += 1; + if (forceRefresh) bumpProviderQuotaInvalidationEpoch(); const epoch = invalidationEpoch; const promise = (async (): Promise => { - const previous = cache && cache.key === key ? cache.response.reports : []; + const previousCache = getProviderQuotaReportCache(); + const previous = previousCache && previousCache.key === key ? previousCache.response.reports : []; const probeResults = await Promise.all( Object.entries(config.providers).map(([name, provider]) => ( maybeFetchProviderQuota(name, provider, config, forceRefresh, prefetchedCodexSnapshot) @@ -3296,7 +322,7 @@ export async function fetchProviderQuotaReports(config: OcxConfig, forceRefresh && generationMismatchedProviders.size === 0 ) { const reports = response.reports.filter(item => mayCommitProviderQuotaKey(item.provider, writerGeneration)); - cache = { key, ts: Date.now(), response: { ...response, reports } }; + setProviderQuotaReportCache({ key, ts: Date.now(), response: { ...response, reports } }); replaceCachedProviderQuotas(reports, routingEvidence); notifyProviderQuotaSnapshot(reports, config); } @@ -3311,3 +337,222 @@ export async function fetchProviderQuotaReports(config: OcxConfig, forceRefresh if (inflight.get(key) === entry) inflight.delete(key); } } + +async function readExplicitAccountQuota(provider: string, accountId: string, configured?: OcxProviderConfig): Promise<{ + result: ProviderQuotaProbeResult; + identity: string | undefined; + isCurrent: () => boolean; +} | null> { + const target = explicitQuotaConfig(provider, configured); + if (!target || !explicitQuotaDestination(provider, target)) return null; + const config = { ...target }; + const epoch = explicitAccountEpoch; + const accessToken = await getTokenForAccountQuotaProbe(provider, accountId); + const credential = getAccountCredential(provider, accountId); + if (!credential || credential.access !== accessToken) return null; + // Pair the post-renewal credential with the destination captured before renewal. + const identity = explicitQuotaIdentity(provider, accountId, config); + const isCurrent = () => epoch === explicitAccountEpoch + && identity === explicitQuotaIdentity(provider, accountId, configured); + if (!isCurrent()) return null; + let result: ProviderQuotaProbeResult; + switch (provider) { + case "xai": result = await fetchXaiQuota(provider, { accessToken, upstreamAccountId: credential.accountId }); break; + case "cursor": result = await fetchCursorQuota(provider, accessToken); break; + case "kimi": result = await fetchKimiQuota(provider, config, accessToken); break; + case "command-code": result = await fetchCommandCodeQuota(provider, config, accessToken); break; + default: return null; + } + return { result, identity, isCurrent }; +} + +async function fetchExplicitAccountQuota(provider: string, accountId: string, force: boolean, configured?: OcxProviderConfig): Promise { + const key = accountCacheKey(provider, accountId); + const identity = explicitQuotaIdentity(provider, accountId, configured); + const previous = accountQuotaCache.get(key); + const cached = identity && previous?.identity === identity && previous.isCurrent?.() ? previous : undefined; + if (!force && cached && Date.now() - cached.ts < ACCOUNT_QUOTA_TTL_MS + && (!cached.quota || Date.now() - cached.quota.updatedAt < LAST_GOOD_MAX_AGE_MS)) return cached; + const flightKey = `${key}\u0000${identity ?? "missing"}`; + const running = accountQuotaInflight.get(flightKey); + if (running) return running; + const epoch = explicitAccountEpoch; + const lastGood = cached?.quota && Date.now() - cached.quota.updatedAt < LAST_GOOD_MAX_AGE_MS ? cached.quota : null; + const flight = (async (): Promise => { + let read: Awaited> = null; + try { read = await readExplicitAccountQuota(provider, accountId, configured); } catch { /* unavailable */ } + const isCurrent = read?.isCurrent ?? (() => epoch === explicitAccountEpoch && !!identity + && identity === explicitQuotaIdentity(provider, accountId, configured)); + const result = read?.result; + const current = epoch === explicitAccountEpoch && isCurrent(); + const quota = current && result && typeof result !== "symbol" ? result.quota : null; + const empty = result === AUTHORITATIVE_EMPTY_QUOTA; + const entry: AccountQuotaCacheEntry = { + ts: Date.now(), + quota: quota ?? (current && result !== TERMINAL_QUOTA_FAILURE && !empty + && lastGood && Date.now() - lastGood.updatedAt < LAST_GOOD_MAX_AGE_MS ? lastGood : null), + ...(!current || (!quota && !empty) ? { unavailable: true as const } : {}), + identity: read?.identity ?? identity, + isCurrent: () => epoch === explicitAccountEpoch && isCurrent(), + }; + if (entry.isCurrent?.()) accountQuotaCache.set(key, entry); + return entry; + })().finally(() => { if (accountQuotaInflight.get(flightKey) === flight) accountQuotaInflight.delete(flightKey); }); + accountQuotaInflight.set(flightKey, flight); + return flight; +} + +async function fetchExplicitCurrentQuota(provider: string, config: OcxProviderConfig, liveConfig: OcxConfig): Promise { + const id = getAccountSet(provider)?.activeAccountId; + if (!id) return null; + const read = await readExplicitAccountQuota(provider, id, config); + if (!read) return null; + const isCurrent = () => liveConfig.providers[provider] === config + && read.isCurrent() && getAccountSet(provider)?.activeAccountId === id; + if (!isCurrent()) return TERMINAL_QUOTA_FAILURE; + if (read.result && typeof read.result !== "symbol") accountReportCurrent.set(read.result, isCurrent); + return read.result; +} + + +async function fetchAccountQuota( + provider: string, + accountId: string, + forceRefresh: boolean, + providerConfig?: OcxProviderConfig, +): Promise { + if (!supportsPerAccountQuota(provider)) return { ts: Date.now(), quota: null, unavailable: true }; + if (explicitAccountReader(provider)) return fetchExplicitAccountQuota(provider, accountId, forceRefresh, providerConfig); + if (provider === "anthropic") hydrateAccountQuotaCache(); + const key = accountCacheKey(provider, accountId); + const writerGeneration = captureConfigGeneration(); + const cached = accountQuotaCache.get(key); + if (!forceRefresh && cached && Date.now() - cached.ts < ACCOUNT_QUOTA_TTL_MS) { + if (provider === "google-antigravity" && cached.quotaFailure && cached.quotaFailureIsCurrent?.() !== true) return { ...cached, quotaFailure: undefined }; + return provider === "anthropic" ? { ...cached, quota: normalizeAnthropicQuota(cached.quota, Date.now()) } : cached; + } + const joinable = accountQuotaInflight.get(key); + if (joinable) return joinable; + + const epoch = explicitAccountEpoch; + const probe = (async (): Promise => { + let diagnosticIdentity: string | undefined; + let quotaFailure: QuotaFailureCode | undefined; + const quotaFailureIsCurrent = () => { + try { return epoch === explicitAccountEpoch && diagnosticIdentity !== undefined && diagnosticIdentity === antigravityQuotaDiagnosticIdentity(accountId); } + catch { return false; } + }; + const diagnosticFields = () => quotaFailure && quotaFailureIsCurrent() ? { quotaFailure, quotaFailureIsCurrent } : {}; + try { + if (provider === "google-antigravity") diagnosticIdentity = antigravityQuotaDiagnosticIdentity(accountId); + let quota: ProviderQuota | null; + let kiroSnapshot: KiroUsageSnapshot | null = null; + if (provider === "kiro") { + // Kiro resolves the bearer and its routing metadata from ONE account-scoped + // snapshot. It deliberately does not use getTokenForAccountQuotaProbe: that + // helper refuses to refresh a background `local-cli` slot because Anthropic's + // lock can adopt a mismatched Claude CLI identity, but Kiro marks every + // CLI-imported credential `local-cli`, so the same rule would blank the quota of + // every inactive pool account the moment its token expired. + kiroSnapshot = await fetchKiroUsageSnapshot(await kiroUsageContextForAccount(accountId)); + quota = kiroSnapshot?.quota ?? null; + } else { + const token = await getTokenForAccountQuotaProbe(provider, accountId); + if (provider === "google-antigravity") { + // Per-account Gem/Cla windows (#1082). The project id is part of the stored + // credential; without it the probe cannot be made, and that is "unavailable", + // never 0%. + const credential = getAccountCredential(provider, accountId); + diagnosticIdentity = credential?.access === token ? antigravityQuotaDiagnosticIdentity(accountId, credential) : undefined; + if (!diagnosticIdentity || !credential?.projectId) throw new Error("antigravity account unavailable"); + const result = await probeAntigravityUsageQuota(token, credential.projectId); + quota = result.kind === "available" ? result.quota : null; + if (result.kind === "unavailable") quotaFailure = result.failure; + } else if (provider === "anthropic") { + quota = await fetchAnthropicUsageQuota(token); + } else { + return { ts: Date.now(), quota: null, unavailable: true }; + } + } + if (!quota) { + // Preserve last-good bars and mark unavailable; advance TTL so failures + // negative-cache instead of re-probing on every GUI poll. + const entry: AccountQuotaCacheEntry = { + ts: Date.now(), + // Settle once for all joiners against observations committed during the probe. + quota: provider === "anthropic" + ? normalizeAnthropicQuota(accountQuotaCache.get(key)?.quota, Date.now()) : cached?.quota ?? null, + unavailable: true, + ...diagnosticFields(), + }; + if (mayCommitAccountQuotaKey(key, writerGeneration)) { + accountQuotaCache.set(key, entry); + if (provider === "kiro") commitKiroAccountUsageState(key, null); + sweepExpiredOnWrite(entry.ts); + } + return entry; + } + const entry: AccountQuotaCacheEntry = { + ts: Date.now(), quota: provider === "anthropic" ? normalizeAnthropicQuota(quota, Date.now()) : quota, + }; + if (mayCommitAccountQuotaKey(key, writerGeneration)) { + accountQuotaCache.set(key, entry); + // Exhaustion state rides the SAME commit guard as the quota row: a probe from a + // superseded config generation must not publish either half. + if (provider === "kiro") commitKiroAccountUsageState(key, kiroSnapshot); + sweepExpiredOnWrite(entry.ts); + } + return entry; + } catch { + if (provider === "google-antigravity") quotaFailure = "account_unavailable"; + const entry: AccountQuotaCacheEntry = { + ts: Date.now(), + quota: provider === "anthropic" + ? normalizeAnthropicQuota(accountQuotaCache.get(key)?.quota, Date.now()) : cached?.quota ?? null, + unavailable: true, + ...diagnosticFields(), + }; + if (mayCommitAccountQuotaKey(key, writerGeneration)) { + accountQuotaCache.set(key, entry); + sweepExpiredOnWrite(entry.ts); + } + return entry; + } + })().finally(() => { + if (accountQuotaInflight.get(key) === probe) accountQuotaInflight.delete(key); + }); + accountQuotaInflight.set(key, probe); + return probe; +} + +/** + * Per-account quota rows for a provider's logged-in accounts. Probes run in parallel; a + * single failing account never blocks the others. + */ +export async function fetchProviderAccountQuotas( + provider: string, + forceRefresh = false, + providerConfig?: OcxProviderConfig, +): Promise { + if (!supportsPerAccountQuota(provider)) return []; + const set = getAccountSet(provider); + if (!set) return []; + return mapQuotaRoster(set.accounts, async account => { + const entry = await fetchAccountQuota(provider, account.id, forceRefresh, providerConfig); + const result: ProviderAccountQuota = { + accountId: account.id, + quota: provider === "anthropic" ? normalizeAnthropicQuota(entry.quota, Date.now()) : entry.quota, + ...(entry.unavailable ? { unavailable: true as const } : {}), + ...(entry.unavailable && entry.quotaFailure && entry.quotaFailureIsCurrent?.() === true ? { quotaFailure: entry.quotaFailure } : {}), + }; + if (entry.quotaFailureIsCurrent) Object.defineProperty(result, "quotaFailureIsCurrent", { value: entry.quotaFailureIsCurrent }); + if (!explicitAccountReader(provider)) return result; + const identity = entry.identity; + Object.defineProperty(result, "isCurrent", { value: () => { + if (entry.isCurrent) return entry.isCurrent(); + const credential = getAccountCredential(provider, account.id); + return !!credential && (!identity || explicitQuotaIdentity(provider, account.id, providerConfig) === identity); + } }); + return result; + }); +} diff --git a/src/providers/quota/account-cache.ts b/src/providers/quota/account-cache.ts new file mode 100644 index 0000000000..93c437a61d --- /dev/null +++ b/src/providers/quota/account-cache.ts @@ -0,0 +1,441 @@ +import { createHash } from "node:crypto"; +import { getValidAccessTokenForAccount } from "../../oauth"; +import { getAccountCredential, getAccountSet } from "../../oauth/store"; +import type { GenerationContext } from "../../lib/state-store-sweeper"; +import { ACCOUNT_QUOTA_TTL_MS, toFiniteNumber } from "../quota-wire"; +import { clearKiroAccountUsageState, reconcileKiroAccountUsageState } from "../kiro-usage"; +import { cancelPendingAccountQuotaPersist, readPersistedAccountQuotas, schedulePersistAccountQuotas } from "../account-quota-disk"; +import { replaceCachedProviderQuotas } from "../quota-routing-cache"; +import { getProviderRegistryEntry } from "../registry"; +import { getProviderQuotaReportCache, hasQuotaRows, routingEvidence, setProviderQuotaReportCache } from "./report-cache"; +import { isCanonicalCommandCodeBaseUrl, isCanonicalKimiCodeBaseUrl } from "./vendor-probes-key"; +import type { AccountQuotaMode, ProviderQuota, ProviderQuotaWindow, QuotaFailureCode } from "../quota-types"; +import type { OcxConfig, OcxProviderConfig } from "../../types"; + +/** Match oauth/index REFRESH_SKEW_MS — use stored access without refresh when still fresh. */ +const ACCOUNT_TOKEN_SKEW_MS = 60_000; + +/** + * Anthropic and Kiro both report usage per CREDENTIAL, so every logged-in account can be + * probed with its own bearer token — the active-account selection and the local usage log + * are irrelevant here. Mirrors the Codex pool behaviour + * (codex/auth-api.ts:fetchPoolAccountQuota), including a per-account TTL so N accounts cost + * at most N upstream calls per window. `ACCOUNT_QUOTA_TTL_MS` lives in `quota-wire.ts` + * because the Kiro exhaustion reader applies the same staleness bound. + */ +export type AccountQuotaCacheEntry = { + ts: number; + quota: ProviderQuota | null; + /** Last probe failed (429 / network / expired login); still may hold last-good quota. */ + unavailable?: true; + quotaFailure?: QuotaFailureCode; + quotaFailureIsCurrent?: () => boolean; + /** Private new-reader identity; never persisted or serialized. */ + identity?: string; + isCurrent?: () => boolean; +}; +/** Expired measurements become unknown; missing reset evidence never implies a fresh allowance. */ +export function normalizeAnthropicQuota(quota: ProviderQuota | null | undefined, now: number): ProviderQuota | null { + if (!quota) return null; + const validReset = (resetAt: unknown): resetAt is number => typeof resetAt === "number" + && Number.isFinite(resetAt) && resetAt > 0 && Number.isFinite(new Date(resetAt).getTime()); + let result = quota; + for (const [percent, reset] of [ + ["fiveHourPercent", "fiveHourResetAt"], + ["weeklyPercent", "weeklyResetAt"], + ["monthlyPercent", "monthlyResetAt"], + ] as const) { + const resetAt = quota[reset]; + if (resetAt === undefined) continue; + const valid = validReset(resetAt); + if (valid && resetAt > now) continue; + if (result === quota) result = { ...quota }; + if (valid) delete result[percent]; + delete result[reset]; + } + // Persisted rows validate only the outer quota object, so custom data may be malformed. + if (quota.customWindows !== undefined) { + const windows = Array.isArray(quota.customWindows) ? quota.customWindows : []; + const retained: ProviderQuotaWindow[] = []; + let changed = !Array.isArray(quota.customWindows); + for (const window of windows) { + if (!window || typeof window !== "object" || typeof window.label !== "string" || !window.label.trim() + || typeof window.percent !== "number" || !Number.isFinite(window.percent) + || window.percent < 0 || window.percent > 100) { + changed = true; + continue; + } + if (validReset(window.resetAt) && window.resetAt <= now) { + changed = true; + continue; + } + if (window.resetAt !== undefined && !validReset(window.resetAt)) { + const normalized = { ...window }; + delete normalized.resetAt; + retained.push(normalized); + changed = true; + } else { + retained.push(window); + } + } + if (changed) { + if (result === quota) result = { ...quota }; + if (retained.length) result.customWindows = retained; + else delete result.customWindows; + } + } + return hasQuotaRows(result) ? result : null; +} + +export const accountQuotaCache = new Map(); +export let explicitAccountEpoch = 0; + +/** + * Seed the cache from the last run, once. + * + * Without this a restart forgets every measurement, so the pool opens its next turn with + * no idea which account has room — the exact blindness pre-dispatch selection exists to + * remove. A hydrated row is still subject to the ordinary TTL, so it orders the first + * request and is replaced by a live probe immediately after. + */ +let diskHydrated = false; +export function hydrateAccountQuotaCache(): void { + if (diskHydrated) return; + diskHydrated = true; + for (const [key, quota] of readPersistedAccountQuotas()) { + // Disk stores observation time, not the Anthropic usage probe's clock. + if (!accountQuotaCache.has(key)) { + const anthropic = key.startsWith("anthropic\u0000"); + accountQuotaCache.set(key, { + ts: anthropic ? 0 : quota.updatedAt, + quota: anthropic ? normalizeAnthropicQuota(quota, Date.now()) : quota, + }); + } + } +} + +export function persistAccountQuotaCache(): void { + schedulePersistAccountQuotas(function* () { + const now = Date.now(); + for (const [key, entry] of accountQuotaCache) { + const quota = key.startsWith("anthropic\u0000") ? normalizeAnthropicQuota(entry.quota, now) : entry.quota; + if (quota) yield [key, quota] as [string, ProviderQuota]; + } + }); +} +export const accountQuotaInflight = new Map>(); +let lastReconciledGeneration = 0; +let liveAccountQuotaKeys = new Set(); +let liveProviderQuotaKeys = new Set(); + +export function mayCommitAccountQuotaKey(key: string, writerGeneration: number): boolean { + return writerGeneration >= lastReconciledGeneration || liveAccountQuotaKeys.has(key); +} + +export function mayCommitProviderQuotaKey(key: string, writerGeneration: number): boolean { + return writerGeneration >= lastReconciledGeneration || liveProviderQuotaKeys.has(key); +} + +export interface ProviderAccountQuota { + accountId: string; + quota: ProviderQuota | null; + /** Set when the probe could not reach upstream (expired login, 429, network). */ + unavailable?: true; + quotaFailure?: QuotaFailureCode; + quotaFailureIsCurrent?: () => boolean; + isCurrent?: () => boolean; +} + +/** Providers whose per-account quota can be probed. Extend as other OAuth APIs are covered. */ +export function supportsPerAccountQuota(provider: string): boolean { + return provider === "anthropic" || provider === "kiro" || provider === "google-antigravity" + || explicitAccountReader(provider); +} + +export function explicitAccountReader(provider: string): boolean { + return provider === "xai" || provider === "cursor" || provider === "kimi" || provider === "command-code"; +} + +export function providerOAuthAccountQuotaMode(provider: string): AccountQuotaMode { + return hasPassiveAccountQuota(provider) ? "passive" : supportsPerAccountQuota(provider) ? "probe" : "unsupported"; +} + +export function accountCacheKey(provider: string, accountId: string): string { + return `${provider}\u0000${accountId}`; +} + +/** + * Synchronous last-good per-account quota read for routing. Never probes the network. + * Returns null when nothing is cached (or the cached row has no bars). + */ +export function getCachedProviderAccountQuota(provider: string, accountId: string): ProviderQuota | null { + const entry = accountQuotaCache.get(accountCacheKey(provider, accountId)); + if (entry?.isCurrent && !entry.isCurrent()) return null; + return provider === "anthropic" ? normalizeAnthropicQuota(entry?.quota, Date.now()) : entry?.quota ?? null; +} + +/** Test-only: seed or clear the per-account quota cache without probing upstream. */ +export function setCachedProviderAccountQuotaForTests( + provider: string, + accountId: string, + quota: ProviderQuota | null, +): void { + const key = accountCacheKey(provider, accountId); + if (quota === null) { + accountQuotaCache.delete(key); + return; + } + accountQuotaCache.set(key, { ts: Date.now(), quota }); +} + +/** Unified headers report utilization fractions and epoch-second reset times. */ +function anthropicHeaderResetAt(value: string | null): number | undefined { + const seconds = toFiniteNumber(value); + if (seconds === undefined || seconds <= 0) return undefined; + const timestamp = seconds * 1000; + return Number.isFinite(new Date(timestamp).getTime()) ? timestamp : undefined; +} + +export function parseAnthropicRateLimitHeaders(headers: Headers): ProviderQuota | null { + const fiveHourPercent = normalizeUtilizationFraction(headers.get("anthropic-ratelimit-unified-5h-utilization")); + const weeklyPercent = normalizeUtilizationFraction(headers.get("anthropic-ratelimit-unified-7d-utilization")); + if (fiveHourPercent === undefined && weeklyPercent === undefined) return null; + const fiveHourResetAt = anthropicHeaderResetAt(headers.get("anthropic-ratelimit-unified-5h-reset")); + const weeklyResetAt = anthropicHeaderResetAt(headers.get("anthropic-ratelimit-unified-7d-reset")); + return { + ...(fiveHourPercent !== undefined ? { fiveHourPercent } : {}), + ...(fiveHourPercent !== undefined && fiveHourResetAt !== undefined ? { fiveHourResetAt } : {}), + ...(weeklyPercent !== undefined ? { weeklyPercent } : {}), + ...(weeklyPercent !== undefined && weeklyResetAt !== undefined ? { weeklyResetAt } : {}), + updatedAt: Date.now(), + }; +} + +/** Reject unknown scales; round fraction conversion for persisted/displayed percentages. */ +function normalizeUtilizationFraction(value: string | null): number | undefined { + const numeric = toFiniteNumber(value); + if (numeric === undefined || numeric < 0 || numeric > 1) return undefined; + return Math.round(numeric * 10_000) / 100; +} + +/** + * Merge serving-account observations without advancing the usage probe's clock or + * erasing model-specific windows. The caller owns credential attribution; this guard + * prevents a retired account key from being revived by an older config generation. + */ +export function recordAnthropicAccountQuotaFromHeaders( + accountId: string, + headers: Headers, + writerGeneration: number, +): void { + if (!accountId) return; + const observed = parseAnthropicRateLimitHeaders(headers); + if (!observed) return; + const key = accountCacheKey("anthropic", accountId); + if (!mayCommitAccountQuotaKey(key, writerGeneration)) return; + // Hydrate before writing, for the same reason `recordPassiveAccountQuota` does: this write + // arrives unprompted from the request path, and `persistAccountQuotaCache` serializes the + // whole map. Landing before any reader has hydrated would persist this single row and erase + // every other provider's saved row. + hydrateAccountQuotaCache(); + const previous = accountQuotaCache.get(key); + accountQuotaCache.set(key, { + ...previous, + // Headers do not prove that the last usage probe succeeded. + ts: previous?.ts ?? 0, + quota: normalizeAnthropicQuota({ + ...normalizeAnthropicQuota(previous?.quota, observed.updatedAt), ...observed, + }, observed.updatedAt), + }); + persistAccountQuotaCache(); +} + +/** + * Providers whose per-account quota is OBSERVED in-band, never probed. + * + * Deliberately separate from `supportsPerAccountQuota` rather than folded into it. That + * predicate gates explicit upstream readers. Meta publishes no quota endpoint, so it + * remains a cache-only observation even when every probe reader is account-scoped. + */ +export function hasPassiveAccountQuota(provider: string): boolean { + return provider === "meta-muse"; +} + +/** + * Record a quota observed in-band on a streaming turn. + * + * The CALLER captures `writerGeneration` when it resolves the serving credential, not + * this function at write time. A streaming turn is a long await, and a generation + * captured immediately before the write cannot see a config or account change that + * happened EARLIER in the same turn — which is exactly the case the fence exists for. + */ +export function recordPassiveAccountQuota( + provider: string, + accountId: string, + quota: ProviderQuota, + writerGeneration: number, +): void { + if (!hasPassiveAccountQuota(provider) || !accountId) return; + const key = accountCacheKey(provider, accountId); + if (!mayCommitAccountQuotaKey(key, writerGeneration)) return; + // Hydrate BEFORE writing, not only on the read path. `persistAccountQuotaCache` + // serializes the whole in-memory map, so a passive write that lands before anything + // has read the cache would persist this one row and erase every other provider's + // saved row -- and `diskHydrated` would then stop any later reader from recovering + // them. A probe writer cannot hit this because its own read hydrates first; an + // observation arrives unprompted, so it must hydrate itself. + hydrateAccountQuotaCache(); + accountQuotaCache.set(key, { ts: Date.now(), quota }); + // Persisted so a restart keeps the last observation: with no probe to re-establish it, + // a forgotten row stays forgotten until the user happens to run another streaming turn. + persistAccountQuotaCache(); + // sweepExpiredOnWrite is deliberately NOT called. Existing probe writers call it + // because they run on a poll; this runs on the request path, where a state sweep does + // not belong. Passive rows are still reclaimed by generation reconciliation + // (reconcileProviderAccountQuotaRows) and by the disk reader's age bound. +} + +/** + * Cache-only per-account rows for a passive provider. Never probes, never refreshes. + * + * An account with no observation is OMITTED rather than returned with `quota: null` and + * `unavailable`: that pair means "a probe was attempted and failed", and no probe was + * ever attempted here. A user who has not yet run a streaming turn simply has no + * measurement, which is not an error state. + */ +export function readPassiveProviderAccountQuotas(provider: string): ProviderAccountQuota[] { + if (!hasPassiveAccountQuota(provider)) return []; + // Idempotent, and otherwise only reached from probe paths a passive provider never + // enters — without it a restart shows nothing until the next streaming turn, even + // though the row is sitting on disk. + hydrateAccountQuotaCache(); + const set = getAccountSet(provider); + if (!set) return []; + const rows: ProviderAccountQuota[] = []; + for (const account of set.accounts) { + const entry = accountQuotaCache.get(accountCacheKey(provider, account.id)); + if (entry?.quota) rows.push({ accountId: account.id, quota: entry.quota }); + } + return rows; +} + +export function sweepExpiredProviderAccountQuotaRows(now = Date.now()): number { + let removed = 0; + for (const [key, entry] of accountQuotaCache) { + // Anthropic observations extend retention, never the usage probe's eligibility clock. + const retainedAt = key.startsWith("anthropic\u0000") + ? Math.max(entry.ts, entry.quota?.updatedAt ?? 0) + : entry.ts; + if (retainedAt + ACCOUNT_QUOTA_TTL_MS > now) continue; + accountQuotaCache.delete(key); + removed += 1; + } + return removed; +} + +export function reconcileProviderAccountQuotaRows(context: GenerationContext): number { + if (context.generation <= lastReconciledGeneration) return 0; + let removed = 0; + for (const key of accountQuotaCache.keys()) { + if (context.oauthAccountKeys.has(key)) continue; + accountQuotaCache.delete(key); + removed += 1; + } + // Kiro exhaustion rows are keyed identically, so they retire with their quota row; a + // verdict outliving its account would hand the replacement a cooldown it never earned. + removed += reconcileKiroAccountUsageState(context.oauthAccountKeys); + const cachedReports = getProviderQuotaReportCache(); + if (cachedReports) { + const reports = cachedReports.response.reports.filter(report => context.providerNames.has(report.provider)); + removed += cachedReports.response.reports.length - reports.length; + setProviderQuotaReportCache({ ...cachedReports, response: { ...cachedReports.response, reports } }); + replaceCachedProviderQuotas(reports, routingEvidence); + } + liveAccountQuotaKeys = new Set(context.oauthAccountKeys); + liveProviderQuotaKeys = new Set(context.providerNames); + lastReconciledGeneration = context.generation; + return removed; +} + +/** Test-only reset so a direct reconcile call in one file cannot leak across files. */ +export function resetProviderQuotaReconcileStateForTests(): void { + lastReconciledGeneration = 0; + liveAccountQuotaKeys = new Set(); + liveProviderQuotaKeys = new Set(); +} + +/** Drop cached per-account rows (all, or just one provider's). */ +export function clearAccountQuotaCache(provider?: string): void { + explicitAccountEpoch += 1; + if (!provider) { + accountQuotaCache.clear(); + accountQuotaInflight.clear(); + clearKiroAccountUsageState(); + // A cleared cache must not be re-seeded from the file it was just cleared of, and any + // pending write of the old rows is abandoned. + diskHydrated = false; + cancelPendingAccountQuotaPersist(); + return; + } + const prefix = `${provider}\u0000`; + for (const key of [...accountQuotaCache.keys()]) { + if (key.startsWith(prefix)) accountQuotaCache.delete(key); + } + clearKiroAccountUsageState(prefix); + // Drop in-flight probes too so a late resolve cannot repopulate after logout/remove. + for (const key of [...accountQuotaInflight.keys()]) { + if (key.startsWith(prefix)) accountQuotaInflight.delete(key); + } + persistAccountQuotaCache(); +} + +/** + * Resolve a bearer for quota probing without silently adopting a newer global + * Claude CLI credential into a background multiauth slot. + * + * - Fresh stored access → use as-is (no refresh). + * - Active account with expired access → normal refresh path. + * - Background `local-cli` with expired access → fail closed (unavailable): + * `getValidAccessTokenForAccount` can persist a mismatched Claude CLI identity. + * - Background ordinary OAuth (`source !== "local-cli"`) → safe to refresh; + * Anthropic's lock only adopts disk credentials for `local-cli` rows. + */ +export async function getTokenForAccountQuotaProbe(provider: string, accountId: string): Promise { + const stored = getAccountCredential(provider, accountId); + if (!stored) throw new Error("account credential missing"); + if (stored.expires > Date.now() + ACCOUNT_TOKEN_SKEW_MS) return stored.access; + const activeId = getAccountSet(provider)?.activeAccountId; + if (activeId !== accountId && stored.source === "local-cli") { + throw new Error("background local-cli token expired; skip CLI-adopting refresh for quota probe"); + } + return getValidAccessTokenForAccount(provider, accountId); +} + +export function explicitQuotaConfig(provider: string, configured?: OcxProviderConfig): OcxProviderConfig | undefined { + if (configured) return configured; + const entry = getProviderRegistryEntry(provider); + return entry ? { adapter: entry.adapter, baseUrl: entry.baseUrl, authMode: "oauth" } : undefined; +} + +export function explicitQuotaIdentity(provider: string, accountId: string, configured?: OcxProviderConfig): string | undefined { + const credential = getAccountCredential(provider, accountId); + const target = explicitQuotaConfig(provider, configured); + if (!credential || !target) return undefined; + return quotaCredentialIdentity(provider, accountId, credential, target); +} + +export function quotaCredentialIdentity(provider: string, accountId: string, credential: NonNullable>, target: OcxProviderConfig): string { + return createHash("sha256").update(JSON.stringify([ + provider, accountId, credential.access, credential.refresh, credential.expires, + credential.accountId, credential.projectId, credential.source, + target.adapter, target.baseUrl, target.authMode, target.disabled === true, + ])).digest("hex"); +} + +export function explicitQuotaDestination(provider: string, config: OcxProviderConfig): boolean { + if (config.disabled === true || config.authMode !== "oauth") return false; + if (provider === "kimi") return isCanonicalKimiCodeBaseUrl(config.baseUrl); + if (provider === "command-code") return isCanonicalCommandCodeBaseUrl(config.baseUrl); + // These readers use fixed canonical billing origins, never config.baseUrl. + return provider === "xai" || provider === "cursor"; +} diff --git a/src/providers/quota/antigravity.ts b/src/providers/quota/antigravity.ts new file mode 100644 index 0000000000..bafdc1eee0 --- /dev/null +++ b/src/providers/quota/antigravity.ts @@ -0,0 +1,295 @@ +import { antigravityUserAgent } from "../../adapters/client-fingerprint"; +import { DestinationDnsResolutionError } from "../../lib/destination-policy"; +import { PinnedHttpError } from "../../lib/pinned-http"; +import { ProviderOutboundPolicyError, providerOutboundPost, providerRedirectError, type ProviderOutboundDependencies } from "../../lib/provider-outbound"; +import { getValidAccessToken } from "../../oauth"; +import { getAccountCredential, getCredential } from "../../oauth/store"; +import { asRecord, normalizePercent, normalizeResetAt, readQuotaJson, REQUEST_TIMEOUT_MS, toFiniteNumber } from "../quota-wire"; +import { report, type ProviderQuotaReport } from "./report-cache"; +import { quotaCredentialIdentity } from "./account-cache"; +import type { ProviderQuota, ProviderQuotaWindow, QuotaFailureCode } from "../quota-types"; + +export function antigravityQuotaDiagnosticIdentity(accountId: string, credential = getAccountCredential("google-antigravity", accountId)): string | undefined { + return credential ? quotaCredentialIdentity("google-antigravity", accountId, credential, { + adapter: "google", baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE, authMode: "oauth", + }) : undefined; +} + + +function quotaInfoEntries(modelInfo: Record): Record[] { + const entries: Record[] = []; + const add = (value: unknown, tier?: string) => { + const rec = asRecord(value); + if (!rec) return; + entries.push(tier ? { ...rec, tier } : rec); + }; + const addArray = (value: unknown) => { + if (!Array.isArray(value)) return; + for (const entry of value) add(entry); + }; + + if (Array.isArray(modelInfo.quotaInfo)) addArray(modelInfo.quotaInfo); + else add(modelInfo.quotaInfo); + addArray(modelInfo.quotaInfos); + + const byTier = asRecord(modelInfo.quotaInfoByTier); + if (byTier) { + for (const [tier, value] of Object.entries(byTier)) { + if (Array.isArray(value)) { + for (const entry of value) add(entry, tier); + } else { + add(value, tier); + } + } + } + return entries; +} + +function classifyAntigravityFamily(modelId: string, modelInfo: Record, quotaInfo: Record): "Gem" | "Cla" | null { + const displayName = typeof modelInfo.displayName === "string" ? modelInfo.displayName : ""; + const tier = typeof quotaInfo.tier === "string" ? quotaInfo.tier : ""; + const haystack = `${modelId} ${displayName} ${tier}`.toLowerCase(); + if (haystack.includes("gemini")) return "Gem"; + if (haystack.includes("claude") || haystack.includes("opus") || haystack.includes("sonnet") || haystack.includes("gpt-oss") || haystack.includes("gpt_oss")) return "Cla"; + return null; +} + +function antigravityUsedPercent(quotaInfo: Record): number | undefined { + const target = asRecord(quotaInfo.remaining) ?? quotaInfo; + const remaining = normalizePercent(toFiniteNumber(target.remainingFraction) !== undefined + ? toFiniteNumber(target.remainingFraction)! * 100 + : toFiniteNumber(target.remainingPercentage) !== undefined + ? toFiniteNumber(target.remainingPercentage)! * 100 + : undefined); + if (remaining === undefined) return undefined; + return normalizePercent(100 - remaining); +} + +/** Gem/Cla windows from a `fetchAvailableModels` body; shared by the provider and account probes. */ +function antigravityWindowsFromModels(body: Record | null): ProviderQuotaWindow[] { + const models = asRecord(body?.models); + if (!models) return []; + + const windows = new Map(); + for (const [modelId, rawModelInfo] of Object.entries(models)) { + const modelInfo = asRecord(rawModelInfo); + if (!modelInfo) continue; + for (const quotaInfo of quotaInfoEntries(modelInfo)) { + const label = classifyAntigravityFamily(modelId, modelInfo, quotaInfo); + if (!label || windows.has(label)) continue; + const percent = antigravityUsedPercent(quotaInfo); + if (percent === undefined) continue; + windows.set(label, { + label, + percent, + ...(normalizeResetAt(quotaInfo.resetTime) !== undefined ? { resetAt: normalizeResetAt(quotaInfo.resetTime) } : {}), + }); + } + } + + const customWindows = ["Gem", "Cla"].flatMap(label => { + const window = windows.get(label); + return window ? [window] : []; + }); + return customWindows; +} + +/** + * Parse Google Antigravity quota from `v1internal:retrieveUserQuotaSummary`. + * Groups contain Gemini models and Claude/3P models, each with 5h and weekly limit buckets. + */ +function parseAntigravityQuotaSummary(body: Record | null): ProviderQuota | null { + const groups = Array.isArray(body?.groups) ? (body.groups as unknown[]) : []; + if (groups.length === 0) return null; + + const customWindowsMap = new Map(); + + for (const rawGroup of groups) { + const group = asRecord(rawGroup); + if (!group) continue; + const groupName = `${typeof group.displayName === "string" ? group.displayName : ""} ${typeof group.description === "string" ? group.description : ""}`.toLowerCase(); + const isGemini = groupName.includes("gemini"); + const isClaude = groupName.includes("claude") || groupName.includes("3p") || groupName.includes("gpt"); + + const buckets = Array.isArray(group.buckets) ? (group.buckets as unknown[]) : []; + for (const rawBucket of buckets) { + const bucket = asRecord(rawBucket); + if (!bucket) continue; + const windowStr = `${typeof bucket.window === "string" ? bucket.window : ""} ${typeof bucket.bucketId === "string" ? bucket.bucketId : ""} ${typeof bucket.displayName === "string" ? bucket.displayName : ""}`.toLowerCase(); + const percent = antigravityUsedPercent(bucket); + if (percent === undefined) continue; + const resetAt = normalizeResetAt(bucket.resetTime); + + const isWeekly = windowStr.includes("week"); + const is5h = windowStr.includes("5h") || windowStr.includes("five"); + + if (isGemini) { + const label = is5h ? "Gem" : isWeekly ? "Gem (Weekly)" : ""; + if (label && !customWindowsMap.has(label)) { + customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); + } + } else if (isClaude) { + const label = is5h ? "Cla" : isWeekly ? "Cla (Weekly)" : ""; + if (label && !customWindowsMap.has(label)) { + customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); + } + } else { + const baseLabel = typeof group.displayName === "string" ? group.displayName : "Other"; + const label = isWeekly ? `${baseLabel} (Weekly)` : baseLabel; + if (!customWindowsMap.has(label)) { + customWindowsMap.set(label, { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }); + } + } + } + } + + const PREFERRED_ORDER = ["Gem", "Gem (Weekly)", "Cla", "Cla (Weekly)"]; + const customWindows = Array.from(customWindowsMap.values()).sort((a, b) => { + const ia = PREFERRED_ORDER.indexOf(a.label); + const ib = PREFERRED_ORDER.indexOf(b.label); + if (ia !== -1 && ib !== -1) return ia - ib; + if (ia !== -1) return -1; + if (ib !== -1) return 1; + return a.label.localeCompare(b.label); + }); + + if (customWindows.length === 0) { + return null; + } + + return { + customWindows, + updatedAt: Date.now(), + }; +} + +const ANTIGRAVITY_ACCOUNT_QUOTA_BASE = "https://daily-cloudcode-pa.googleapis.com"; +const ANTIGRAVITY_QUOTA_SUMMARY_URL = `${ANTIGRAVITY_ACCOUNT_QUOTA_BASE}/v1internal:retrieveUserQuotaSummary`; +const ANTIGRAVITY_QUOTA_MODELS_URL = `${ANTIGRAVITY_ACCOUNT_QUOTA_BASE}/v1internal:fetchAvailableModels`; + +/** Only these fixed accounting destinations may use transparent Fake-IP DNS. */ +export function isCanonicalAntigravityQuotaUrl(name: string, url: string): boolean { + return name === "google-antigravity" + && (url === ANTIGRAVITY_QUOTA_SUMMARY_URL || url === ANTIGRAVITY_QUOTA_MODELS_URL); +} + +let antigravityOutboundDependencies: ProviderOutboundDependencies = { + isCanonicalUrl: isCanonicalAntigravityQuotaUrl, +}; + +/** Test seam: inject resolver/pinned transport for provider and per-account probes. */ +export function setAntigravityAccountQuotaTransportForTests(dependencies: ProviderOutboundDependencies | null): void { + antigravityOutboundDependencies = { ...dependencies, isCanonicalUrl: isCanonicalAntigravityQuotaUrl }; +} + +/** + * Per-account Antigravity quota (#1082). Always probes Google's own Cloud Code Assist host + * through the pinned provider-outbound transport: a configured `baseUrl` is a routing choice + * for requests, not a second source of Google's accounting for a stored credential, and fixing + * the destination keeps the `provider\0accountId` cache identity exact across config changes. + * A redirect or non-2xx yields null (unavailable), never a partial row. + */ +type AntigravityQuotaProbeResult = + | { kind: "available"; quota: ProviderQuota; source: "google-antigravity:retrieveUserQuotaSummary" | "google-antigravity:fetchAvailableModels" } + | { kind: "unavailable"; failure: QuotaFailureCode; legacy: { kind: "null" } | { kind: "throw"; error: unknown } }; + +function quotaTransportFailure(error: unknown): QuotaFailureCode { + if (error instanceof ProviderOutboundPolicyError) return "destination_blocked"; + if (error instanceof DestinationDnsResolutionError) return "dns_failed"; + if (error instanceof PinnedHttpError) return error.code === "output_byte_limit" ? "response_unusable" : "timeout"; + if (error instanceof DOMException && error.name === "TimeoutError") return "timeout"; + return "transport_error"; +} + +function quotaHttpFailure(status: number): QuotaFailureCode { + if (status >= 300 && status < 400) return "redirect_blocked"; + if (status === 401 || status === 403) return "access_denied"; + if (status === 429) return "rate_limited"; + return "upstream_error"; +} + +function unavailableAntigravityQuota(failure: QuotaFailureCode): AntigravityQuotaProbeResult { + return { kind: "unavailable", failure, legacy: { kind: "null" } }; +} + +/** + * Prefer a summary network-policy diagnosis over a vaguer fallback. A blocked + * destination is an actionable local-network fact, while "upstream_error" tells + * the operator to go look at Google. A successful models probe still clears + * the first failure completely. + */ +function antigravityUnavailableFailure( + summaryFailure: QuotaFailureCode | undefined, + fallbackFailure: QuotaFailureCode, +): QuotaFailureCode { + if ( + (summaryFailure === "destination_blocked" || summaryFailure === "dns_failed") + && fallbackFailure !== "destination_blocked" + && fallbackFailure !== "dns_failed" + ) { + return summaryFailure; + } + return fallbackFailure; +} + +export async function probeAntigravityUsageQuota(accessToken: string, projectId: string): Promise { + const fetchQuota = (url: string) => providerOutboundPost("google-antigravity", { baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE }, url, { + headers: { + Accept: "application/json", "Content-Type": "application/json", + "User-Agent": antigravityUserAgent(), Authorization: `Bearer ${accessToken}`, + }, + body: JSON.stringify({ project: projectId }), signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }, antigravityOutboundDependencies); + let summaryFailure: QuotaFailureCode | undefined; + try { + const response = await fetchQuota(ANTIGRAVITY_QUOTA_SUMMARY_URL); + if (await providerRedirectError(response, ANTIGRAVITY_QUOTA_SUMMARY_URL)) return unavailableAntigravityQuota("redirect_blocked"); + if (response.status === 401 || response.status === 403) return unavailableAntigravityQuota("access_denied"); + if (response.ok) { + const quota = parseAntigravityQuotaSummary(asRecord(await readQuotaJson(response))); + if (quota) return { kind: "available", quota, source: "google-antigravity:retrieveUserQuotaSummary" }; + } + } catch (error) { + // Existing behavior: summary transport/parse failure may recover through the models probe. + summaryFailure = quotaTransportFailure(error); + } + try { + const response = await fetchQuota(ANTIGRAVITY_QUOTA_MODELS_URL); + if (await providerRedirectError(response, ANTIGRAVITY_QUOTA_MODELS_URL)) { + return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, "redirect_blocked")); + } + if (!response.ok) { + return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, quotaHttpFailure(response.status))); + } + const customWindows = antigravityWindowsFromModels(asRecord(await readQuotaJson(response))); + if (!customWindows.length) { + return unavailableAntigravityQuota(antigravityUnavailableFailure(summaryFailure, "response_unusable")); + } + return { kind: "available", quota: { customWindows, updatedAt: Date.now() }, source: "google-antigravity:fetchAvailableModels" }; + } catch (error) { + // The public compatibility wrapper still rejects this exact fallback error; it never enters a DTO. + return { + kind: "unavailable", + failure: antigravityUnavailableFailure(summaryFailure, quotaTransportFailure(error)), + legacy: { kind: "throw", error }, + }; + } +} + +export async function fetchAntigravityUsageQuota(accessToken: string, projectId: string): Promise { + const result = await probeAntigravityUsageQuota(accessToken, projectId); + if (result.kind === "available") return result.quota; + if (result.legacy.kind === "throw") throw result.legacy.error; + return null; +} + +export async function fetchAntigravityQuota(provider: string): Promise { + const credential = getCredential("google-antigravity"); + if (!credential?.projectId) return null; + let accessToken: string; + try { accessToken = await getValidAccessToken("google-antigravity"); } catch { return null; } + const result = await probeAntigravityUsageQuota(accessToken, credential.projectId); + if (result.kind === "available") return report(provider, result.source, result.quota); + if (result.legacy.kind === "throw") throw result.legacy.error; + return null; +} diff --git a/src/providers/quota/report-cache.ts b/src/providers/quota/report-cache.ts new file mode 100644 index 0000000000..44010ebbf7 --- /dev/null +++ b/src/providers/quota/report-cache.ts @@ -0,0 +1,320 @@ +import { createHash } from "node:crypto"; +import { effectiveCodexAuthAccountId, listCodexAuthAccountsSnapshot } from "../../codex/auth-api"; +import { withoutRetiredCodexQuota, type StoredAccountQuota } from "../../codex/quota"; +import { isMainAccountIdentityGenerationLive } from "../../codex/main-account-cache"; +import { codexPlanKey } from "../../codex/plan"; +import { resolveProviderApiKey } from "../key-store"; +import { apiKeyPoolEntryId } from "../api-keys"; +import { getProviderRegistryEntry, providerCodexAccountMode } from "../registry"; +import { isCanonicalOpenAiForwardProvider, OPENAI_CODEX_PROVIDER_ID } from "../openai-tiers"; +import { CODEX_CAPACITY_MAX_QUOTA_AGE_MS, type CodexCapacityAggregation, type CodexCapacityQuota } from "../codex-capacity"; +import { clearCachedProviderQuotas, providerQuotaRoutingBinding, type ProviderQuotaRoutingEvidence } from "../quota-routing-cache"; +import { clearProviderApiKeyQuotaCache } from "../quota-key-accounts"; +import { QUOTA_JSON_READ_FAILURE, readQuotaJson } from "../quota-wire"; +import type { OcxConfig, OcxProviderConfig } from "../../types"; +import type { ProviderQuota, ProviderRoutingQuota } from "../quota-types"; + +/** Keep a failed probe's previous row at most this long before dropping it. */ +export const LAST_GOOD_MAX_AGE_MS = CODEX_CAPACITY_MAX_QUOTA_AGE_MS; +const nativeMainReportGenerations = new WeakMap(); +export const accountReportCurrent = new WeakMap boolean>(); +export const routingEvidence = new WeakMap(); +export let providerQuotaBeforePublishForTests: (() => void | Promise) | null = null; + +/** Test-only seam for identity/config invalidation after probes but before publication. */ +export function setProviderQuotaBeforePublishForTests( + hook: (() => void | Promise) | null, +): void { + providerQuotaBeforePublishForTests = hook; +} +export const TERMINAL_QUOTA_FAILURE = Symbol("terminal-quota-failure"); +/** + * The probe succeeded and the upstream authoritatively reported NO model-quota windows. + * + * Distinct from `null`, which means "this probe told us nothing" and deliberately preserves + * the last-good row for up to 30 minutes. Collapsing the two would let a stale report outlive + * the authoritative answer that replaced it: a GLM plan whose payload carries only MCP + * `TIME_LIMIT` rows has no model windows, and the dashboard and quota-aware routing must stop + * showing the previous token windows rather than keep them for another half hour. + * + * Suppression is shared with `TERMINAL_QUOTA_FAILURE`; only the reason differs. + */ +export const AUTHORITATIVE_EMPTY_QUOTA = Symbol("authoritative-empty-quota"); +export type ProviderQuotaProbeResult = + | ProviderQuotaReport + | null + | typeof TERMINAL_QUOTA_FAILURE + | typeof AUTHORITATIVE_EMPTY_QUOTA; + +export interface ProviderQuotaReport { + provider: string; + label: string; + source: string; + quota: ProviderQuota; + updatedAt: number; + /** Added by the management response projection, never stored on a cached report. */ + routingQuota?: ProviderRoutingQuota; + reverseEngineered?: boolean; + /** + * The row was OBSERVED in-band on a streaming turn rather than probed. + * + * Age means something different for these. A probed provider re-reads on its own TTL, + * so a row older than the last-good bound means the probe is failing and showing it + * would misrepresent a live number. A passive provider publishes no endpoint at all + * (`hasPassiveAccountQuota`), so its last observation is not a stale reading of + * something fresher — it is the only measurement that exists, and dropping it leaves + * the operator with nothing. Consumers that enforce a freshness bound must exempt + * these and state the observation age instead. + */ + observed?: boolean; + aggregation?: CodexCapacityAggregation; +} + +export interface ProviderQuotaResponse { + generatedAt: number; + reports: ProviderQuotaReport[]; +} + +let cache: { key: string; ts: number; response: ProviderQuotaResponse } | null = null; +export const inflight = new Map }>(); +/** Bumped on cache clear and on force-refresh start; stale-epoch probes lose commit authority. */ +export let invalidationEpoch = 0; + +/** Owner-module accessors: cache reassignment stays inside this file. */ +export function getProviderQuotaReportCache(): { key: string; ts: number; response: ProviderQuotaResponse } | null { + return cache; +} + +export function setProviderQuotaReportCache(next: { key: string; ts: number; response: ProviderQuotaResponse } | null): void { + cache = next; +} + +export function bumpProviderQuotaInvalidationEpoch(): void { + invalidationEpoch += 1; +} + +/** Invalidate the report cache (e.g. after switching a provider's active account). */ +export function clearProviderQuotaCache(): void { + cache = null; + clearCachedProviderQuotas(); + clearProviderApiKeyQuotaCache(); + invalidationEpoch += 1; +} + +function cacheKey(config: OcxConfig): string { + const providers = Object.entries(config.providers) + .map(([name, provider]) => { + const resolvedKey = typeof provider.apiKey === "string" + ? resolveProviderApiKey(provider.apiKey)?.trim() + : undefined; + const activeKeyId = resolvedKey ? apiKeyPoolEntryId(resolvedKey) : "none"; + return `${name}:${provider.adapter}:${provider.authMode ?? "key"}:${providerCodexAccountMode(name, provider) ?? "none"}:${provider.disabled === true ? "off" : "on"}:${provider.baseUrl}:${activeKeyId}`; + }) + .sort() + .join("|"); + return `${config.defaultProvider}|${providers}`; +} + +export type CodexAuthAccountsSnapshotPromise = ReturnType; + +export function hasCodexPoolProvider(config: OcxConfig): boolean { + return Object.entries(config.providers).some(([name, provider]) => ( + provider.disabled !== true + && isBuiltInChatGptForwardProvider(name, provider) + && providerCodexAccountMode(name, provider) !== "direct" + )); +} + +function quotaSignatureValue(quota: CodexCapacityQuota | null): unknown { + if (!quota) return null; + return { + fiveHourPercent: quota.fiveHourPercent, + fiveHourResetAt: quota.fiveHourResetAt, + weeklyPercent: quota.weeklyPercent, + weeklyResetAt: quota.weeklyResetAt, + monthlyPercent: quota.monthlyPercent, + monthlyResetAt: quota.monthlyResetAt, + updatedAt: quota.updatedAt, + customWindows: [...(quota.customWindows ?? [])] + .map(window => ({ label: window.label, percent: window.percent, resetAt: window.resetAt })) + .sort((a, b) => a.label.localeCompare(b.label)), + }; +} + +export function providerQuotaFromCodexQuota( + quota: StoredAccountQuota | Omit | null | undefined, +): CodexCapacityQuota | null { + if (!quota) return null; + // Direct snapshots bypass account DTOs; sanitize here as well as at ingestion. + quota = withoutRetiredCodexQuota(quota); + if (!quota) return null; + const projected: CodexCapacityQuota = { + ...(quota.shortPercent !== undefined ? { fiveHourPercent: quota.shortPercent } : {}), + ...(quota.shortResetAt !== undefined ? { fiveHourResetAt: quota.shortResetAt } : {}), + ...(quota.weeklyPercent !== undefined ? { weeklyPercent: quota.weeklyPercent } : {}), + ...(quota.weeklyResetAt !== undefined ? { weeklyResetAt: quota.weeklyResetAt } : {}), + ...(quota.monthlyPercent !== undefined ? { monthlyPercent: quota.monthlyPercent } : {}), + ...(quota.monthlyResetAt !== undefined ? { monthlyResetAt: quota.monthlyResetAt } : {}), + ...(quota.customWindows !== undefined ? { customWindows: quota.customWindows } : {}), + updatedAt: "updatedAt" in quota ? quota.updatedAt : Date.now(), + }; + return hasQuotaRows(projected) ? projected : null; +} + +/** Hash only presentation-relevant state; account ids and email addresses never enter the key. */ +export function cacheKeyWithAggregationState( + config: OcxConfig, + prefetchedSnapshot?: CodexAuthAccountsSnapshotPromise, +): string | Promise { + const base = cacheKey(config); + if (!hasCodexPoolProvider(config)) return base; + return (async () => { + try { + const activeId = effectiveCodexAuthAccountId(config); + const snapshot = await (prefetchedSnapshot ?? listCodexAuthAccountsSnapshot(config, false)); + const rows = snapshot.accounts.map(account => ({ + isMain: account.isMain, + active: account.id === activeId, + plan: codexPlanKey(account.plan) ?? null, + paused: account.paused, + needsReauth: account.needsReauth === true, + quota: quotaSignatureValue(providerQuotaFromCodexQuota(account.quota)), + })); + const canonicalRows = rows.map(row => JSON.stringify(row)).sort(); + const digest = createHash("sha256").update(JSON.stringify(canonicalRows)).digest("hex").slice(0, 24); + return `${base}|codex-pool:${digest}`; + } catch { + return `${base}|codex-pool:unavailable`; + } + })(); +} + +function publicCapacityWindow(window: import("../codex-capacity").CodexCapacityWindowAggregation) { + const { totalWeight: _totalWeight, consumedWeight: _consumedWeight, remainingWeight: _remainingWeight, ...safe } = window; + return safe; +} + +/** Management API metadata intentionally omits configured/weighted unit counts. */ +export function publicCapacityAggregation( + aggregation: CodexCapacityAggregation, + presentation: NonNullable, +): CodexCapacityAggregation { + const safeCurrentAccount = presentation === "coverage-only" && aggregation.currentAccount + ? { ...aggregation.currentAccount, quota: null } + : aggregation.currentAccount; + return { + ...aggregation, + presentation, + ...(safeCurrentAccount ? { currentAccount: safeCurrentAccount } : {}), + ...(aggregation.fiveHour ? { fiveHour: publicCapacityWindow(aggregation.fiveHour) } : {}), + ...(aggregation.weekly ? { weekly: publicCapacityWindow(aggregation.weekly) } : {}), + ...(aggregation.monthly ? { monthly: publicCapacityWindow(aggregation.monthly) } : {}), + ...(aggregation.customWindows ? { + customWindows: aggregation.customWindows.map(window => ({ + label: window.label, + ...publicCapacityWindow(window), + })), + } : {}), + }; +} + +export function hasQuotaRows(quota: ProviderQuota | null | undefined): quota is ProviderQuota { + if (!quota) return false; + return typeof quota.fiveHourPercent === "number" + || typeof quota.weeklyPercent === "number" + || typeof quota.monthlyPercent === "number" + || quota.creditsUsd?.unlimited === true + || typeof quota.creditsUsd?.percent === "number" + || !!quota.customWindows?.some(window => typeof window.percent === "number"); +} + +export function providerLabel(providerId: string): string { + return getProviderRegistryEntry(providerId)?.label ?? providerId; +} + +/** Test-only access to the quota reader's deadline and cancellation contract. */ +export async function readProviderQuotaJsonForTests(response: Response, timeoutMs: number): Promise { + const result = await readQuotaJson(response, timeoutMs); + return result === QUOTA_JSON_READ_FAILURE ? null : result; +} + +export function isBuiltInChatGptForwardProvider(name: string, provider: OcxProviderConfig): boolean { + return name === OPENAI_CODEX_PROVIDER_ID && isCanonicalOpenAiForwardProvider(provider); +} + +export function report( + provider: string, + source: string, + quota: ProviderQuota, + aggregation?: CodexCapacityAggregation, +): ProviderQuotaReport | null { + if (!hasQuotaRows(quota)) return null; + return { + provider, + label: providerLabel(provider), + source, + quota, + updatedAt: quota.updatedAt, + ...(aggregation ? { aggregation } : {}), + }; +} + +/** + * Publish a credential-bound report, and routing evidence only when the producer + * hands over its inference-only projection. + * + * The projection is deliberately not defaulted to the display quota. A producer must + * decide that its rows really do constrain inference on the probed credential; omitting + * the argument leaves the report display-only, so a new producer cannot inherit + * provider-veto authority merely by calling this helper. Ownership alone is not the + * scope decision: providerQuotaRoutingBinding resolving is necessary, never sufficient. + */ +export function keyReport( + provider: string, + source: string, + quota: ProviderQuota, + config: OcxProviderConfig, + probedCredential: string, + inferenceQuota?: ProviderQuota, +): ProviderQuotaReport | null { + const result = report(provider, source, quota); + if (!result || !inferenceQuota) return result; + const binding = providerQuotaRoutingBinding(provider, config, probedCredential); + if (binding) routingEvidence.set(result, { quota: inferenceQuota, binding }); + return result; +} + +export function tagNativeMainReport( + value: ProviderQuotaReport | null, + generation: number, +): ProviderQuotaReport | null { + if (value) nativeMainReportGenerations.set(value, generation); + return value; +} + +/** + * Test-only seam: publish exactly as a credential-bound producer does, and hand back the + * routing evidence the publication actually attached. + * + * Live producers all pass a projection today, so no probe fixture can prove the OTHER half + * of the contract: that omitting it stays display-only. Routing an omitted argument through + * the real helper keeps that provable, and a re-introduced `= quota` default would be + * observed here (a defaulted parameter also fires for an explicitly undefined argument). + */ +export function publishKeyReportForTests( + provider: string, + source: string, + quota: ProviderQuota, + config: OcxProviderConfig, + probedCredential: string, + inferenceQuota?: ProviderQuota, +): { report: ProviderQuotaReport | null; routing: ProviderQuotaRoutingEvidence | undefined } { + const result = keyReport(provider, source, quota, config, probedCredential, inferenceQuota); + return { report: result, routing: result ? routingEvidence.get(result) : undefined }; +} + +export function isProviderQuotaReportCurrent(value: ProviderQuotaReport): boolean { + const generation = nativeMainReportGenerations.get(value); + return (generation === undefined || isMainAccountIdentityGenerationLive(generation)) + && (accountReportCurrent.get(value)?.() ?? true); +} diff --git a/src/providers/quota/vendor-probes-key.ts b/src/providers/quota/vendor-probes-key.ts new file mode 100644 index 0000000000..17594f548e --- /dev/null +++ b/src/providers/quota/vendor-probes-key.ts @@ -0,0 +1,1243 @@ +import { resolveProviderApiKey } from "../key-store"; +import { getProviderRegistryEntry, registryEntryForProviderDestination } from "../registry"; +import { isCanonicalOllamaCloudUrl } from "../../adapters/ollama-native-url"; +import { QUOTA_JSON_READ_FAILURE, asRecord, normalizePercent, normalizeResetAt, readQuotaJson, REQUEST_TIMEOUT_MS, toFiniteNumber } from "../quota-wire"; +import { + AUTHORITATIVE_EMPTY_QUOTA, + hasQuotaRows, + keyReport, + report, + TERMINAL_QUOTA_FAILURE, + type ProviderQuotaProbeResult, + type ProviderQuotaReport, +} from "./report-cache"; +import { getTokenForAccountQuotaProbe } from "./account-cache"; +import type { AccountQuotaMode, ProviderQuota, ProviderQuotaCreditsUsd } from "../quota-types"; +import type { OcxProviderConfig } from "../../types"; + +const KIMI_CODE_BASE_URL = "https://api.kimi.com/coding/v1"; +const KIMI_CODE_USAGE_URL = `${KIMI_CODE_BASE_URL}/usages`; +const COMMAND_CODE_BASE_URL = "https://api.commandcode.ai"; +const COMMAND_CODE_WHOAMI_URL = `${COMMAND_CODE_BASE_URL}/alpha/whoami`; +const COMMAND_CODE_CREDITS_URL = `${COMMAND_CODE_BASE_URL}/alpha/billing/credits`; +const COMMAND_CODE_SUBSCRIPTIONS_URL = `${COMMAND_CODE_BASE_URL}/alpha/billing/subscriptions`; +const COMMAND_CODE_USAGE_URL = `${COMMAND_CODE_BASE_URL}/alpha/usage/summary`; +const A6API_BASE_URL = "https://api.a6api.com"; +const OPENCODE_GO_BASE_URL = "https://opencode.ai/zen/go/v1"; +const OPENCODE_GO_USAGE_URL = `${OPENCODE_GO_BASE_URL}/usage`; +const OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"; +const DEEPSEEK_BASE_URL = "https://api.deepseek.com"; +const CLINE_BASE_URL = "https://api.cline.bot"; +const OLLAMA_CLOUD_BASE_URL = "https://ollama.com"; +const OLLAMA_CLOUD_USAGE_URL = `${OLLAMA_CLOUD_BASE_URL}/api/usage`; +const ZAI_BASE_URL = "https://api.z.ai"; +const ZAI_CN_BASE_URL = "https://open.bigmodel.cn"; +const MINIMAX_REMAINS_URL = "https://www.minimax.io/v1/token_plan/remains"; +const MOONSHOT_BASE_URL = "https://api.moonshot.ai/v1"; +const VENICE_BASE_URL = "https://api.venice.ai/api/v1"; +const SYNTHETIC_BASE_URL = "https://api.synthetic.new/v2"; +const DEEPINFRA_BASE_URL = "https://api.deepinfra.com"; +const NEURALWATT_BASE_URL = "https://api.neuralwatt.com/v1"; + + +function isCanonicalA6apiBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === A6API_BASE_URL || normalized === `${A6API_BASE_URL}/v1`; +} + +function isCanonicalOpenCodeGoBaseUrl(baseUrl: string): boolean { + return normalizedBaseUrl(baseUrl) === OPENCODE_GO_BASE_URL; +} + +function isCanonicalOpenRouterBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === OPENROUTER_BASE_URL; +} + +function isCanonicalDeepSeekBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === DEEPSEEK_BASE_URL || normalized === `${DEEPSEEK_BASE_URL}/v1`; +} + +function isCanonicalClineBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === CLINE_BASE_URL || normalized === `${CLINE_BASE_URL}/api/v1`; +} + +function isCanonicalOllamaCloudBaseUrl(baseUrl?: string): boolean { + if (!baseUrl) return false; + try { + return isCanonicalOllamaCloudUrl(baseUrl); + } catch { + return false; + } +} + +function zaiQuotaMonitorHost(baseUrl: string): string | null { + // Admission and destination selection must share one mapping: admitting a new + // international wire must never fall through to the CN host/authentication scheme. + switch (normalizedBaseUrl(baseUrl)) { + case ZAI_BASE_URL: + case `${ZAI_BASE_URL}/api/coding/paas/v4`: + case `${ZAI_BASE_URL}/api/anthropic`: + case `${ZAI_BASE_URL}/api/v1`: + return ZAI_BASE_URL; + case ZAI_CN_BASE_URL: + case `${ZAI_CN_BASE_URL}/api/coding/paas/v4`: + case `${ZAI_CN_BASE_URL}/api/v1`: + return ZAI_CN_BASE_URL; + default: + return null; + } +} + +function isCanonicalZaiBaseUrl(baseUrl: string): boolean { + return zaiQuotaMonitorHost(baseUrl) !== null; +} + +function isCanonicalMinimaxBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === "https://api.minimax.io/v1" || normalized === "https://api.minimaxi.com/v1"; +} + +function isCanonicalMoonshotBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === MOONSHOT_BASE_URL || normalized === "https://api.moonshot.cn/v1"; +} + +function isCanonicalVeniceBaseUrl(baseUrl: string): boolean { + return normalizedBaseUrl(baseUrl) === VENICE_BASE_URL; +} + +function isCanonicalSyntheticBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === SYNTHETIC_BASE_URL || normalized === "https://api.synthetic.new/openai/v1"; +} + +function isCanonicalDeepInfraBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + return normalized === DEEPINFRA_BASE_URL || normalized === `${DEEPINFRA_BASE_URL}/v1/openai`; +} + +function isCanonicalNeuralwattBaseUrl(baseUrl: string): boolean { + return normalizedBaseUrl(baseUrl) === NEURALWATT_BASE_URL; +} + +function a6apiPayload(value: unknown): Record | null { + const body = asRecord(value); + return asRecord(body?.data) ?? body; +} + +function firstFinite(record: Record | null, names: string[]): number | undefined { + if (!record) return undefined; + for (const name of names) { + const value = toFiniteNumber(record[name]); + if (value !== undefined) return value; + } + return undefined; +} + +async function fetchA6apiQuota(provider: string, config: OcxProviderConfig): Promise { + // Never send a configured API key to a lookalike host or through a redirect. + if (!isCanonicalA6apiBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const headers = { Accept: "application/json", Authorization: `Bearer ${apiKey}` } as const; + const [subscriptionResponse, tokenResponse] = await Promise.all([ + fetch(`${A6API_BASE_URL}/dashboard/billing/subscription`, { + headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }), + fetch(`${A6API_BASE_URL}/api/usage/token/`, { + headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }), + ]); + if (!subscriptionResponse.ok || !tokenResponse.ok) { + const statuses = [subscriptionResponse.status, tokenResponse.status]; + // 408/429 are transient (timeout/throttle), not invalid-account signals: keep the + // last-good row like 5xx/network failures. 401/403 (bad key) and 404 (contract change) + // stay terminal. + return statuses.some(status => status >= 400 && status < 500 && status !== 429 && status !== 408) + ? TERMINAL_QUOTA_FAILURE + : null; + } + const [subscriptionBody, tokenBody] = await Promise.all([ + readQuotaJson(subscriptionResponse), + readQuotaJson(tokenResponse), + ]); + if (subscriptionBody === QUOTA_JSON_READ_FAILURE || tokenBody === QUOTA_JSON_READ_FAILURE) return null; + const subscription = a6apiPayload(subscriptionBody); + const token = a6apiPayload(tokenBody); + const unlimited = token?.unlimited_quota === true + || token?.unlimited_quota === 1 + || token?.unlimited_quota === "true"; + const normalizedExpiry = normalizeResetAt(token?.expires_at); + const expiry = normalizedExpiry && normalizedExpiry > 0 + ? { expiresAt: normalizedExpiry } + : {}; + if (unlimited) { + // Every row is an API-credit constraint on inference, so the display quota is also + // the routing projection. Passing it explicitly is the opt-in. + const quota: ProviderQuota = { + creditsUsd: { + used: 0, + limit: 0, + remaining: 0, + percent: 0, + unlimited: true, + ...expiry, + }, + customWindows: [{ label: "Unlimited API credits", percent: 0 }], + updatedAt: Date.now(), + }; + return keyReport(provider, "a6api:billing", quota, config, apiKey, quota); + } + const limitUsd = firstFinite(subscription, ["hard_limit_usd"]); + const grantedUnits = firstFinite(token, ["total_granted"]); + const usedUnits = firstFinite(token, ["total_used"]); + const availableUnits = firstFinite(token, ["total_available"]); + const reconciledUnits = usedUnits !== undefined && availableUnits !== undefined + ? usedUnits + availableUnits + : undefined; + const reconciliationTolerance = grantedUnits !== undefined + ? Math.abs(grantedUnits) * 1e-9 + : 0; + if (limitUsd === undefined || grantedUnits === undefined || usedUnits === undefined + || availableUnits === undefined || limitUsd <= 0 || grantedUnits <= 0 + || usedUnits < 0 || availableUnits < 0 + || reconciledUnits === undefined + || Math.abs(reconciledUnits - grantedUnits) > reconciliationTolerance) return TERMINAL_QUOTA_FAILURE; + const usdPerUnit = limitUsd / grantedUnits; + const usedUsd = usedUnits * usdPerUnit; + const remainingUsd = Math.max(0, availableUnits * usdPerUnit); + const percent = normalizePercent((usedUsd / limitUsd) * 100); + if (percent === undefined) return TERMINAL_QUOTA_FAILURE; + const label = `API credits ($${remainingUsd.toFixed(2)} of $${limitUsd.toFixed(2)} remaining)`; + const quota: ProviderQuota = { + creditsUsd: { + used: usedUsd, + limit: limitUsd, + remaining: remainingUsd, + percent, + ...expiry, + }, + customWindows: [{ label, percent }], + updatedAt: Date.now(), + }; + // The credit balance funds inference itself, so display and routing scope agree. + return keyReport(provider, "a6api:billing", quota, config, apiKey, quota); +} + +function parseOpenCodeGoUsageWindow(value: unknown): { percent: number; resetAt?: number } | null { + const row = asRecord(value); + if (!row) return null; + const percent = normalizePercent(row.percent); + if (percent === undefined) return null; + const resetAt = normalizeResetAt(row.resetsAt); + return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; +} + +async function fetchOpenCodeGoQuota(provider: string, config: OcxProviderConfig): Promise { + // Never send a configured API key when the provider destination is not the built-in Go endpoint. + if (!isCanonicalOpenCodeGoBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(OPENCODE_GO_USAGE_URL, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const usage = asRecord(body?.usage); + if (!usage) return null; + const rolling = parseOpenCodeGoUsageWindow(usage.rolling); + const weekly = parseOpenCodeGoUsageWindow(usage.weekly); + const monthly = parseOpenCodeGoUsageWindow(usage.monthly); + const quota: ProviderQuota = { + ...(rolling ? { + fiveHourPercent: rolling.percent, + ...(rolling.resetAt !== undefined ? { fiveHourResetAt: rolling.resetAt } : {}), + } : {}), + ...(weekly ? { + weeklyPercent: weekly.percent, + ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), + } : {}), + ...(monthly ? { + monthlyPercent: monthly.percent, + ...(monthly.resetAt !== undefined ? { monthlyResetAt: monthly.resetAt } : {}), + } : {}), + updatedAt: Date.now(), + }; + return keyReport(provider, "opencode-go:usage", quota, config, apiKey, quota); +} + +/** + * OpenRouter `GET /api/v1/key` — the key's own credit balance and optional + * per-key spending cap. `limit` is the configured cap (absent = uncapped); + * `usage` is lifetime spend; `limit_remaining` is what is left of the cap. + * When no cap is set there is no hard limit to meter against, so no bar is + * produced — the provider falls back to its documented reference. + */ +async function fetchOpenRouterQuota(provider: string, config: OcxProviderConfig): Promise { + // Never send a configured API key to a lookalike host or through a redirect. + if (!isCanonicalOpenRouterBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${OPENROUTER_BASE_URL}/key`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + if (!data) return null; + const limit = toFiniteNumber(data.limit); + const limitRemaining = toFiniteNumber(data.limit_remaining); + const usage = toFiniteNumber(data.usage); + // A successful no-cap response is a DELIBERATE change, not a transient + // failure: the old capped row must be dropped, not preserved as last-good. + if (limit === undefined || limit <= 0) return TERMINAL_QUOTA_FAILURE; + // Prefer the authoritative remaining-cap value when present: `usage` is + // lifetime accumulated spend and overstates a reset or re-capped key. + const used = limitRemaining !== undefined + ? Math.max(0, limit - limitRemaining) + : usage !== undefined && usage >= 0 ? usage : undefined; + if (used === undefined) return null; + const percent = normalizePercent((used / limit) * 100); + if (percent === undefined) return null; + const remaining = Math.max(0, limit - used); + const label = `API credits ($${remaining.toFixed(2)} of $${limit.toFixed(2)} remaining)`; + // The per-key spending cap stops every request this credential can make, so the + // whole report is inference-wide routing evidence. + const quota: ProviderQuota = { + customWindows: [{ label, percent }], + updatedAt: Date.now(), + }; + return keyReport(provider, "openrouter:key-info", quota, config, apiKey, quota); +} + +/** + * DeepSeek `GET /user/balance` — the account's granted + topped-up credit + * balance. The payload places `total_balance` / `granted_balance` inside + * entries of `balance_infos` (one row per currency); the row for the account's + * currency is selected by preference. `granted_balance` is a CURRENT balance + * component, not the original grant ceiling, so no consumed percentage is + * fabricated — the balance is reported as a balance-only window. + */ +async function fetchDeepSeekQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalDeepSeekBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${DEEPSEEK_BASE_URL}/user/balance`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + // The payload nests balances under `balance_infos` rows keyed by currency; + // prefer a USD row, then CNY, then the first row that parses. + const infos = Array.isArray(body?.balance_infos) ? body.balance_infos as unknown[] : null; + const rows = infos + ? infos.map((raw): Record | null => asRecord(raw)).filter((r): r is Record => r !== null) + : []; + const pick = (currency: string): Record | null => + rows.find(row => String(row.currency ?? "").toUpperCase() === currency) ?? null; + const preferred = pick("USD") ?? pick("CNY") ?? rows[0] ?? null; + if (!preferred) return null; + const totalBalance = toFiniteNumber(preferred.total_balance); + const grantedBalance = toFiniteNumber(preferred.granted_balance); + const toppedUp = toFiniteNumber(preferred.topped_up_balance); + const balance = totalBalance ?? grantedBalance ?? toppedUp; + if (balance === undefined || balance < 0) return null; + const label = grantedBalance !== undefined && grantedBalance > 0 + ? `API balance ($${balance.toFixed(2)} total, $${grantedBalance.toFixed(2)} granted)` + : `API balance ($${balance.toFixed(2)})`; + return report(provider, "deepseek:balance", { + customWindows: [{ label, percent: 0 }], + updatedAt: Date.now(), + }); +} + +/** + * ClinePass `GET /api/v1/users/me/plan/usage-limits` — the subscription's + * rolling five-hour, weekly, and monthly utilization, matching the existing + * ProviderQuota windows directly. The endpoint 404s (or returns a null plan) + * for accounts without an active ClinePass, which is a no-report, not an error. + */ +async function fetchClineQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalClineBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${CLINE_BASE_URL}/api/v1/users/me/plan/usage-limits`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + // 404 = no active plan; a plain "no plan" is a no-report, everything else + // 4xx (except 408/429) is a credential/contract problem. + if (response.status === 404) return null; + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + const limits = Array.isArray(data?.limits) ? data.limits : null; + if (!limits) return null; + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + for (const raw of limits) { + const row = asRecord(raw); + if (!row) continue; + const percent = normalizePercent(row.percentUsed); + if (percent === undefined) continue; + const resetAt = normalizeResetAt(row.resetsAt); + if (row.type === "five_hour") { + quota.fiveHourPercent = percent; + if (resetAt !== undefined) quota.fiveHourResetAt = resetAt; + windows += 1; + } else if (row.type === "weekly") { + quota.weeklyPercent = percent; + if (resetAt !== undefined) quota.weeklyResetAt = resetAt; + windows += 1; + } else if (row.type === "monthly") { + quota.monthlyPercent = percent; + if (resetAt !== undefined) quota.monthlyResetAt = resetAt; + windows += 1; + } + } + return windows > 0 ? keyReport(provider, "cline:plan-usage-limits", quota, config, apiKey, quota) : null; +} + +/** + * Ollama Cloud `GET https://ollama.com/api/usage` — returns account usage. + * Legacy plans report rolling 5-hour `limits.session.usage` and 7-day + * `limits.weekly.usage`. Migrated monthly-credit plans report + * `limits.monthly.usage`. `usage` values are normalized fractions (0..1). + */ +function parseOllamaPercent(usageValue: unknown): number | undefined { + const usage = toFiniteNumber(usageValue); + if (usage === undefined || usage < 0) return undefined; + const percent = Math.round(usage * 10000) / 100; + return normalizePercent(percent); +} + +export function parseOllamaCloudQuota(body: Record | null): ProviderQuota | null { + if (!body) return null; + const limits = asRecord(body.limits); + if (!limits) return null; + + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + + const session = asRecord(limits.session); + if (session) { + const percent = parseOllamaPercent(session.usage); + if (percent !== undefined) { + quota.fiveHourPercent = percent; + windows += 1; + } + } + + const weekly = asRecord(limits.weekly); + if (weekly) { + const percent = parseOllamaPercent(weekly.usage); + if (percent !== undefined) { + quota.weeklyPercent = percent; + windows += 1; + } + } + + const monthly = asRecord(limits.monthly); + if (monthly) { + const percent = parseOllamaPercent(monthly.usage); + if (percent !== undefined) { + quota.monthlyPercent = percent; + windows += 1; + } + } + + return windows > 0 ? quota : null; +} + +async function fetchOllamaCloudQuota(provider: string, config: OcxProviderConfig): Promise { + const effectiveBaseUrl = config.baseUrl ?? getProviderRegistryEntry(provider)?.baseUrl ?? ""; + if (!isCanonicalOllamaCloudBaseUrl(effectiveBaseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(OLLAMA_CLOUD_USAGE_URL, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + if (response.status === 404) return null; + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const quota = parseOllamaCloudQuota(body); + return quota ? keyReport(provider, "ollama-cloud:usage", quota, config, apiKey, quota) : null; +} + +/** + * Z.AI GLM Coding Plan `GET /api/monitor/usage/quota/limit` — the coding-plan + * limits arrive as a `limits` array of `TOKENS_LIMIT` (newer plans call the + * same rows `CREDIT_LIMIT`) and `TIME_LIMIT` rows. `TOKENS_LIMIT`/`CREDIT_LIMIT` + * rows carry the window length as `unit`/`number`: unit 3 is hours (number 5 → + * the rolling five-hour window), unit 6 is weeks (number 1 → the weekly + * window). Every row's `percentage` is the consumed share (falling + * back to `currentValue`/`usage` when absent) and `nextResetTime` (unix ms) + * the window reset. + * + * `TIME_LIMIT` rows are deliberately ignored (issue #1168). They are the shared + * monthly MCP *call* allowance for Web Search / Web Reader / Zread — not a + * model-token budget — and `ProviderQuota.monthlyPercent` is consumed as a + * model-capacity signal: `headroomOf()` in `src/oauth/account-quota-rank.ts` + * takes the MAX across every window, so a user who spent their MCP search + * allowance would be ranked as having no model capacity left, and the dashboard + * would draw a full monthly bar for a plan whose model tokens are untouched. + * A payload carrying only `TIME_LIMIT` rows therefore reports no quota at all, + * which is the honest answer rather than a fabricated one. + */ +export function parseZaiQuotaLimits(data: Record | null): ProviderQuota | null { + const limits = Array.isArray(data?.limits) ? data.limits as unknown[] : null; + if (!limits) return null; + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + for (const raw of limits) { + const row = asRecord(raw); + if (!row) continue; + // Gate on row type before deriving a percentage: an MCP row must not even + // contribute a parsed value to a model-quota report. + if (row.type !== "TOKENS_LIMIT" && row.type !== "CREDIT_LIMIT") continue; + const resetAt = normalizeResetAt(row.nextResetTime); + let percent = normalizePercent(row.percentage); + if (percent === undefined) { + const used = toFiniteNumber(row.currentValue); + const total = toFiniteNumber(row.usage); + if (used !== undefined && total !== undefined && total > 0) { + percent = normalizePercent((used / total) * 100); + } + } + if (percent === undefined) continue; + const unit = toFiniteNumber(row.unit); + const number = toFiniteNumber(row.number); + if (unit === 3 && number === 5) { + quota.fiveHourPercent = percent; + if (resetAt !== undefined) quota.fiveHourResetAt = resetAt; + windows += 1; + } else if (unit === 6 && number === 1) { + quota.weeklyPercent = percent; + if (resetAt !== undefined) quota.weeklyResetAt = resetAt; + windows += 1; + } + } + return windows > 0 ? quota : null; +} + +/** + * Legacy Z.AI payload shape: percent fields with window identifiers directly on + * the data object (optionally nested under `quota`). Kept as a fallback so + * older responses keep rendering when the `limits` array is absent. + */ +function parseZaiQuotaLegacyFields(data: Record | null): ProviderQuota | null { + if (!data) return null; + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + const percentAt = (key: string): number | undefined => { + const value = normalizePercent(data[key]); + if (value !== undefined) return value; + const nested = asRecord(data.quota); + return nested ? normalizePercent(nested[key]) : undefined; + }; + const fiveHour = percentAt("fiveHourPercent") ?? percentAt("fiveHourUsage") ?? percentAt("fiveHourUsed"); + const weekly = percentAt("weeklyPercent") ?? percentAt("weeklyUsage") ?? percentAt("weeklyUsed"); + const monthly = percentAt("monthlyPercent") ?? percentAt("mcpPercent") ?? percentAt("monthlyMCPUsage"); + if (fiveHour !== undefined) { + quota.fiveHourPercent = fiveHour; + windows += 1; + } + if (weekly !== undefined) { + quota.weeklyPercent = weekly; + windows += 1; + } + if (monthly !== undefined) { + quota.monthlyPercent = monthly; + windows += 1; + } + return windows > 0 ? quota : null; +} + +/** + * Fetches the Z.AI GLM Coding Plan quota — on whichever region the provider + * points at (api.z.ai or open.bigmodel.cn). The `limits` array shape is + * preferred; older field-name payloads fall back to the legacy parser. + * + * Authentication differs by host (issue #1168). `api.z.ai` takes the API key as + * a Bearer token per Z.AI's API reference; `open.bigmodel.cn` expects the key + * directly in `Authorization` with no scheme prefix and answers a Bearer header + * with an auth error, which is why BigModel Coding Plan quota never rendered. + * The host is already canonicalized by `isCanonicalZaiBaseUrl` above and + * `redirect: "error"` stays set, so the bare key cannot travel to a lookalike + * host or follow a redirect off-origin. + */ +async function fetchZaiQuota(provider: string, config: OcxProviderConfig): Promise { + const monitorHost = zaiQuotaMonitorHost(config.baseUrl); + if (!monitorHost) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const authorization = monitorHost === ZAI_CN_BASE_URL ? apiKey : `Bearer ${apiKey}`; + const response = await fetch(`${monitorHost}/api/monitor/usage/quota/limit`, { + headers: { Accept: "application/json", Authorization: authorization }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + if (!body || body.success === false) return null; + const data = asRecord(body.data) ?? body; + if (Array.isArray(data?.limits)) { + const quota = parseZaiQuotaLimits(data); + // A well-formed `limits[]` we fully understood is authoritative even when it yields no + // model window — for example a plan reporting only the monthly MCP `TIME_LIMIT` row. + // Returning `null` here would preserve the previous token windows for up to 30 minutes + // and keep quota-aware routing acting on a report the provider has already superseded. + return quota + ? keyReport(provider, "zai:quota-limit", quota, config, apiKey, quota) + : AUTHORITATIVE_EMPTY_QUOTA; + } + const legacy = parseZaiQuotaLegacyFields(data); + if (!legacy) return null; + // The legacy monthly figure also carries MCP usage; it is display evidence, not + // proof that model inference is unavailable. Modern TOKEN_LIMIT rows above are scoped. + const inferenceQuota = { ...legacy }; + delete inferenceQuota.monthlyPercent; + delete inferenceQuota.monthlyResetAt; + return keyReport(provider, "zai:quota-limit", legacy, config, apiKey, inferenceQuota); +} + +/** + * MiniMax Token Plan `GET /v1/token_plan/remains` — the subscription's + * remaining quota as a countdown-time value (ms). The endpoint does not expose + * the plan's total duration, so no percentage is fabricated from a presumed + * window: the remaining time is reported as a duration-only window. When the + * API supplies a total (`total_time` / `plan_duration_ms`), a consumed share + * is derived from it. Region selects the host: `minimax` → www.minimax.io, + * `minimax-cn` → api.minimaxi.com. + */ +async function fetchMinimaxQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalMinimaxBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const cnHost = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.minimaxi.com"); + const remainsUrl = cnHost ? "https://api.minimaxi.com/v1/token_plan/remains" : MINIMAX_REMAINS_URL; + const response = await fetch(remainsUrl, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + if (!body || body.success === false) return null; + const data = asRecord(body.data) ?? body; + const remainsMs = toFiniteNumber(data.remains_time ?? data.remainsTime); + if (remainsMs === undefined || remainsMs < 0) return null; + const hours = Math.floor(remainsMs / 3_600_000); + const label = `Token Plan remaining (${hours}h)`; + // Only derive a consumed share when the API actually reports the plan total; + // a presumed window (e.g. 30 days) would fabricate utilization. A valid + // response that omits the total after a prior refresh had it is a DELIBERATE + // contract change — the old row must be dropped (terminal), not preserved as + // a transient last-good. + const totalMs = toFiniteNumber(data.total_time ?? data.plan_duration_ms ?? data.total_duration_ms); + if (totalMs === undefined || totalMs <= 0) return TERMINAL_QUOTA_FAILURE; + const consumed = Math.max(0, totalMs - remainsMs); + const percent = normalizePercent((consumed / totalMs) * 100); + if (percent === undefined) return null; + return report(provider, "minimax:token-plan-remains", { + customWindows: [{ label, percent }], + updatedAt: Date.now(), + }); +} + +/** + * Moonshot/Kimi `GET /v1/users/me/balance` — the account's available balance + * (voucher + cash). Renders a single balance window against the sum of + * voucher + cash when positive (there is no per-window rate limit to meter). + */ +async function fetchMoonshotQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalMoonshotBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const host = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.moonshot.cn") ? "https://api.moonshot.cn/v1" : MOONSHOT_BASE_URL; + const response = await fetch(`${host}/users/me/balance`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + if (!data) return null; + const available = toFiniteNumber(data.available_balance); + const voucher = toFiniteNumber(data.voucher_balance); + const cash = toFiniteNumber(data.cash_balance); + if (available === undefined || available < 0) return null; + // Moonshot exposes no per-window quota ceiling, only a balance — report it + // as a balance-only window (percent 0) rather than a fabricated utilization. + // Currency is host-scoped: China platform (api.moonshot.cn) bills in CNY; + // the international platform (api.moonshot.ai) bills in USD. Do not force + // either side into the other unit — the number is correct, only the unit + // must match the host. + const isChinaHost = host.startsWith("https://api.moonshot.cn"); + const money = (n: number) => isChinaHost ? `¥${n.toFixed(2)}` : `$${n.toFixed(2)}`; + const unit = isChinaHost ? "CNY" : "USD"; + const label = voucher !== undefined && cash !== undefined + ? `Balance (${money(available)} ${unit} available, ${money(voucher)} voucher)` + : `Balance (${money(available)} ${unit} available)`; + return report(provider, "moonshot:balance", { + customWindows: [{ label, percent: 0 }], + updatedAt: Date.now(), + }); +} + +/** + * Venice `GET /api/v1/billing/balance` — DIEM (native credits) or USD balance. + * Shows the remaining balance; epoch allocation progress when present. + */ +async function fetchVeniceQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalVeniceBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${VENICE_BASE_URL}/billing/balance`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + if (!data) return null; + const diemBalance = toFiniteNumber(data.balance); + const usdBalance = toFiniteNumber(data.balance_usd); + const epochUsed = toFiniteNumber(data.diem_epoch_used); + const epochAllocated = toFiniteNumber(data.diem_epoch_allocated); + if (diemBalance === undefined && usdBalance === undefined) return null; + const label = diemBalance !== undefined + ? `DIEM balance (${Math.round(diemBalance)})` + : `USD balance ($${usdBalance?.toFixed(2) ?? "?"})`; + if (epochAllocated !== undefined && epochAllocated > 0 && epochUsed !== undefined) { + const percent = normalizePercent((epochUsed / epochAllocated) * 100); + if (percent === undefined) return null; + return report(provider, "venice:billing-balance", { + customWindows: [{ label, percent }], + updatedAt: Date.now(), + }); + } + return report(provider, "venice:billing-balance", { + customWindows: [{ label, percent: 0 }], + updatedAt: Date.now(), + }); +} + +/** + * Synthetic `GET /v2/quotas` — the known quota lanes (rolling 5-hour, + * weekly token, search-hourly) mapped onto the quota windows. + */ +async function fetchSyntheticQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalSyntheticBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${SYNTHETIC_BASE_URL}/quotas`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + const percentAt = (key: string): number | undefined => { + const value = normalizePercent(data?.[key]); + if (value !== undefined) return value; + const nested = asRecord(data?.quota) ?? asRecord(data?.quotas); + return nested ? normalizePercent(nested[key]) : undefined; + }; + const fiveHour = percentAt("rollingFiveHourLimit"); + const weekly = percentAt("weeklyTokenLimit"); + if (fiveHour !== undefined) { + quota.fiveHourPercent = fiveHour; + windows += 1; + } + if (weekly !== undefined) { + quota.weeklyPercent = weekly; + windows += 1; + } + const search = asRecord(data?.search); + const searchHourly = search ? normalizePercent(search.hourly) : undefined; + if (searchHourly !== undefined) { + quota.customWindows = [...(quota.customWindows ?? []), { label: "Search hourly", percent: searchHourly }]; + windows += 1; + } + const inferenceQuota = { ...quota }; + delete inferenceQuota.customWindows; // search.hourly does not constrain model inference. + return windows > 0 ? keyReport(provider, "synthetic:quotas", quota, config, apiKey, inferenceQuota) : null; +} + +/** + * DeepInfra `GET /payment/checklist?compute_owed=true` — prepaid balance, + * recent spend, spending limit, and suspension state. Renders a balance + * window (prepaid funds are a negative `stripe_balance` → positive available). + */ +async function fetchDeepInfraQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalDeepInfraBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${DEEPINFRA_BASE_URL}/payment/checklist?compute_owed=true`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + if (!data) return null; + const stripeBalance = toFiniteNumber(data.stripe_balance); + const spendLimit = toFiniteNumber(data.spending_limit); + const total = toFiniteNumber(data.total_amount_due); + if (stripeBalance === undefined) return null; + // Prepaid funds are negative; a positive value is money owed. + const available = stripeBalance < 0 ? -stripeBalance : 0; + if (spendLimit !== undefined && spendLimit > 0) { + const spent = total !== undefined && total > 0 ? total : Math.max(0, spendLimit - available); + const percent = normalizePercent((spent / spendLimit) * 100); + if (percent === undefined) return null; + return report(provider, "deepinfra:billing-checklist", { + customWindows: [{ label: `Billing cycle spend ($${spent.toFixed(2)} of $${spendLimit.toFixed(2)})`, percent }], + updatedAt: Date.now(), + }); + } + return report(provider, "deepinfra:billing-checklist", { + customWindows: [{ label: `Prepaid balance ($${available.toFixed(2)})`, percent: 0 }], + updatedAt: Date.now(), + }); +} + +/** + * Neuralwatt `GET /v1/quota` — subscription kWh usage (primary window) and + * prepaid USD credit balance (secondary). + */ +async function fetchNeuralwattQuota(provider: string, config: OcxProviderConfig): Promise { + if (!isCanonicalNeuralwattBaseUrl(config.baseUrl)) return null; + const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); + if (!apiKey) return null; + const response = await fetch(`${NEURALWATT_BASE_URL}/quota`, { + headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const body = asRecord(await readQuotaJson(response)); + const data = asRecord(body?.data) ?? body; + const quota: ProviderQuota = { updatedAt: Date.now() }; + let windows = 0; + const subscription = asRecord(data?.subscription); + const kwhUsed = subscription ? toFiniteNumber(subscription.kwh_used) : undefined; + const kwhIncluded = subscription ? toFiniteNumber(subscription.kwh_included) : undefined; + if (kwhUsed !== undefined && kwhIncluded !== undefined && kwhIncluded > 0) { + const percent = normalizePercent((kwhUsed / kwhIncluded) * 100); + if (percent !== undefined) { + quota.fiveHourPercent = percent; + const periodEnd = subscription ? normalizeResetAt(subscription.current_period_end) : undefined; + if (periodEnd !== undefined) quota.fiveHourResetAt = periodEnd; + windows += 1; + } + } + const balance = asRecord(data?.balance); + const totalCredits = balance ? toFiniteNumber(balance.total_credits_usd) : undefined; + const remainingCredits = balance ? toFiniteNumber(balance.credits_remaining_usd) : undefined; + if (totalCredits !== undefined && totalCredits > 0 && remainingCredits !== undefined) { + // Utilization is CONSUMED credits, not the remaining share. + const used = Math.max(0, totalCredits - remainingCredits); + const percent = normalizePercent((used / totalCredits) * 100); + if (percent !== undefined) { + quota.customWindows = [...(quota.customWindows ?? []), { label: "Prepaid credits", percent }]; + windows += 1; + } + } + return windows > 0 ? report(provider, "neuralwatt:quota", quota) : null; +} + + +function normalizedBaseUrl(value: string): string | null { + try { + const url = new URL(value); + if (url.username || url.password || url.search || url.hash) return null; + return `${url.origin.toLowerCase()}${url.pathname.replace(/\/+$/, "")}`; + } catch { + return null; + } +} + +function quotaResetAt(row: Record): number | undefined { + return normalizeResetAt(row.resetTime ?? row.resetAt ?? row.reset_time ?? row.reset_at); +} + +export function isCanonicalKimiCodeBaseUrl(baseUrl: string): boolean { + return normalizedBaseUrl(baseUrl) === KIMI_CODE_BASE_URL; +} + +export function isCanonicalCommandCodeBaseUrl(baseUrl: string): boolean { + const normalized = normalizedBaseUrl(baseUrl); + // OAuth preset points at the API root; the Provider-API preset at /provider/v1. + return normalized === COMMAND_CODE_BASE_URL || normalized === `${COMMAND_CODE_BASE_URL}/provider/v1`; +} + +/** Prefer the nested `data` shell when the outer object is only an envelope. */ +function unwrapKimiQuotaPayload(value: unknown): Record | null { + const body = asRecord(value); + if (!body) return null; + const nested = asRecord(body.data); + if (!nested) return body; + // A null/non-usable outer field is a placeholder, not data — an envelope like + // { usage: null, data: { usage: {...} } } must still unwrap to the nested payload. + const usable = (field: unknown): boolean => field !== undefined && field !== null; + const outerHasUsage = usable(body.usage) || usable(body.limits) || usable(body.totalQuota); + const nestedHasUsage = usable(nested.usage) || usable(nested.limits) || usable(nested.totalQuota); + return !outerHasUsage && nestedHasUsage ? nested : body; +} + +function kimiLimitLabel(item: Record, detail: Record): string { + return [item.name, item.title, item.scope, detail.name, detail.title] + .filter((value): value is string => typeof value === "string") + .join(" ") + .toLowerCase(); +} + +function parseKimiQuotaRow(value: unknown, resetFallback?: Record): { percent: number; resetAt?: number } | null { + const row = asRecord(value); + if (!row) return null; + const resetAt = quotaResetAt(row) ?? (resetFallback ? quotaResetAt(resetFallback) : undefined); + const limit = toFiniteNumber(row.limit); + if (limit !== undefined && limit > 0) { + let used = toFiniteNumber(row.used); + if (used === undefined) { + const remaining = toFiniteNumber(row.remaining); + if (remaining !== undefined) used = limit - remaining; + } + if (used !== undefined) { + const percent = normalizePercent((used / limit) * 100); + if (percent !== undefined) return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; + } + } + // Some payloads expose utilisation directly when limit/used arithmetic is absent. + const direct = normalizePercent(row.utilization ?? row.percent ?? row.usedPercent ?? row.used_percent); + return direct === undefined ? null : { percent: direct, ...(resetAt !== undefined ? { resetAt } : {}) }; +} + +function isKimiFiveHourLimit(item: Record, detail: Record, window: Record): boolean { + const duration = toFiniteNumber(window.duration ?? item.duration ?? detail.duration); + const unit = String(window.timeUnit ?? item.timeUnit ?? detail.timeUnit ?? "").toUpperCase(); + if ((unit.includes("MINUTE") && duration === 300) || (unit.includes("HOUR") && duration === 5)) return true; + return /(^|\b)5\s*(?:h|hour)/.test(kimiLimitLabel(item, detail)); +} + +function isKimiWeeklyLimit(item: Record, detail: Record, window: Record): boolean { + const duration = toFiniteNumber(window.duration ?? item.duration ?? detail.duration); + const unit = String(window.timeUnit ?? item.timeUnit ?? detail.timeUnit ?? "").toUpperCase(); + if ((unit.includes("DAY") && duration === 7) || (unit.includes("HOUR") && duration === 168)) return true; + return /weekly|7\s*(?:d|day)/.test(kimiLimitLabel(item, detail)); +} + +function parseKimiQuotaPayload(value: unknown): ProviderQuota | null { + const body = unwrapKimiQuotaPayload(value); + if (!body) return null; + let weekly = parseKimiQuotaRow(body.usage); + const total = parseKimiQuotaRow(body.totalQuota); + let fiveHour: { percent: number; resetAt?: number } | null = null; + if (Array.isArray(body.limits)) { + for (const rawItem of body.limits) { + const item = asRecord(rawItem); + if (!item) continue; + const detail = asRecord(item.detail) ?? item; + const window = asRecord(item.window) ?? {}; + if (!fiveHour && isKimiFiveHourLimit(item, detail, window)) { + fiveHour = parseKimiQuotaRow(detail, window); + } + if (!weekly && isKimiWeeklyLimit(item, detail, window)) { + weekly = parseKimiQuotaRow(detail, window); + } + if (fiveHour && weekly) break; + } + } + const quota: ProviderQuota = { + ...(fiveHour ? { + fiveHourPercent: fiveHour.percent, + ...(fiveHour.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), + } : {}), + ...(weekly ? { + weeklyPercent: weekly.percent, + ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), + } : {}), + ...(total ? { customWindows: [{ label: "Total subscription credits", percent: total.percent, ...(total.resetAt !== undefined ? { resetAt: total.resetAt } : {}) }] } : {}), + updatedAt: Date.now(), + }; + return hasQuotaRows(quota) ? quota : null; +} + +async function resolveKimiQuotaBearer(config: OcxProviderConfig, accountId?: string): Promise { + if (config.authMode === "oauth") { + try { + return accountId ? await getTokenForAccountQuotaProbe("kimi", accountId) : null; + } catch { + return null; + } + } + // ACTIVE key only: silently walking apiKeyPool when the primary env reference is + // unresolved would render a quota bar for a DIFFERENT account than the one routing + // requests — a wrong meter is worse than no meter. + const primary = resolveProviderApiKey(config.apiKey)?.trim(); + return primary || null; +} + +export async function fetchKimiQuota(provider: string, config: OcxProviderConfig, accessToken: string): Promise { + // Never release credentials to a user-edited or lookalike provider host. + if (!isCanonicalKimiCodeBaseUrl(config.baseUrl)) return null; + if (!accessToken) return null; + const response = await fetch(KIMI_CODE_USAGE_URL, { + headers: { Accept: "application/json", Authorization: `Bearer ${accessToken}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + const quota = parseKimiQuotaPayload(await readQuotaJson(response)); + return quota ? keyReport(provider, "kimi:usages", quota, config, accessToken, quota) : null; +} + +/** + * Command Code rolling window: `{ cap, used, resetAt }` off /alpha/billing/credits, + * normalized to a percent with an optional reset timestamp. + */ +function parseCommandCodeWindow(value: unknown): { percent: number; resetAt?: number } | null { + const row = asRecord(value); + if (!row) return null; + const cap = toFiniteNumber(row.cap); + const used = toFiniteNumber(row.used); + if (cap === undefined || used === undefined || cap <= 0 || used < 0) return null; + const percent = normalizePercent((used / cap) * 100); + if (percent === undefined) return null; + const resetAt = quotaResetAt(row); + return { percent, ...(resetAt !== undefined ? { resetAt } : {}) }; +} + +/** Soft-fail GET returning a parsed record, or null when unavailable. */ +async function fetchCommandCodeJson(url: string, bearer: string): Promise | null> { + try { + const response = await fetch(url, { + headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + return asRecord(await readQuotaJson(response)); + } catch { + return null; + } +} + +/** + * Soft-fail period spend (used) against the remaining credit pools → creditsUsd. + * Period scoping: `since=` keeps spend aligned with the + * pools' billing cycle, and `currentPeriodEnd` becomes expiresAt. + */ +async function fetchCommandCodeSpend( + bearer: string, + credits: Record | null, + orgQuery: string, +): Promise { + if (!credits) return undefined; + const subscriptionBody = await fetchCommandCodeJson(`${COMMAND_CODE_SUBSCRIPTIONS_URL}${orgQuery}`, bearer); + const subscription = asRecord(subscriptionBody?.data) ?? subscriptionBody; + const periodStart = typeof subscription?.currentPeriodStart === "string" ? subscription.currentPeriodStart.trim() : ""; + // Unscoped /usage/summary is lifetime spend; mixing it with current-cycle + // remaining pools produces a wrong percent. Omit creditsUsd until a period exists. + if (!periodStart) return undefined; + const sinceQuery = `${orgQuery ? "&" : "?"}since=${encodeURIComponent(periodStart)}`; + const expiresAt = normalizeResetAt(subscription?.currentPeriodEnd); + const summaryBody = await fetchCommandCodeJson(`${COMMAND_CODE_USAGE_URL}${orgQuery}${sinceQuery}`, bearer); + const summary = asRecord(summaryBody?.data) ?? summaryBody; + const used = toFiniteNumber(summary?.totalCost) ?? toFiniteNumber(summary?.totalMonthlyCredits); + if (used === undefined || used < 0) return undefined; + const pools = [credits.monthlyCredits, credits.purchasedCredits, credits.freeCredits] + .map(value => toFiniteNumber(value)) + .filter((value): value is number => value !== undefined); + // Field presence is what separates a real balance from absent data: an exhausted + // all-zero account still reports remaining=0, while no remaining-credit field at + // all means there is nothing to meter. + if (pools.length === 0) return undefined; + const remaining = pools.reduce((sum, value) => sum + Math.max(0, value ?? 0), 0); + const limit = used + remaining; + const percent = normalizePercent(limit > 0 ? (used / limit) * 100 : 0); + // Purchased credits roll over past the subscription period end, so an expiry is + // only truthful when the aggregate contains no non-expiring purchased pool. + const purchased = toFiniteNumber(credits.purchasedCredits) ?? 0; + return percent === undefined + ? undefined + : { + used, + limit, + remaining, + percent, + ...(expiresAt !== undefined && purchased <= 0 ? { expiresAt } : {}), + }; +} + +/** OAuth access token or ACTIVE Provider-API key for the Command Code quota probe. */ +async function resolveCommandCodeQuotaBearer(config: OcxProviderConfig, accountId?: string): Promise { + if (config.authMode === "oauth") { + try { + return accountId ? await getTokenForAccountQuotaProbe("command-code", accountId) : null; + } catch { + return null; + } + } + // ACTIVE key only: a quota bar for a different account than the one routing + // requests is a wrong meter, not a helpful one. + return resolveProviderApiKey(config.apiKey)?.trim() || null; +} + +/** + * Command Code `GET /alpha/billing/credits` — the same Bearer surface the CLI's + * usage view uses (windowLimits.fiveHour / windowLimits.weekly), plus soft + * whoami (team orgId scoping) and subscription-scoped spend for creditsUsd. + */ +export async function fetchCommandCodeQuota(provider: string, config: OcxProviderConfig, bearer: string): Promise { + // Never release credentials to a user-edited or lookalike provider host. + if (!isCanonicalCommandCodeBaseUrl(config.baseUrl)) return null; + if (!bearer) return null; + const whoamiBody = await fetchCommandCodeJson(COMMAND_CODE_WHOAMI_URL, bearer); + const whoami = asRecord(whoamiBody?.data) ?? whoamiBody; + const org = asRecord(whoami?.org); + const orgId = typeof org?.id === "string" && org.id.trim() ? org.id.trim() : null; + const orgQuery = orgId ? `?orgId=${encodeURIComponent(orgId)}` : ""; + const response = await fetch(`${COMMAND_CODE_CREDITS_URL}${orgQuery}`, { + headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) { + return response.status >= 400 && response.status < 500 && response.status !== 408 && response.status !== 429 + ? TERMINAL_QUOTA_FAILURE + : null; + } + const raw = asRecord(await readQuotaJson(response)); + const body = asRecord(raw?.data) ?? raw; + const credits = asRecord(body?.credits); + const limits = asRecord(body?.windowLimits); + if (!credits && !limits) return null; + const fiveHour = parseCommandCodeWindow(limits?.fiveHour); + const weekly = parseCommandCodeWindow(limits?.weekly); + const creditsUsd = await fetchCommandCodeSpend(bearer, credits, orgQuery); + const quota: ProviderQuota = { + ...(fiveHour ? { + fiveHourPercent: fiveHour.percent, + ...(fiveHour.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), + } : {}), + ...(weekly ? { + weeklyPercent: weekly.percent, + ...(weekly.resetAt !== undefined ? { weeklyResetAt: weekly.resetAt } : {}), + } : {}), + ...(creditsUsd ? { creditsUsd } : {}), + updatedAt: Date.now(), + }; + // Rolling windows and the credit balance both gate inference on this bearer. + return keyReport(provider, "command-code:credits", quota, config, bearer, quota); +} + + +type KeyQuotaReader = (name: string, provider: OcxProviderConfig) => Promise; + +/** Same selector drives cheap capabilities and uncached reads; never resolves credentials. */ +export function keyQuotaReaderForProvider(name: string, provider: OcxProviderConfig): KeyQuotaReader | null { + if (provider.disabled === true || (provider.authMode ?? "key") !== "key") return null; + if (isCanonicalKimiCodeBaseUrl(provider.baseUrl)) { + return async (id, config) => { + const bearer = await resolveKimiQuotaBearer(config); + return bearer ? fetchKimiQuota(id, config, bearer) : null; + }; + } + if (name === "commandcode" && isCanonicalCommandCodeBaseUrl(provider.baseUrl)) { + return async (id, config) => { + const bearer = await resolveCommandCodeQuotaBearer(config); + return bearer ? fetchCommandCodeQuota(id, config, bearer) : null; + }; + } + if (registryEntryForProviderDestination(provider)?.id === "opencode-go") return fetchOpenCodeGoQuota; + if (isCanonicalA6apiBaseUrl(provider.baseUrl)) return fetchA6apiQuota; + if (name === "openrouter" && isCanonicalOpenRouterBaseUrl(provider.baseUrl)) return fetchOpenRouterQuota; + if (name === "deepseek" && isCanonicalDeepSeekBaseUrl(provider.baseUrl)) return fetchDeepSeekQuota; + if (name === "cline-pass" && isCanonicalClineBaseUrl(provider.baseUrl)) return fetchClineQuota; + if (isCanonicalOllamaCloudBaseUrl(provider.baseUrl ?? getProviderRegistryEntry(name)?.baseUrl)) return fetchOllamaCloudQuota; + // #4201: the Responses preset is the same domestic GLM Coding Plan subscription on the OpenAI + // Responses wire, so it reads the same monitor endpoint. Eligibility stays a name list AND the + // canonical-URL guard: the guard is what keeps BigModel's bare-key Authorization from reaching a + // lookalike host, so a same-named custom destination still dispatches nothing. + if (["zai", "glm", "glm-cn", "zhipu-bigmodel-coding", "zhipu-bigmodel-responses"].includes(name) && isCanonicalZaiBaseUrl(provider.baseUrl)) return fetchZaiQuota; + if (["minimax", "minimax-cn"].includes(name) && isCanonicalMinimaxBaseUrl(provider.baseUrl)) return fetchMinimaxQuota; + if (name === "moonshot" && isCanonicalMoonshotBaseUrl(provider.baseUrl)) return fetchMoonshotQuota; + if (name === "venice" && isCanonicalVeniceBaseUrl(provider.baseUrl)) return fetchVeniceQuota; + if (name === "synthetic" && isCanonicalSyntheticBaseUrl(provider.baseUrl)) return fetchSyntheticQuota; + if (name === "deepinfra" && isCanonicalDeepInfraBaseUrl(provider.baseUrl)) return fetchDeepInfraQuota; + if (name === "neuralwatt" && isCanonicalNeuralwattBaseUrl(provider.baseUrl)) return fetchNeuralwattQuota; + return null; +} + +export function providerApiKeyQuotaMode(name: string, provider: OcxProviderConfig): AccountQuotaMode { + return keyQuotaReaderForProvider(name, provider) ? "probe" : "unsupported"; +} diff --git a/src/providers/quota/vendor-probes-oauth.ts b/src/providers/quota/vendor-probes-oauth.ts new file mode 100644 index 0000000000..7b9c9e7df5 --- /dev/null +++ b/src/providers/quota/vendor-probes-oauth.ts @@ -0,0 +1,590 @@ +import { effectiveCodexAuthAccountId, fetchMainAccountInfoSnapshot, listCodexAuthAccountsSnapshot } from "../../codex/auth-api"; +import { MAIN_CODEX_ACCOUNT_ID } from "../../codex/main-account"; +import { getValidAccessToken } from "../../oauth"; +import { getAccountCredential, getAccountSet } from "../../oauth/store"; +import { fetchMuseKeyQuotaSnapshot } from "../muse-key-quota"; +import { XAI_GROK_CLIENT_VERSION, XAI_GROK_COMPATIBILITY } from "../xai-transport"; +import { + commitKiroAccountUsageState, + fetchKiroUsageSnapshot, + type KiroUsageSnapshot, + kiroUsageContextForAccount, +} from "../kiro-usage"; +import { captureConfigGeneration } from "../../lib/state-store-sweeper"; +import { aggregateCodexPoolCapacity, CODEX_CAPACITY_MAX_QUOTA_AGE_MS, type CodexCapacityQuota } from "../codex-capacity"; +import { asRecord, normalizePercent, normalizeResetAt, readQuotaJson, REQUEST_TIMEOUT_MS, toFiniteNumber } from "../quota-wire"; +import { providerCodexAccountMode } from "../registry"; +import { + hasQuotaRows, + providerLabel, + providerQuotaFromCodexQuota, + publicCapacityAggregation, + report, + tagNativeMainReport, + type CodexAuthAccountsSnapshotPromise, + type ProviderQuotaReport, +} from "./report-cache"; +import { + accountCacheKey, + accountQuotaCache, + hydrateAccountQuotaCache, + mayCommitAccountQuotaKey, + persistAccountQuotaCache, +} from "./account-cache"; +import type { OcxConfig, OcxProviderConfig } from "../../types"; +import type { ProviderQuota, ProviderQuotaWindow } from "../quota-types"; + +const XAI_BILLING_URL = "https://cli-chat-proxy.grok.com/v1/billing"; +const XAI_CREDITS_URL = `${XAI_BILLING_URL}?format=credits`; + +export async function fetchChatGptForwardQuota( + config: OcxConfig, + provider: string, + providerConfig: OcxProviderConfig, + forceRefresh: boolean, + prefetchedSnapshot?: CodexAuthAccountsSnapshotPromise, +): Promise { + if (providerCodexAccountMode(provider, providerConfig) === "direct") { + const snapshot = await fetchMainAccountInfoSnapshot(forceRefresh); + const quota = providerQuotaFromCodexQuota(snapshot.info.quota); + if (quota) quota.updatedAt = Date.now(); + return quota + ? tagNativeMainReport(report(provider, "chatgpt:wham", quota), snapshot.mainIdentityGeneration) + : null; + } + const snapshot = await (prefetchedSnapshot ?? listCodexAuthAccountsSnapshot(config, forceRefresh)); + const accounts = snapshot.accounts; + const activeId = effectiveCodexAuthAccountId(config); + const capacityAccounts = accounts.map(account => ({ + ...account, + active: account.id === activeId, + quota: providerQuotaFromCodexQuota(account.quota), + })); + const active = capacityAccounts.find(account => account.active) + ?? capacityAccounts.find(account => account.id === MAIN_CODEX_ACCOUNT_ID) + ?? capacityAccounts[0]; + const now = Date.now(); + const capacity = aggregateCodexPoolCapacity(capacityAccounts, now); + if (capacity.aggregation && capacity.quota) { + return tagNativeMainReport( + report( + provider, + "chatgpt:wham", + capacity.quota as ProviderQuota, + publicCapacityAggregation(capacity.aggregation, "aggregate"), + ), + snapshot.mainIdentityGeneration, + ); + } + const activeUsable = !!active && !active.paused && active.needsReauth !== true; + const quota = activeUsable && active?.quota + ? { ...active.quota, updatedAt: active.quota.updatedAt ?? Date.now() } as CodexCapacityQuota + : null; + const quotaFresh = !!quota + && Number.isFinite(quota.updatedAt) + && now - quota.updatedAt < CODEX_CAPACITY_MAX_QUOTA_AGE_MS; + if (quota && quotaFresh) { + const fallback = report( + provider, + "chatgpt:wham", + quota as ProviderQuota, + capacity.aggregation + ? publicCapacityAggregation(capacity.aggregation, "effective-account-fallback") + : undefined, + ); + return tagNativeMainReport(fallback, snapshot.mainIdentityGeneration); + } + if (capacity.aggregation) { + const updatedAt = Date.now(); + return tagNativeMainReport( + { + provider, + label: providerLabel(provider), + source: "chatgpt:wham", + quota: { updatedAt }, + updatedAt, + aggregation: publicCapacityAggregation(capacity.aggregation, "coverage-only"), + }, + snapshot.mainIdentityGeneration, + ); + } + return null; +} + +function centsValue(value: unknown): number | undefined { + const rec = asRecord(value); + return rec ? toFiniteNumber(rec.val) : undefined; +} + +/** Decode JWT payload `sub` for xAI weekly credits when the stored credential lacks accountId. */ +function xaiUserIdFromAccessToken(accessToken: string): string | undefined { + const parts = accessToken.split("."); + if (parts.length < 2 || !parts[1]) return undefined; + try { + const payload = JSON.parse(Buffer.from(parts[1], "base64url").toString("utf8")) as { sub?: unknown }; + return typeof payload.sub === "string" && payload.sub.trim() ? payload.sub.trim() : undefined; + } catch { + return undefined; + } +} + +/** + * Grok Build weekly credits envelope: + * `{ config: { creditUsagePercent?, currentPeriod: { type: "USAGE_PERIOD_TYPE_WEEKLY", end } } }`. + * Omitted percent is treated as 0 (proto3 default). + */ +export function parseXaiCreditsResponse(value: unknown): { percent: number; resetAt?: number } | null { + const body = asRecord(value); + const config = asRecord(body?.config); + if (!config) return null; + const period = asRecord(config.currentPeriod); + if (!period || period.type !== "USAGE_PERIOD_TYPE_WEEKLY") return null; + let percent = 0; + if (config.creditUsagePercent !== undefined) { + const normalized = normalizePercent(config.creditUsagePercent); + if (normalized === undefined) return null; + percent = normalized; + } + const resetAt = normalizeResetAt(period.end); + return { + percent, + ...(resetAt !== undefined ? { resetAt } : {}), + }; +} + +async function fetchXaiWeeklyCredits(accessToken: string, userId: string): Promise { + try { + const response = await fetch(XAI_CREDITS_URL, { + redirect: "error", + headers: { + Accept: "application/json", + Authorization: `Bearer ${accessToken}`, + [XAI_GROK_COMPATIBILITY.headers.tokenAuth]: "xai-grok-cli", + [XAI_GROK_COMPATIBILITY.headers.authenticateResponse]: "authenticate-response", + "x-userid": userId, + [XAI_GROK_COMPATIBILITY.headers.clientVersion]: XAI_GROK_CLIENT_VERSION, + }, + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + const parsed = parseXaiCreditsResponse(await readQuotaJson(response)); + if (!parsed) return null; + return { + weeklyPercent: parsed.percent, + ...(parsed.resetAt !== undefined ? { weeklyResetAt: parsed.resetAt } : {}), + updatedAt: Date.now(), + }; + } catch { + return null; + } +} + +export async function fetchXaiQuota(provider: string, context: { accessToken: string; upstreamAccountId?: string }): Promise { + const { accessToken } = context; + + // Prefer the SuperGrok weekly credits window that actually gates prompting (#1283). + const userId = context.upstreamAccountId?.trim() || xaiUserIdFromAccessToken(accessToken); + if (userId) { + const weekly = await fetchXaiWeeklyCredits(accessToken, userId); + if (weekly) return report(provider, "xai:grok-billing-credits", weekly); + } + + // Legacy monthly dollar pool — retained when weekly is unavailable. + try { + const response = await fetch(XAI_BILLING_URL, { + redirect: "error", + headers: { Accept: "application/json", Authorization: `Bearer ${accessToken}` }, + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + const body = asRecord(await readQuotaJson(response)); + const config = asRecord(body?.config); + if (!config) return null; + const limitCents = centsValue(config.monthlyLimit); + const usedCents = centsValue(config.used); + if (limitCents === undefined || usedCents === undefined || limitCents <= 0) return null; + const percent = normalizePercent((usedCents / limitCents) * 100); + if (percent === undefined) return null; + return report(provider, "xai:grok-billing", { + monthlyPercent: percent, + monthlyResetAt: normalizeResetAt(config.billingPeriodEnd), + updatedAt: Date.now(), + }); + } catch { + return null; + } +} + +function parseClaudeBucket(value: unknown): { percent?: number; resetAt?: number } | null { + const rec = asRecord(value); + if (!rec) return null; + const percent = normalizePercent(rec.utilization); + const resetAt = normalizeResetAt(rec.resets_at); + if (percent === undefined && resetAt === undefined) return null; + return { percent, resetAt }; +} + +function parseClaudeLimit(value: unknown): { label: string; percent: number; resetAt?: number } | null { + const rec = asRecord(value); + if (!rec) return null; + const percent = normalizePercent(rec.percent); + if (percent === undefined) return null; + const scope = asRecord(rec.scope); + const model = asRecord(scope?.model); + const rawLabel = String(model?.display_name ?? "").trim(); + if (!rawLabel) return null; + const lowerLabel = rawLabel.toLowerCase(); + const label = lowerLabel.includes("fable") ? "Fable" + : lowerLabel.includes("opus") ? "Opus" + : lowerLabel.includes("sonnet") ? "Sonnet" + : rawLabel; + const resetAt = normalizeResetAt(rec.resets_at); + return { label, percent, ...(resetAt !== undefined ? { resetAt } : {}) }; +} + +/** Claude's OAuth usage endpoint, probed with ONE account's own bearer token. */ +const anthropicUsageInflight = new Map>(); + +/** + * Anthropic per-credential usage. + * + * This endpoint reports quota only. Its body carries `five_hour`, `seven_day`, the + * model-scoped weekly buckets (`seven_day_fable`/`_opus`/`_sonnet`) and a `limits` array, + * and **no subscription or tier field** — nor does the OAuth token response, which yields only + * `account.uuid` and `account.email_address` (`src/oauth/anthropic.ts`). That is why + * `OAuthAccountSummary.plan` is `null` for Anthropic rather than populated here (#3777); it is + * a missing upstream field, not an unfinished mapping. + * + * A tier must not be inferred from what is here. Percentages are normalized per account, so a + * Max x5 seat at 50% is byte-identical to a Max x20 seat at 50%, and the presence of a + * model-scoped window tracks entitlement rather than seat size. Populate `plan` only when + * upstream returns the tier itself. + */ +export async function fetchAnthropicUsageQuota(accessToken: string): Promise { + const joinable = anthropicUsageInflight.get(accessToken); + if (joinable) return joinable; + + const probe = (async (): Promise => { + const response = await fetch("https://api.anthropic.com/api/oauth/usage", { + headers: { + Accept: "application/json, text/plain, */*", + "Content-Type": "application/json", + "User-Agent": "claude-cli/2.1.63 (external, cli)", + "anthropic-beta": "claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14,context-management-2025-06-27,prompt-caching-scope-2026-01-05", + Authorization: `Bearer ${accessToken}`, + }, + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + const body = asRecord(await readQuotaJson(response)); + if (!body) return null; + const fiveHour = parseClaudeBucket(body.five_hour); + const sevenDay = parseClaudeBucket(body.seven_day); + const fable = parseClaudeBucket(body.seven_day_fable); + const opus = parseClaudeBucket(body.seven_day_opus); + const sonnet = parseClaudeBucket(body.seven_day_sonnet); + const customWindows: ProviderQuotaWindow[] = []; + if (fable?.percent !== undefined) customWindows.push({ label: "Fable", percent: fable.percent, ...(fable.resetAt !== undefined ? { resetAt: fable.resetAt } : {}) }); + if (opus?.percent !== undefined) customWindows.push({ label: "Opus", percent: opus.percent, ...(opus.resetAt !== undefined ? { resetAt: opus.resetAt } : {}) }); + if (sonnet?.percent !== undefined) customWindows.push({ label: "Sonnet", percent: sonnet.percent, ...(sonnet.resetAt !== undefined ? { resetAt: sonnet.resetAt } : {}) }); + const knownLabels = new Set(customWindows.map(window => window.label.toLowerCase())); + const limits = Array.isArray(body.limits) ? body.limits : []; + for (const rawLimit of limits) { + const limitRecord = asRecord(rawLimit); + // `session` and `weekly_all` mirror the canonical five-hour and weekly + // buckets above; only model-scoped weekly limits add a third window. + if (String(limitRecord?.kind ?? "").trim().toLowerCase() !== "weekly_scoped") continue; + const limit = parseClaudeLimit(rawLimit); + if (!limit || knownLabels.has(limit.label.toLowerCase())) continue; + knownLabels.add(limit.label.toLowerCase()); + customWindows.push(limit); + } + const quota: ProviderQuota = { + // Claude's 5-hour window is a first-class rate limit, same as the Codex login 5h/weekly + // rows: report it in the canonical fields so the dashboard renders it with the standard + // "5-hour limit" label and ordering instead of as a generic extra window. + ...(fiveHour?.percent !== undefined ? { fiveHourPercent: fiveHour.percent } : {}), + ...(fiveHour?.resetAt !== undefined ? { fiveHourResetAt: fiveHour.resetAt } : {}), + ...(sevenDay?.percent !== undefined ? { weeklyPercent: sevenDay.percent } : {}), + ...(sevenDay?.resetAt !== undefined ? { weeklyResetAt: sevenDay.resetAt } : {}), + ...(customWindows.length > 0 ? { customWindows } : {}), + updatedAt: Date.now(), + }; + // Empty / schema-changed payloads must not cache as "success with no bars". + return hasQuotaRows(quota) ? quota : null; + })().finally(() => { + if (anthropicUsageInflight.get(accessToken) === probe) anthropicUsageInflight.delete(accessToken); + }); + anthropicUsageInflight.set(accessToken, probe); + return probe; +} + +export async function fetchAnthropicQuota(provider: string): Promise { + // Capture the account we intend to probe before awaiting — a mid-flight active + // switch must not seed the wrong account's cache with this response. + const probedAccountId = getAccountSet("anthropic")?.activeAccountId; + const probedAccountKey = probedAccountId ? accountCacheKey("anthropic", probedAccountId) : null; + const writerGeneration = captureConfigGeneration(); + let accessToken: string; + try { + accessToken = await getValidAccessToken("anthropic"); + } catch { + return null; + } + const quota = await fetchAnthropicUsageQuota(accessToken); + if (!quota) return null; + // Share the active-account probe with the per-account cache so Providers-page + // loads do not double-hit Anthropic's rate-limited usage endpoint. + if (probedAccountId && probedAccountKey) { + const stillOwnsToken = getAccountCredential("anthropic", probedAccountId)?.access === accessToken; + if (stillOwnsToken && mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { + accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota }); + } + } + return report(provider, "anthropic:oauth-usage", quota); +} + +/** + * Provider-level Kiro row: the active account's usage, shown on the Providers page. + * + * The per-account cache is seeded from the same probe so opening that page does not read + * the active account twice, and the account id is captured before the await so a + * concurrent account switch cannot file this answer under the wrong account. + */ +export async function fetchKiroQuota(provider: string): Promise { + const probedAccountId = getAccountSet("kiro")?.activeAccountId; + if (!probedAccountId) return null; + const probedAccountKey = accountCacheKey("kiro", probedAccountId); + const writerGeneration = captureConfigGeneration(); + let snapshot: KiroUsageSnapshot | null; + try { + snapshot = await fetchKiroUsageSnapshot(await kiroUsageContextForAccount(probedAccountId)); + } catch { + return null; + } + if (!snapshot) return null; + if (mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { + accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota: snapshot.quota }); + commitKiroAccountUsageState(probedAccountKey, snapshot); + } + return report(provider, "kiro:usage-limits", snapshot.quota); +} + +/** + * Provider-level row probed from the key endpoint, for an account that CAN be probed. + * + * Written through the same account cache the passive path reads, so the measurement + * survives a restart and the per-account rows at oauth-account-routes.ts:313 pick it up + * with no mode change. Deliberately does not flip providerOAuthAccountQuotaMode: that + * mode selects readPassiveProviderAccountQuotas, and the probed per-account path it would + * switch to is gated on supportsPerAccountQuota, which has no meta-muse reader, so the + * GUI account list would go from showing observations to showing nothing. + */ +export async function fetchMuseKeyQuota(provider: string): Promise { + const probedAccountId = getAccountSet(provider)?.activeAccountId; + if (!probedAccountId) return null; + const oauthAccessToken = getAccountCredential(provider, probedAccountId)?.muse?.oauthAccessToken; + // An imported or pasted credential has no account token and never will: it is + // capability, not provider id, that decides whether a probe is possible. + if (!oauthAccessToken) return null; + const probedAccountKey = accountCacheKey(provider, probedAccountId); + const writerGeneration = captureConfigGeneration(); + const quota = await fetchMuseKeyQuotaSnapshot(probedAccountId, oauthAccessToken); + if (!quota) return null; + if (mayCommitAccountQuotaKey(probedAccountKey, writerGeneration)) { + // Hydrate before writing, for the same reason recordPassiveAccountQuota does: + // persistAccountQuotaCache serializes the whole in-memory map. + hydrateAccountQuotaCache(); + accountQuotaCache.set(probedAccountKey, { ts: Date.now(), quota }); + persistAccountQuotaCache(); + } + return report(provider, `${provider}:key-endpoint`, quota); +} +/** + * Provider-level row for a passive provider: the ACTIVE account's last observed + * subscription windows, the same shape `fetchAnthropicQuota` and `fetchKiroQuota` + * return. + * + * Cache-only. A dashboard load or `ocx account refresh` must never spend an inference + * turn, so `forceRefresh` does not exist on this path — there is nothing to refresh. + * `report.updatedAt` is the observation time, which is what both GUI surfaces render + * as the relative age of the row. + */ +export async function fetchPassiveProviderQuota(provider: string): Promise { + const activeId = getAccountSet(provider)?.activeAccountId; + if (!activeId) return null; + // Idempotent; without it a proxy restart shows nothing until the next streaming turn + // even though the last observation is on disk. + hydrateAccountQuotaCache(); + const entry = accountQuotaCache.get(accountCacheKey(provider, activeId)); + if (!entry?.quota) return null; + const built = report(provider, `${provider}:subscription-observation`, entry.quota); + // Tagged here rather than inside report(), which every probed path shares. + return built ? { ...built, observed: true } : null; +} + +// --------------------------------------------------------------------------- +// Per-account quota (multiauth) +// --------------------------------------------------------------------------- + + +/** Cursor included usage via api2.cursor.sh (Bearer from OAuth) — unofficial, may change. */ +export async function fetchCursorQuota(provider: string, accessToken: string): Promise { + + const authHeaders = { + Accept: "application/json", + Authorization: `Bearer ${accessToken}`, + "User-Agent": "opencodex-quota", + } as const; + + // Prefer dashboard period usage (Pro/Team/Ultra spend allowance in USD cents). + // Field names follow Cursor's Connect RPC shape (limit/remaining/includedSpend), not usedCents. + try { + const periodRes = await fetch("https://api2.cursor.sh/aiserver.v1.DashboardService/GetCurrentPeriodUsage", { + method: "POST", + redirect: "error", + headers: { + ...authHeaders, + "Content-Type": "application/json", + "Connect-Protocol-Version": "1", + }, + body: "{}", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (periodRes.ok) { + const body = asRecord(await readQuotaJson(periodRes)); + const planUsage = asRecord(body?.planUsage); + if (planUsage) { + const resetAt = normalizeResetAt(body?.billingCycleEnd ?? planUsage.billingCycleEnd ?? body?.periodEnd); + + // Primary meter: overall included allowance (Cursor Settings → Usage total %). + // autoPercentUsed / apiPercentUsed are secondary pools and must not replace the total. + const limit = toFiniteNumber(planUsage.limit ?? planUsage.limitCents ?? planUsage.totalLimitCents); + const remaining = toFiniteNumber(planUsage.remaining ?? planUsage.remainingCents); + const includedSpend = toFiniteNumber(planUsage.includedSpend ?? planUsage.usedCents ?? planUsage.used); + const totalSpend = toFiniteNumber(planUsage.totalSpend); + let used: number | undefined; + if (includedSpend !== undefined) used = includedSpend; + else if (limit !== undefined && remaining !== undefined) used = Math.max(0, limit - remaining); + else if (totalSpend !== undefined) used = totalSpend; + const totalPercent = normalizePercent(planUsage.totalPercentUsed ?? planUsage.percentUsed) + ?? (limit !== undefined && limit > 0 && used !== undefined + ? normalizePercent((used / limit) * 100) + : undefined); + + const autoPercent = normalizePercent(planUsage.autoPercentUsed); + const apiPercent = normalizePercent(planUsage.apiPercentUsed); + const customWindows: ProviderQuotaWindow[] = []; + if (autoPercent !== undefined) { + customWindows.push({ + label: "First-party models", + percent: autoPercent, + ...(resetAt !== undefined ? { resetAt } : {}), + }); + } + if (apiPercent !== undefined) { + customWindows.push({ + label: "API usage", + percent: apiPercent, + ...(resetAt !== undefined ? { resetAt } : {}), + }); + } + + if (totalPercent !== undefined || customWindows.length > 0) { + const built = report(provider, "cursor:period-usage", { + ...(totalPercent !== undefined ? { + monthlyPercent: totalPercent, + ...(resetAt !== undefined ? { monthlyResetAt: resetAt } : {}), + } : {}), + ...(customWindows.length > 0 ? { customWindows } : {}), + updatedAt: Date.now(), + }); + if (built) return { ...built, reverseEngineered: true }; + } + } + } + } catch { + /* fall through */ + } + + // /api/usage/summary — same host, sometimes richer than /auth/usage for Team plans. + try { + const summaryRes = await fetch("https://api2.cursor.sh/api/usage/summary", { + headers: authHeaders, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (summaryRes.ok) { + const body = asRecord(await readQuotaJson(summaryRes)); + const individual = asRecord(body?.individualUsage); + const plan = asRecord(individual?.plan); + if (plan) { + const used = toFiniteNumber(plan.used); + const limit = toFiniteNumber(plan.limit); + const percent = normalizePercent(plan.totalPercentUsed) + ?? (used !== undefined && limit !== undefined && limit > 0 + ? normalizePercent((used / limit) * 100) + : undefined); + if (percent !== undefined) { + const built = report(provider, "cursor:usage-summary", { + monthlyPercent: percent, + monthlyResetAt: normalizeResetAt(body?.billingCycleEnd), + updatedAt: Date.now(), + }); + if (built) return { ...built, reverseEngineered: true }; + } + } + } + } catch { + /* fall through to /auth/usage */ + } + + const response = await fetch("https://api2.cursor.sh/auth/usage", { + headers: authHeaders, + redirect: "error", + signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), + }); + if (!response.ok) return null; + const body = asRecord(await readQuotaJson(response)); + if (!body) return null; + + // Prefer the gpt-4 bucket (historical "fast requests"); else first model with used+limit. + let used: number | undefined; + let limit: number | undefined; + const gpt4 = asRecord(body["gpt-4"]); + if (gpt4) { + used = toFiniteNumber(gpt4.numRequests ?? gpt4.used); + limit = toFiniteNumber(gpt4.maxRequestUsage ?? gpt4.limit ?? gpt4.maxRequests); + } + if (used === undefined || limit === undefined || limit <= 0) { + for (const [key, value] of Object.entries(body)) { + if (key === "startOfMonth" || key === "billingCycleStart") continue; + const bucket = asRecord(value); + if (!bucket) continue; + const bucketUsed = toFiniteNumber(bucket.numRequests ?? bucket.used); + const bucketLimit = toFiniteNumber(bucket.maxRequestUsage ?? bucket.limit ?? bucket.maxRequests); + if (bucketUsed !== undefined && bucketLimit !== undefined && bucketLimit > 0) { + used = bucketUsed; + limit = bucketLimit; + break; + } + } + } + if (used === undefined || limit === undefined || limit <= 0) return null; + const percent = normalizePercent((used / limit) * 100); + if (percent === undefined) return null; + const startOfMonth = normalizeResetAt(body.startOfMonth ?? body.billingCycleStart); + // Next reset = same day next month, computed in UTC to avoid timezone-shifted rollover. + const monthlyResetAt = startOfMonth !== undefined + ? (() => { + const start = new Date(startOfMonth); + return Date.UTC(start.getUTCFullYear(), start.getUTCMonth() + 1, start.getUTCDate()); + })() + : undefined; + const built = report(provider, "cursor:auth-usage", { + monthlyPercent: percent, + ...(monthlyResetAt !== undefined ? { monthlyResetAt } : {}), + updatedAt: Date.now(), + }); + return built ? { ...built, reverseEngineered: true } : null; +} diff --git a/src/responses/state.ts b/src/responses/state.ts index f9195196a2..a36435aa0b 100644 --- a/src/responses/state.ts +++ b/src/responses/state.ts @@ -1,27 +1,42 @@ -import { chmodSync, existsSync, lstatSync, mkdirSync, opendirSync, readFileSync, rmSync, statSync, unlinkSync } from "node:fs"; -import { uptime } from "node:os"; +import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, statSync, unlinkSync } from "node:fs"; import { dirname, join } from "node:path"; import { atomicWriteFileAsync, getConfigDir, resolveWriteTarget } from "../config"; import { enforceAppOwnedMemoryBudget, type RetainedStoreSnapshot } from "../lib/app-owned-memory"; import { windowsSecretAclApplies } from "../lib/windows-secret-acl"; import type { OcxProviderContinuationState } from "../types"; import { - cleanupSupersededResponseSpillPublication, - createResponseSpillPublicationControl, deleteResponseSpill, - MAX_RESPONSE_SPILL_PAYLOAD_BYTES, noteStubSwapForTest, readResponseSpill, recoverOrphanedResponseSpills, responseSpillDirectory, responseSpillPayloadCap, - markResponseSpillPublicationSuperseded, - prospectiveResponseSpillBytes, - type ResponseSpillPublicationControl, type ResponseSpillRef, writeResponseSpillDurably, - writeResponseSpillDurablyAsync, } from "./spill-store"; +import { clientCarriedPrefixLength, providerIssuedIdentity } from "./state/replay-fingerprint"; +export type { ResponseStateTempRecoveryResult, ResponseStateTempRecoveryOptions } from "./state/temp-recovery"; +export { recoverStaleResponseStateTemps, reclaimAbandonedResponseStateTemps, inspectAbandonedResponseStateTemps, sweepAbandonedResponseStateTemps } from "./state/temp-recovery"; +import { recoverStaleResponseStateTemps } from "./state/temp-recovery"; +export type { ResponseSpillWriteFailureCode, ResponseSpillWriteStatus, ResponseSpillWriteFailureOrigin } from "./state/spill-failure"; +import type { ResponseSpillWriteFailureCode, ResponseSpillWriteStatus, ResponseSpillWriteFailureOrigin } from "./state/spill-failure"; +export { responseAdmissionCountersForTests } from "./state/spill-failure"; +import { admissionCounters, noteSpillWriteFailure, noteSpillWriteSuccess, spillCounters, spillWriteHealth } from "./state/spill-failure"; +import { loadSnapshotEntry } from "./state/snapshot-codec"; +export { flushPendingResponseSpillsForTests, awaitResponseSpillPublicationTailForTests, pendingResponseSpillMetricsForTests, setResponseSpillShutdownBudgetForTests, setResponseSpillAsyncAclAttemptBudgetForTests, setResponseSpillShutdownTerminalizationPassLimitForTests } from "./state/spill-queue"; +import { + bindSpillQueueStore, + cancelPendingResponseSpill, + drainResponseSpillPublications, + queuePendingResponseSpill, + replaceWithPendingResponseSpill, + resetSpillQueueForTests, + spillQueueAccounting, + spillQueueHoldsResidentCandidate, + spillQueuePendingBytes, + spillQueueResidentCandidates, + spillQueueSupersededSpillFor, +} from "./state/spill-queue"; const MAX_STORED_RESPONSES = 1_000; const RESPONSE_TTL_MS = 60 * 60 * 1_000; @@ -67,28 +82,10 @@ const SNAPSHOT_TOTAL_MAX_BYTES = 24 * 1024 * 1024; * bound, so anything we wrote ourselves always loads; guards against externally * planted or pre-cap unbounded files being parsed whole). */ const SNAPSHOT_FILE_MAX_BYTES = 32 * 1024 * 1024; -const STALE_TEMP_GRACE_MS = 15 * 60 * 1_000; -const STALE_TEMP_MAX_ENTRIES = 4_096; -const STALE_TEMP_MAX_CLEANUPS = 512; -/** Absorbs `os.uptime()` granularity only. It is deliberately NOT the safety margin: - * the unconditional 15-minute grace above is (see the boot floor in the scan loop). */ -const BOOT_FLOOR_SKEW_MS = 60 * 1_000; -/** Per-tick budget for the periodic reclaim. Smaller than the startup budget because the - * periodic pass runs synchronously on the serving process's event loop every 60 s. */ -const PERIODIC_TEMP_MAX_ENTRIES = 512; -const PERIODIC_TEMP_MAX_CLEANUPS = 64; -/** Wall-clock ceiling for one periodic scan. An entry cap bounds syscalls, not time: on a - * network-mounted config dir each `lstat` can cost 10-20 ms, which would stall in-flight - * streams. Reclaim is idempotent, so a truncated tick simply resumes on the next one. */ -const PERIODIC_TEMP_SCAN_DEADLINE_MS = 25; -const RESPONSE_STATE_TEMP_NAME = /^responses-state\.json\.ocx\.(\d+)\.(\d+)\.tmp$/; const MAX_SNAPSHOT_REWRITE_ATTEMPTS = 4; -const RESPONSE_SPILL_SHUTDOWN_BUDGET_MS = 5_000; -const RESPONSE_SPILL_SHUTDOWN_FALLBACK_RESERVE_MS = 4_000; -const RESPONSE_SPILL_ASYNC_ACL_ATTEMPT_BUDGET_MS = 30_000; const RESPONSE_SPILL_SHUTDOWN_TERMINALIZATION_MAX_PASSES = MAX_STORED_RESPONSES + 1; -interface ResidentResponseState { +export interface ResidentResponseState { kind: "resident"; createdAt: number; clientThreadId?: string; @@ -99,7 +96,7 @@ interface ResidentResponseState { sizeBytes: number; } -interface SpilledResponseState { +export interface SpilledResponseState { kind: "spill"; createdAt: number; clientThreadId?: string; @@ -110,14 +107,14 @@ interface SpilledResponseState { sizeBytes: number; } -interface SpillFailedResponseState { +export interface SpillFailedResponseState { kind: "spill-failed"; createdAt: number; sizeBytes: number; } -type StoredResponseState = ResidentResponseState | SpilledResponseState | SpillFailedResponseState; -type ResidentInput = Omit; +export type StoredResponseState = ResidentResponseState | SpilledResponseState | SpillFailedResponseState; +export type ResidentInput = Omit; export type PreviousResponseReplayFailure = { code: "previous_response_not_found"; @@ -169,124 +166,8 @@ async function snapshotOnDiskMatches(path: string, payload: string, payloadBytes return false; } } -const spillCounters = { - writes: 0, writeFailures: 0, readFailures: 0, - aclRetryReturnedTimeouts: 0, aclTimeoutMemoRefusals: 0, -}; - -export type ResponseSpillWriteFailureCode = - | "EACLRETRYEXHAUSTED" - | "ETIMEDOUT" - | "EACCES" - | "ENOSPC" - | "EFBIG" - | "EIO" - | "ECAPACITY" - | "ELOOP" - | "EUNKNOWN"; - -export type ResponseSpillWriteStatus = "initial" | "healthy" | "degraded"; - -export type ResponseSpillWriteFailureOrigin = - | "retry_returned_timeout" - | "timeout_memo_refusal"; - -interface ResponseSpillWriteHealth { - consecutiveFailures: number; - lastFailureCode: ResponseSpillWriteFailureCode | null; - lastFailureOrigin: ResponseSpillWriteFailureOrigin | null; - lastFailureAt: number | null; - lastSuccessAt: number | null; -} - -const spillWriteHealth: ResponseSpillWriteHealth = { - consecutiveFailures: 0, - lastFailureCode: null, - lastFailureOrigin: null, - lastFailureAt: null, - lastSuccessAt: null, -}; - -/** - * Collapse filesystem/runtime errors into a fixed privacy-safe diagnostic union. - * Messages and paths are deliberately ignored: this projection is returned by the - * authenticated memory endpoint, and a nested `cause` can contain a username or - * workspace path even when the public wrapper does not. - */ -function classifySpillWriteFailure(error: unknown): ResponseSpillWriteFailureCode { - let cursor = error; - for (let depth = 0; depth < 4 && cursor && typeof cursor === "object"; depth += 1) { - const record = cursor as { code?: unknown; cause?: unknown }; - const code = typeof record.code === "string" ? record.code.toUpperCase() : ""; - switch (code) { - case "EACLRETRYEXHAUSTED": return "EACLRETRYEXHAUSTED"; - case "ETIMEDOUT": return "ETIMEDOUT"; - case "EACCES": - case "EPERM": return "EACCES"; - case "ENOSPC": - case "EDQUOT": return "ENOSPC"; - case "EFBIG": return "EFBIG"; - case "EIO": return "EIO"; - case "ECAPACITY": return "ECAPACITY"; - case "ELOOP": return "ELOOP"; - } - cursor = record.cause; - } - return "EUNKNOWN"; -} - -/** The spill writer preserves ACL errors in cause; only a fixed memo marker is diagnostic. */ -function spillAclMemoRefusalOrigin(error: unknown): "timeout_memo_refusal" | null { - let cursor = error; - for (let depth = 0; depth < 4 && cursor && typeof cursor === "object"; depth += 1) { - const record = cursor as { code?: unknown; aclFailureOrigin?: unknown; cause?: unknown }; - if ((record.code === "ETIMEDOUT" || record.code === "EACLRETRYEXHAUSTED") - && record.aclFailureOrigin === "timeout_memo_refusal") { - return "timeout_memo_refusal"; - } - cursor = record.cause; - } - return null; -} - -function noteSpillWriteSuccess(): void { - spillCounters.writes += 1; - spillWriteHealth.consecutiveFailures = 0; - spillWriteHealth.lastSuccessAt = now(); -} - -function noteSpillWriteFailure( - error: unknown, - override?: ResponseSpillWriteFailureCode, - retryOrigin: ResponseSpillWriteFailureOrigin | null = null, -): void { - const code = override ?? classifySpillWriteFailure(error); - const origin = code === "ETIMEDOUT" || code === "EACLRETRYEXHAUSTED" - ? spillAclMemoRefusalOrigin(error) ?? retryOrigin - : null; - spillCounters.writeFailures += 1; - spillWriteHealth.consecutiveFailures += 1; - spillWriteHealth.lastFailureCode = code; - spillWriteHealth.lastFailureOrigin = origin; - spillWriteHealth.lastFailureAt = now(); - // Count terminal publications, not ACL calls or a transient first attempt. - if (origin === "retry_returned_timeout") spillCounters.aclRetryReturnedTimeouts += 1; - else if (origin === "timeout_memo_refusal") spillCounters.aclTimeoutMemoRefusals += 1; -} -/** - * Admission-boundary observability (test-visible). directSpills: oversized - * candidates routed straight to durable spill without a resident stay or - * unrelated demotion. oversizedDrops: candidates above the single-spill - * payload ceiling, tombstoned instead of retained. snapshotOversizedRefusals: - * snapshot files refused before parse. - */ -const admissionCounters = { directSpills: 0, oversizedDrops: 0, snapshotOversizedRefusals: 0 }; let replayScopeMismatchDrops = 0; -/** Test-only: admission-boundary counters (proves the new paths fire). */ -export function responseAdmissionCountersForTests(): Readonly { - return admissionCounters; -} // Superseded spill generations awaiting a durable snapshot before unlink // (review C1-1: unlinking at swap time races a crash against the debounced // snapshot — the reloaded OLD stub would point at a deleted file). @@ -299,99 +180,6 @@ const pendingSpillUnlinks: ResponseSpillRef[] = []; // structured 400 — bounded-loss, never silent corruption or unbounded disk. const PENDING_SPILL_UNLINKS_MAX = 128; -/** - * Windows keeps the candidate replayable while required ACL hardening runs off the event loop. - * Pending bytes are pinned, not evictable; cap them below the process-owned 512 MiB ceiling so an - * icacls outage cannot turn the serialized queue into an unbounded resident backlog. - */ -const MAX_PENDING_RESPONSE_SPILL_BYTES = MAX_RESPONSE_SPILL_PAYLOAD_BYTES; - -interface PendingResponseSpill { - id: string; - candidate: ResidentResponseState | null; - supersededSpill?: ResponseSpillRef; - directAdmission: boolean; - running: boolean; - cancelled: boolean; - released: boolean; - sizeBytes: number; - /** Peak on-disk bytes reserved for this publication; released exactly once on settle. */ - reservedBytes: number; - publicationControl: ResponseSpillPublicationControl; -} - -const pendingResponseSpills = new Set(); -const pendingResponseSpillById = new Map(); -let pendingResponseSpillBytes = 0; -/** - * On-disk bytes a queued publication is about to occupy but has not yet installed into - * `states`. - * - * `spilledResponseBytes()` walks installed spills and deferred unlinks — files that - * already exist. It cannot see one that `writeResponseSpillDurablyAsync` is in the - * middle of creating, and on Windows that middle can last as long as `icacls` takes. - * Without a reservation the cap holds only when writes are fast, which is not a cap. - * - * The reserved figure is the PEAK footprint, not the payload: publication can fall back - * from hard-linking to an exclusive copy, and during that fallback the destination copy - * and the temp file exist simultaneously. Reserving one envelope would leave the overshoot - * intact at half its magnitude. - * - * Ownership is single: a job holds its reservation from queue until - * `releasePendingResponseSpill`, which every exit from the publication path reaches - * through the `finally` in `runPendingResponseSpill` and through cancellation of a - * not-yet-running job. A leaked reservation is monotonic — it would ratchet the usable - * cap toward zero — so the release must stay on the settlement path rather than in a - * parallel bookkeeping pass. - */ -let reservedResponseSpillBytes = 0; -/** - * Paths a failed cleanup left on the volume, with the bytes each one occupies. - * - * A failed unlink leaves a real file behind, so the cap has to keep seeing it. But a - * never-decremented total would be phantom debt: a Windows lock that clears a moment - * later, or the async writer's own retry, can remove the file while the charge stays - * forever — and with 256 MiB payloads two conservative charges consume the whole default - * cap, after which nothing can spill for the life of the process. - * - * So the debt is per PATH, priced at what that path actually holds, and settled the - * moment the path is gone. `reconcileUnreclaimableSpillPaths` re-checks on every read of - * the accounted total, which is the same tick that would otherwise refuse an admission. - */ -const unreclaimableSpillPaths = new Map(); - -function chargeUnreclaimableSpillPath(path: string | null | undefined, bytes: number): void { - if (!path || bytes <= 0) return; - unreclaimableSpillPaths.set(path, bytes); -} - -/** Drop charges for paths that have since disappeared; returns the surviving total. */ -function reconcileUnreclaimableSpillPaths(): number { - let total = 0; - for (const [path, bytes] of [...unreclaimableSpillPaths]) { - if (existsSync(path)) total += bytes; - else unreclaimableSpillPaths.delete(path); - } - return total; -} - -/** - * Peak on-disk footprint of publishing this candidate: temp plus destination copy. - * - * Measured from the production serializer rather than from `candidate.sizeBytes`. The - * resident measurement omits the `version` field the published envelope carries, so - * pricing an admission by it undercounts and lets a request sitting exactly at the cap - * still exceed it. Falls back to the resident figure only when serialization fails, which - * is the same condition that will fail the publication itself. - */ -function publicationFootprintBytes(id: string, candidate: ResidentResponseState): number { - const exact = prospectiveResponseSpillBytes(id, spillPayloadForResident(candidate)); - return (exact ?? candidate.sizeBytes) * 2; -} -let responseSpillPublicationTail: Promise = Promise.resolve(); -let responseSpillShutdownBudgetOverride: { totalMs: number; fallbackReserveMs: number } | null = null; -let responseSpillShutdownTerminalizationPassLimitOverride: number | null = null; -let responseSpillAsyncAclAttemptBudgetOverride: number | null = null; function deferSupersededSpill(ref: ResponseSpillRef | undefined): void { if (!ref) return; @@ -401,470 +189,6 @@ function deferSupersededSpill(ref: ResponseSpillRef | undefined): void { } } -function releasePendingResponseSpill(job: PendingResponseSpill): void { - if (job.released) return; - job.released = true; - pendingResponseSpillBytes = Math.max(0, pendingResponseSpillBytes - job.sizeBytes); - reservedResponseSpillBytes = Math.max(0, reservedResponseSpillBytes - job.reservedBytes); - pendingResponseSpills.delete(job); - if (pendingResponseSpillById.get(job.id) === job) pendingResponseSpillById.delete(job.id); - job.candidate = null; -} - -function cancelPendingResponseSpill(id: string): ResponseSpillRef | undefined { - const job = pendingResponseSpillById.get(id); - if (!job) return undefined; - pendingResponseSpillById.delete(id); - job.cancelled = true; - markResponseSpillPublicationSuperseded(job.publicationControl); - const superseded = job.supersededSpill; - // Ownership TRANSFERS to the caller. Leaving the ref on the cancelled job would let the - // accounting walk count the same physical file twice — once here and once on the - // replacement — and an overcount evicts live continuations to make room for bytes that - // are not there. - delete job.supersededSpill; - // A queued job has not captured the candidate in an async frame yet, so release it now. - // A running job retains its accounting until settlement and will discard its stale file. - if (!job.running) releasePendingResponseSpill(job); - return superseded; -} - -function isAclTimeout(error: unknown): boolean { - return !!error && typeof error === "object" && "code" in error - && String((error as { code?: unknown }).code) === "ETIMEDOUT"; -} - -function spillPayloadForResident(candidate: ResidentResponseState): Parameters[1] { - return { - createdAt: candidate.createdAt, - ...(candidate.clientThreadId ? { clientThreadId: candidate.clientThreadId } : {}), - items: candidate.items, - ...(candidate.providerOutputStart !== undefined ? { providerOutputStart: candidate.providerOutputStart } : {}), - ...(candidate.providers ? { providers: candidate.providers } : {}), - }; -} - -async function runPendingResponseSpill(job: PendingResponseSpill): Promise { - if (job.cancelled || !job.candidate) return; - job.running = true; - const candidate = job.candidate; - let ref: ResponseSpillRef | null = null; - let exhaustedAclRetry = false; - let aclRetryFailureOrigin: ResponseSpillWriteFailureOrigin | null = null; - try { - const state = spillPayloadForResident(candidate); - try { - ref = await writeResponseSpillDurablyAsync(job.id, state, { - aclBudgetMs: responseSpillAsyncAclAttemptBudgetMs(), - publicationControl: job.publicationControl, - }); - } catch (error) { - if (!isAclTimeout(error)) throw error; - // The ACL helper permits exactly one caller-owned recovery budget. The resident generation - // remains replayable during both attempts, so a transient timeout never becomes a tombstone. - try { - ref = await writeResponseSpillDurablyAsync(job.id, state, { - aclBudgetMs: responseSpillAsyncAclAttemptBudgetMs(), - retryTimedOutOnce: true, - publicationControl: job.publicationControl, - }); - } catch (retryError) { - exhaustedAclRetry = isAclTimeout(retryError); - // A returned timeout can also mean an exhausted budget before the next OS command. - aclRetryFailureOrigin = spillAclMemoRefusalOrigin(retryError) - ?? (exhaustedAclRetry ? "retry_returned_timeout" : null); - throw retryError; - } - } - if (ref.payloadBytes > responseSpillPayloadCap()) { - deleteResponseSpill(ref); - ref = null; - if (job.directAdmission) admissionCounters.oversizedDrops += 1; - throw Object.assign(new Error("Response spill payload exceeds replay ceiling"), { code: "EFBIG" }); - } - if (states.get(job.id) !== candidate || job.cancelled) { - deleteResponseSpill(ref); - ref = null; - return; - } - if (swapResidentForSpill(job.id, candidate, ref)) { - ref = null; - noteSpillWriteSuccess(); - if (job.directAdmission) admissionCounters.directSpills += 1; - deferSupersededSpill(job.supersededSpill); - } - } catch (error) { - if (ref) deleteResponseSpill(ref); - if (states.get(job.id) === candidate && !job.cancelled) { - noteSpillWriteFailure(error, exhaustedAclRetry ? "EACLRETRYEXHAUSTED" : undefined, aclRetryFailureOrigin); - replaceWithSpillFailure(job.id, candidate); - deferSupersededSpill(job.supersededSpill); - } - } finally { - const cancelled = job.cancelled; - releasePendingResponseSpill(job); - recomputeOldestResident(); - if (!cancelled) { - schedulePersist(); - pruneResponses(); - enforceAppOwnedMemoryBudget(); - } - } -} - -function queuePendingResponseSpill( - id: string, - candidate: ResidentResponseState, - options: { supersededSpill?: ResponseSpillRef; directAdmission?: boolean } = {}, -): void { - const inheritedSpill = cancelPendingResponseSpill(id) ?? options.supersededSpill; - if (pendingResponseSpillBytes + candidate.sizeBytes > MAX_PENDING_RESPONSE_SPILL_BYTES) { - noteSpillWriteFailure(null, "ECAPACITY"); - replaceWithSpillFailure(id, candidate); - deferSupersededSpill(inheritedSpill); - return; - } - // Enforce the disk cap BEFORE the temp or destination file is created. Deleting the - // overflow afterwards is not equivalent: on Windows the file can outlive the decision - // by as long as ACL hardening takes, which is the window the measured 6.8 GiB - // accumulated in. Reclaim first, and only refuse if the peak footprint still does not - // fit — an eviction pass can free a live continuation's worth of room. - const footprint = publicationFootprintBytes(id, candidate); - // The superseded generation this job is about to own is already off `states` and not - // yet on the job, so it is invisible to the walk. Price it here or admission decides - // against a total that is short by a whole envelope. - const inheritedBytes = inheritedSpill?.payloadBytes ?? 0; - if (accountedResponseSpillBytes() + footprint + inheritedBytes > spillByteCap()) { - enforceSpilledResponseBudget(); - if (accountedResponseSpillBytes() + footprint + inheritedBytes > spillByteCap()) { - noteSpillWriteFailure(null, "ECAPACITY"); - replaceWithSpillFailure(id, candidate); - deferSupersededSpill(inheritedSpill); - return; - } - } - const job: PendingResponseSpill = { - id, - candidate, - ...(inheritedSpill ? { supersededSpill: inheritedSpill } : {}), - directAdmission: options.directAdmission === true, - running: false, - cancelled: false, - released: false, - sizeBytes: candidate.sizeBytes, - reservedBytes: footprint, - publicationControl: createResponseSpillPublicationControl(), - }; - pendingResponseSpills.add(job); - pendingResponseSpillById.set(id, job); - pendingResponseSpillBytes += job.sizeBytes; - reservedResponseSpillBytes += job.reservedBytes; - recomputeOldestResident(); - responseSpillPublicationTail = responseSpillPublicationTail - .then(() => runPendingResponseSpill(job), () => runPendingResponseSpill(job)); -} - -function replaceWithPendingResponseSpill( - id: string, - candidate: ResidentResponseState, - expected: StoredResponseState | undefined, - options: { directAdmission?: boolean } = {}, -): boolean { - const inheritedSpill = pendingResponseSpillById.get(id)?.supersededSpill - ?? (expected?.kind === "spill" ? expected.spill : undefined); - if (!replaceMapEntry(id, candidate, expected)) return false; - queuePendingResponseSpill(id, candidate, { - ...(inheritedSpill ? { supersededSpill: inheritedSpill } : {}), - directAdmission: options.directAdmission === true, - }); - return true; -} - -/** Test-only: settle every serialized Windows spill publication. */ -export async function flushPendingResponseSpillsForTests(): Promise { - await drainResponseSpillPublications(); -} - -/** Test-only: observe ordinary queue settlement without invoking shutdown fallback. */ -export async function awaitResponseSpillPublicationTailForTests(): Promise { - await responseSpillPublicationTail; -} - -/** Test-only: observe the bounded queue without exposing payloads. */ -export function pendingResponseSpillMetricsForTests(): { count: number; bytes: number } { - return { count: pendingResponseSpills.size, bytes: pendingResponseSpillBytes }; -} - -/** Test-only: shorten the shutdown drain/fallback budget (null restores production values). */ -export function setResponseSpillShutdownBudgetForTests( - budget: { totalMs: number; fallbackReserveMs: number } | null, -): void { - responseSpillShutdownBudgetOverride = budget; -} - -/** Test-only: shorten the ordinary async whole-attempt ACL budget. */ -export function setResponseSpillAsyncAclAttemptBudgetForTests(budgetMs: number | null): void { - responseSpillAsyncAclAttemptBudgetOverride = budgetMs; -} - -function responseSpillAsyncAclAttemptBudgetMs(): number { - return responseSpillAsyncAclAttemptBudgetOverride ?? RESPONSE_SPILL_ASYNC_ACL_ATTEMPT_BUDGET_MS; -} - -/** Test-only: lower the hard terminalization pass guard (null restores production). */ -export function setResponseSpillShutdownTerminalizationPassLimitForTests(limit: number | null): void { - responseSpillShutdownTerminalizationPassLimitOverride = limit; -} - -function responseSpillShutdownTerminalizationPassLimit(): number { - return responseSpillShutdownTerminalizationPassLimitOverride - ?? RESPONSE_SPILL_SHUTDOWN_TERMINALIZATION_MAX_PASSES; -} - -function responseSpillShutdownBudget(): { totalMs: number; fallbackReserveMs: number } { - return responseSpillShutdownBudgetOverride ?? { - totalMs: RESPONSE_SPILL_SHUTDOWN_BUDGET_MS, - fallbackReserveMs: RESPONSE_SPILL_SHUTDOWN_FALLBACK_RESERVE_MS, - }; -} - -function awaitResponseSpillTailUntil(observed: Promise, deadline: number): Promise { - const remaining = deadline - Date.now(); - if (remaining <= 0) return Promise.resolve(false); - return new Promise(resolve => { - let finished = false; - const finish = (settled: boolean): void => { - if (finished) return; - finished = true; - clearTimeout(timer); - resolve(settled); - }; - const timer = setTimeout(() => finish(false), remaining); - observed.then(() => finish(true), () => finish(true)); - }); -} - -function installShutdownFallbackSpill( - job: PendingResponseSpill, - candidate: ResidentResponseState, - aclBudgetMs: number, -): void { - let ref: ResponseSpillRef | null = null; - // Supersession released this job's reservation, but the synchronous write below is the - // largest publication of the shutdown path and has its own link-then-copy fallback - // holding a temp and a destination at once. Re-reserve for its duration so the cap is - // not blind exactly where the drain does its heaviest work, and settle in `finally` so - // every return, throw and mismatch releases it. - const footprint = publicationFootprintBytes(job.id, candidate); - reservedResponseSpillBytes += footprint; - try { - // Supersession released this job, so its superseded generation is no longer visible - // to the accounting walk — but the file is still on the volume until - // `deferSupersededSpill` or a delete takes it. Price it here or the fallback decides - // against a total short by that whole envelope, which is exactly the gap that lets - // `debt + footprint <= cap < old + debt + footprint` publish over budget. - const supersededBytes = job.supersededSpill?.payloadBytes ?? 0; - // The drain must not publish over the cap either. Reclaim first; if the footprint - // still does not fit — which is what unreclaimable cleanup debt looks like — the - // honest close-out is a tombstone, not another file on a volume that is already - // over budget. `replaceWithSpillFailure` is the same fail-closed ending the budget - // exhaustion path uses, so replay reports `spill_failed` and the client resends. - if (accountedResponseSpillBytes() + supersededBytes > spillByteCap()) { - enforceSpilledResponseBudget(); - if (accountedResponseSpillBytes() + supersededBytes > spillByteCap()) { - if (states.get(job.id) === candidate) { - noteSpillWriteFailure(null, "ECAPACITY"); - replaceWithSpillFailure(job.id, candidate); - deferSupersededSpill(job.supersededSpill); - } - throw Object.assign(new Error("Response spill shutdown fallback exceeds the durable disk cap"), { code: "ENOSPC" }); - } - } - ref = writeResponseSpillDurably(job.id, spillPayloadForResident(candidate), { aclBudgetMs }); - if (ref.payloadBytes > responseSpillPayloadCap()) { - deleteResponseSpill(ref); - ref = null; - if (job.directAdmission) admissionCounters.oversizedDrops += 1; - throw Object.assign(new Error("Response spill payload exceeds replay ceiling"), { code: "EFBIG" }); - } - if (states.get(job.id) !== candidate) { - deleteResponseSpill(ref); - ref = null; - return; - } - if (swapResidentForSpill(job.id, candidate, ref)) { - ref = null; - noteSpillWriteSuccess(); - if (job.directAdmission) admissionCounters.directSpills += 1; - deferSupersededSpill(job.supersededSpill); - } - } catch (error) { - if (ref) deleteResponseSpill(ref); - if (states.get(job.id) === candidate) { - noteSpillWriteFailure(error); - replaceWithSpillFailure(job.id, candidate); - deferSupersededSpill(job.supersededSpill); - } - throw error; - } finally { - reservedResponseSpillBytes = Math.max(0, reservedResponseSpillBytes - footprint); - } -} - -function terminalizeShutdownFallbackCandidate( - job: PendingResponseSpill, - candidate: ResidentResponseState, - failureCode: ResponseSpillWriteFailureCode = "ETIMEDOUT", -): void { - if (states.get(job.id) !== candidate) return; - noteSpillWriteFailure(null, failureCode); - replaceWithSpillFailure(job.id, candidate); - deferSupersededSpill(job.supersededSpill); -} - -function pendingShutdownFallbackCandidates(): Array<{ - job: PendingResponseSpill; - candidate: ResidentResponseState; -}> { - return [...pendingResponseSpills] - .map(job => ({ job, candidate: job.candidate })) - .filter((entry): entry is { job: PendingResponseSpill; candidate: ResidentResponseState } => !!entry.candidate); -} - -function supersedeShutdownFallbackBatch( - pending: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, - failures: Error[], -): void { - for (const { job } of pending) { - job.cancelled = true; - markResponseSpillPublicationSuperseded(job.publicationControl); - } - for (const { job } of pending) { - const cleanupFailure = cleanupSupersededResponseSpillPublication(job.publicationControl); - if (cleanupFailure) { - failures.push(cleanupFailure); - // Cleanup failed, so an async temp or destination is STILL on the volume. Releasing - // the reservation would un-account a file that exists, and the fallback write that - // follows reserves only its own footprint — three envelopes on disk priced as two. - // - // Charge the surviving PATHS rather than a flat two envelopes: `clearOwnedPath` - // nulls whichever it managed to remove, so one failure is one file, not two. The - // charge is settled automatically once the path disappears, which a retried unlink - // or a released Windows lock can still do. - const perPath = Math.max(1, Math.floor(job.reservedBytes / 2)); - chargeUnreclaimableSpillPath(job.publicationControl.tempPath, perPath); - chargeUnreclaimableSpillPath(job.publicationControl.destinationPath, perPath); - } - releasePendingResponseSpill(job); - } -} - -function stopAtShutdownTerminalizationPassLimit( - pending: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, - failures: Error[], -): void { - failures.push(Object.assign(new Error("Response spill shutdown terminalization pass limit exceeded"), { code: "ELOOP" })); - supersedeShutdownFallbackBatch(pending, failures); - for (const { job, candidate } of pending) { - terminalizeShutdownFallbackCandidate(job, candidate, "ELOOP"); - } - for (const [id, state] of [...states]) { - if (state.kind !== "resident") continue; - noteSpillWriteFailure(null, "ELOOP"); - replaceWithSpillFailure(id, state); - } - recomputeOldestResident(); - pruneResponses(); - enforceAppOwnedMemoryBudget(); -} - -function terminalizeExhaustedShutdownFallback( - initial: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, - failures: Error[], -): void { - let pending = initial; - let passes = 0; - const passLimit = responseSpillShutdownTerminalizationPassLimit(); - // Every pass replaces each captured resident with a tombstone. Pruning may expose - // another finite batch, but resident count strictly decreases until none can requeue. - while (pending.length > 0) { - if (passes >= passLimit) { - stopAtShutdownTerminalizationPassLimit(pending, failures); - return; - } - passes += 1; - supersedeShutdownFallbackBatch(pending, failures); - for (const { job, candidate } of pending) { - failures.push(Object.assign(new Error("Response spill shutdown fallback budget exhausted"), { code: "ETIMEDOUT" })); - terminalizeShutdownFallbackCandidate(job, candidate); - } - recomputeOldestResident(); - pruneResponses(); - enforceAppOwnedMemoryBudget(); - pending = pendingShutdownFallbackCandidates(); - } -} - -function fallbackPendingResponseSpills(reserveMs: number): Error[] { - const deadline = Date.now() + reserveMs; - const failures: Error[] = []; - for (;;) { - const pending = pendingShutdownFallbackCandidates(); - if (pending.length === 0) return failures; - if (Date.now() >= deadline) { - terminalizeExhaustedShutdownFallback(pending, failures); - return failures; - } - - supersedeShutdownFallbackBatch(pending, failures); - let reserveExhausted = false; - for (let index = 0; index < pending.length; index += 1) { - const { job, candidate } = pending[index]!; - if (states.get(job.id) !== candidate) continue; - const remaining = deadline - Date.now(); - if (remaining <= 0) { - reserveExhausted = true; - for (const exhausted of pending.slice(index)) { - failures.push(Object.assign(new Error("Response spill shutdown fallback budget exhausted"), { code: "ETIMEDOUT" })); - terminalizeShutdownFallbackCandidate(exhausted.job, exhausted.candidate); - } - break; - } - try { - installShutdownFallbackSpill(job, candidate, remaining); - } catch (error) { - failures.push(error instanceof Error ? error : new Error("Response spill shutdown fallback failed")); - } - } - recomputeOldestResident(); - pruneResponses(); - enforceAppOwnedMemoryBudget(); - if (reserveExhausted || Date.now() >= deadline) { - terminalizeExhaustedShutdownFallback(pendingShutdownFallbackCandidates(), failures); - return failures; - } - } -} - -async function drainResponseSpillPublications(): Promise { - const budget = responseSpillShutdownBudget(); - const fallbackReserveMs = Math.min(budget.totalMs, Math.max(1, budget.fallbackReserveMs)); - const drainDeadline = Date.now() + Math.max(0, budget.totalMs - fallbackReserveMs); - - for (;;) { - if (pendingResponseSpills.size === 0) return; - const observed = responseSpillPublicationTail; - const settled = await awaitResponseSpillTailUntil(observed, drainDeadline); - if (!settled) { - const failures = fallbackPendingResponseSpills(fallbackReserveMs); - if (failures.length > 0) { - throw new AggregateError(failures, "Response spill shutdown fallback incomplete"); - } - return; - } - if (observed === responseSpillPublicationTail) return; - } -} function byteCap(): number { return byteCapOverride ?? MAX_STORED_RESPONSE_BYTES; @@ -920,12 +244,9 @@ function accountedResponseSpillBytes(): number { // counting only `states` plus `pendingSpillUnlinks` loses it for the whole publication // — during a copy fallback that is old generation + new temp + new destination, three // envelopes priced as two. - let ownedBySpillJobs = 0; - for (const job of pendingResponseSpills) { - if (job.supersededSpill) ownedBySpillJobs += job.supersededSpill.payloadBytes; - } - return spilledResponseBytes() + reservedResponseSpillBytes + ownedBySpillJobs - + reconcileUnreclaimableSpillPaths(); + const accounting = spillQueueAccounting(); + return spilledResponseBytes() + accounting.reservedBytes + accounting.jobOwnedBytes + + accounting.unreclaimableBytes; } /** Test-only: lower/restore the durable spill cap (null restores the default). */ @@ -969,7 +290,7 @@ function recomputeOldestResident(): void { oldestResidentAt = null; for (const [id, state] of states) { if (state.kind !== "resident") continue; - if (pendingResponseSpillById.get(id)?.candidate === state) continue; + if (spillQueueHoldsResidentCandidate(id, state)) continue; if (oldestResidentAt !== null && state.createdAt >= oldestResidentAt) continue; oldestResidentId = id; oldestResidentAt = state.createdAt; @@ -1134,8 +455,7 @@ function setResidentEntry(id: string, entry: ResidentInput): void { pruneResponses(); return; } - const pending = pendingResponseSpillById.get(id); - if (windowsSecretAclApplies() && (expected?.kind === "spill" || pending?.supersededSpill)) { + if (windowsSecretAclApplies() && (expected?.kind === "spill" || spillQueueSupersededSpillFor(id))) { replaceWithPendingResponseSpill(id, candidate, expected); pruneResponses(); return; @@ -1219,6 +539,23 @@ function admitOversizedCandidate( } } +bindSpillQueueStore({ + swapResidentForSpill, + replaceWithSpillFailure, + deleteEntry, + deferSupersededSpill, + replaceMapEntry, + currentEntry: (id: string) => states.get(id), + residentEntries: () => [...states], + recomputeOldestResident, + schedulePersist, + pruneResponses, + accountedResponseSpillBytes, + spillByteCap, + enforceSpilledResponseBudget, + terminalizationMaxPasses: () => RESPONSE_SPILL_SHUTDOWN_TERMINALIZATION_MAX_PASSES, +}); + // Replay provenance must stay proxy-private: a WeakMap distinguishes replayed history from the // newly appended input suffix without adding an unknown field that native passthrough could send // upstream. The parser uses this boundary to acknowledge historical compaction markers exactly @@ -1241,284 +578,6 @@ function snapshotPath(): string { return join(getConfigDir(), "responses-state.json"); } -interface LegacySnapshotState { - createdAt?: unknown; - clientThreadId?: unknown; - items?: unknown; - providers?: OcxProviderContinuationState; - conversationId?: unknown; - cursorCheckpointUsable?: unknown; -} - -function isSpillRef(value: unknown): value is ResponseSpillRef { - if (!value || typeof value !== "object" || Array.isArray(value)) return false; - const ref = value as ResponseSpillRef; - return ref.version === 1 - && typeof ref.fileName === "string" - && /^[0-9a-f]{64}$/.test(ref.digest) - && Number.isSafeInteger(ref.payloadBytes) - && ref.payloadBytes >= 0; -} - -function loadSnapshotEntry(id: string, value: unknown): void { - if (!value || typeof value !== "object" || Array.isArray(value)) return; - const rec = value as LegacySnapshotState & { kind?: unknown; spill?: unknown }; - if (typeof rec.createdAt !== "number" || !Number.isFinite(rec.createdAt)) return; - const clientThreadId = typeof rec.clientThreadId === "string" && rec.clientThreadId.trim().length > 0 - ? rec.clientThreadId.trim() - : undefined; - // A malformed boundary degrades to "never skip" rather than to a bad index: an untrusted - // snapshot must not be able to authorize dropping conversation history. - const anchorFor = (itemCount: number): number | undefined => { - const raw = (rec as { providerOutputStart?: unknown }).providerOutputStart; - return Number.isSafeInteger(raw) && (raw as number) >= 0 && (raw as number) <= itemCount - ? raw as number - : undefined; - }; - if (rec.kind === "spill") { - if (!isSpillRef(rec.spill)) return; - const base: Omit = { - kind: "spill", - createdAt: rec.createdAt, - ...(clientThreadId ? { clientThreadId } : {}), - // Item count is unknown until materialization, so accept any non-negative integer - // here; the spill payload validator re-checks it against the real array. - ...(anchorFor(Number.MAX_SAFE_INTEGER) !== undefined ? { providerOutputStart: anchorFor(Number.MAX_SAFE_INTEGER) } : {}), - ...(rec.providers ? { providers: rec.providers } : {}), - spill: rec.spill, - }; - replaceMapEntry(id, { ...base, sizeBytes: stubSize(id, base) }); - return; - } - if (rec.kind === "spill-failed") { - replaceMapEntry(id, tombstone(id, rec.createdAt)); - return; - } - if (rec.kind !== undefined && rec.kind !== "resident") return; - if (!Array.isArray(rec.items)) return; - const providers = rec.providers ?? (typeof rec.conversationId === "string" - ? { - cursor: { - conversationId: rec.conversationId, - ...(typeof rec.cursorCheckpointUsable === "boolean" - ? { checkpointUsable: rec.cursorCheckpointUsable } - : {}), - }, - } - : undefined); - const resident = measureResidentEntry(id, { - createdAt: rec.createdAt, - ...(clientThreadId ? { clientThreadId } : {}), - items: rec.items, - ...(anchorFor(rec.items.length) !== undefined ? { providerOutputStart: anchorFor(rec.items.length) } : {}), - ...(providers ? { providers } : {}), - }); - if (!resident) { - replaceMapEntry(id, tombstone(id, rec.createdAt)); - return; - } - // Same admission boundary as live writes: an oversized snapshot row goes - // straight to spill (or tombstone above the payload ceiling) instead of - // entering the resident map and demoting unrelated rows on the first prune. - if (resident.sizeBytes > byteCap()) { - admitOversizedCandidate(id, resident, undefined); - return; - } - replaceMapEntry(id, resident); -} - -export interface ResponseStateTempRecoveryResult { - matched: number; - removed: number; - failed: number; - bytesRemoved: number; - /** Entries that passed EVERY gate and would be reclaimed. In a dry run nothing is - * unlinked, so this is the only honest count to show an operator: `matched` is - * incremented before the file-type, age, boot-floor, and liveness gates. */ - eligible: number; - /** Total size of the `eligible` entries. */ - eligibleBytes: number; - /** The scan stopped on a budget (entry cap, cleanup cap, or deadline) rather than reaching - * the end of the directory, so the counts below describe a prefix of the backlog and not - * the backlog. `eligible > removed + failed` cannot express this: outside a dry run every - * eligible entry is unlinked or failed on the same iteration, so the two are always equal - * and a comparison between them is dead code. */ - truncated: boolean; -} - -interface ResponseStateTempRecoveryIO { - now: () => number; - /** Approximate epoch ms of the current boot; see the boot floor in the scan loop. */ - bootTime: () => number; - list: (dir: string) => Iterable; - inspect: (path: string) => { isFile: boolean; mtimeMs: number; size: number }; - isProcessAlive: (pid: number) => boolean; - unlink: (path: string) => void; -} - -export type ResponseStateTempRecoveryOptions = Partial & { - maxEntries?: number; - maxCleanups?: number; - /** Wall-clock ceiling for the scan, or null/undefined for no deadline (startup path). */ - deadlineMs?: number | null; - /** Report only: apply every gate, count what would be reclaimed, unlink nothing. */ - dryRun?: boolean; -}; - -function processIsAlive(pid: number): boolean { - if (pid === process.pid) return true; - try { - process.kill(pid, 0); - return true; - } catch (error) { - // EPERM means the process exists but cannot be signalled. Unknown platform errors - // are also protected; cleanup should prefer a false negative over touching a live writer. - return (error as NodeJS.ErrnoException).code !== "ESRCH"; - } -} - -const responseStateTempRecoveryIO: ResponseStateTempRecoveryIO = { - now: Date.now, - bootTime: () => Date.now() - uptime() * 1_000, - list: function* list(dir) { - const handle = opendirSync(dir); - try { - for (let entry = handle.readSync(); entry; entry = handle.readSync()) yield entry.name; - } finally { - handle.closeSync(); - } - }, - inspect: path => { - const stat = lstatSync(path); - return { isFile: stat.isFile() && !stat.isSymbolicLink(), mtimeMs: stat.mtimeMs, size: stat.size }; - }, - isProcessAlive: processIsAlive, - unlink: unlinkSync, -}; - -/** - * Recover only abandoned response-state atomic-write files. The exact basename, - * regular-file check, age gate, and PID liveness check protect unrelated/active files. - * Cleanup is capped and best-effort because continuation state is only a cache. Removal - * deliberately uses unlink only: path-based truncation could follow a replacement symlink. - */ -export function recoverStaleResponseStateTemps( - dir = getConfigDir(), - options: ResponseStateTempRecoveryOptions = {}, -): ResponseStateTempRecoveryResult { - const { - maxEntries = STALE_TEMP_MAX_ENTRIES, - maxCleanups = STALE_TEMP_MAX_CLEANUPS, - deadlineMs = null, - dryRun = false, - ...overrides - } = options; - const io = { ...responseStateTempRecoveryIO, ...overrides }; - const result: ResponseStateTempRecoveryResult = { - matched: 0, - removed: 0, - failed: 0, - bytesRemoved: 0, - eligible: 0, - eligibleBytes: 0, - truncated: false, - }; - const startedAt = io.now(); - // One probe per scan, not one per entry. A non-finite or future-dated boot is anomalous, and - // clamping it to "now" would be the WORST response: the floor would then retire the liveness - // probe for every file older than the skew, which is every file past the grace. Disable it - // instead -- an absent floor only costs a missed reclaim, never a wrong one. - const rawBoot = io.bootTime(); - const bootMs = Number.isFinite(rawBoot) && rawBoot <= startedAt ? rawBoot : Number.NEGATIVE_INFINITY; - let names: Iterable; - try { names = io.list(dir); } catch { return result; } - let iterator: Iterator; - try { iterator = names[Symbol.iterator](); } catch { return result; } - let scanned = 0; - // Every early exit runs through this. The production `list` is a generator that closes its - // directory handle in a `finally`, and a `finally` does NOT run when the consumer simply - // stops calling `next()` -- only `return()` resumes the generator to completion. Breaking - // out of the loop directly therefore leaked one directory handle per truncated scan, and the - // periodic reclaim truncates on purpose (entry cap, cleanup cap, deadline), so on a slow - // filesystem that is a leak per tick, forever. - const stopScan = (): ResponseStateTempRecoveryResult => { - try { iterator.return?.(); } catch { /* closing is best-effort; never fail a reclaim on it */ } - return result; - }; - for (;;) { - let next: IteratorResult; - try { next = iterator.next(); } catch { return result; } - if (next.done) break; - const name = next.value; - scanned += 1; - // A dry run performs no cleanups, so bounding it by the cleanup budget would truncate - // the very report an operator uses to size the problem. - if (scanned > maxEntries) { result.truncated = true; return stopScan(); } - if (!dryRun && result.removed + result.failed >= maxCleanups) { result.truncated = true; return stopScan(); } - if (deadlineMs !== null && io.now() - startedAt > deadlineMs) { result.truncated = true; return stopScan(); } - const match = RESPONSE_STATE_TEMP_NAME.exec(name); - if (!match) continue; - result.matched += 1; - const pid = Number(match[1]); - const sequence = Number(match[2]); - if (!Number.isSafeInteger(pid) || pid <= 0 || !Number.isSafeInteger(sequence) || sequence <= 0) continue; - const path = join(dir, name); - let file: ReturnType; - try { file = io.inspect(path); } catch { continue; } - if (!file.isFile || io.now() - file.mtimeMs < STALE_TEMP_GRACE_MS) continue; - // Boot floor. After a reboot the original writer's pid is routinely reused, which makes - // the liveness skip PERMANENT: the 15-minute grace above is a lower bound and never - // expires it, so the file is skipped on every future pass forever. A temp older than - // this boot cannot be owned by the pid we would probe, so the probe is vacuous and we - // retire it. This does NOT claim the file is provably dead: under a shared-volume - // container, suspend-excluding uptime, or a network config dir the computed boot can - // land after the real one. The unconditional 15-minute grace above remains the safety - // floor, and this process's own temps are never touched. - const predatesBoot = file.mtimeMs < bootMs - BOOT_FLOOR_SKEW_MS; - if (pid === process.pid) continue; - if (!predatesBoot && io.isProcessAlive(pid)) continue; - - result.eligible += 1; - result.eligibleBytes += file.size; - if (dryRun) continue; - - try { - io.unlink(path); - result.removed += 1; - result.bytesRemoved += file.size; - } catch (error) { - // Another proxy sharing this config dir may have won the race. A file that is already - // gone is reclaimed, not a failure -- reporting it as one would surface "in use or - // locked" to an operator for a file nobody holds. - if ((error as NodeJS.ErrnoException)?.code === "ENOENT") { - result.removed += 1; - continue; - } - // Locked files remain for a later startup. Do not truncate by path: a same-user - // replacement could turn that fallback into an arbitrary symlink-target write. - result.failed += 1; - } - } - return result; -} - -/** - * Literal config dir plus the snapshot's resolved dir. Atomic writes place their temp beside - * the RESOLVED target, so a symlinked snapshot (dotfiles-managed config dir) strands temps in - * the link's real directory where a scan of the literal dir would never see them. The two - * collapse to one when nothing is symlinked. - */ -function responseStateSweepDirectories(): Set { - const path = snapshotPath(); - let resolvedDir = dirname(path); - try { - resolvedDir = dirname(resolveWriteTarget(path)); - } catch { - /* unresolvable link: sweep the literal dir only */ - } - return new Set([dirname(path), resolvedDir]); -} - /** * Best-effort disk snapshot so previous_response_id chains survive a proxy restart (the * dominant expansion-miss cause: an in-memory-only store dies with the process, and the next @@ -1565,7 +624,14 @@ function ensureLoaded(): void { if ((raw.version === 1 || raw.version === 2) && Array.isArray(raw.states)) { for (const entry of raw.states) { if (!Array.isArray(entry) || entry.length !== 2 || typeof entry[0] !== "string") continue; - loadSnapshotEntry(entry[0], entry[1]); + loadSnapshotEntry(entry[0], entry[1], { + replaceMapEntry, + stubSize, + tombstone, + measureResidentEntry, + admitOversizedCandidate, + byteCap, + }); } } } @@ -1748,89 +814,8 @@ function inputItems(input: unknown): unknown[] { return [input]; } -/** Hard cap for canonicalizing ANY item. Past it, the item is not comparable. */ -const REPLAY_FINGERPRINT_MAX_BYTES = 8 * 1024; -/** Depth ceiling so a pathologically nested item cannot blow the canonicalizer. */ -const REPLAY_FINGERPRINT_MAX_DEPTH = 64; - let replayOverlapSkips = 0; -/** - * Canonical, order-stable fingerprint for one input item, or null when the item cannot be - * compared safely. - * - * Byte-counted DURING the walk rather than serialize-then-measure: a tool result can be - * megabytes and this runs on the request path, so the point of the cap is to stop early, - * not to discover afterwards that we should have. Object keys are sorted so two - * semantically identical items cannot differ by key order alone. - * - * The cap applies to EVERY item. An `id`/`call_id` is additional occurrence evidence, never - * a substitute for content equality, so an over-cap identified tool item is non-comparable - * exactly like an over-cap message. - */ -function replayItemFingerprint(item: unknown): string | null { - const out: string[] = []; - let bytes = 0; - const push = (text: string): boolean => { - bytes += Buffer.byteLength(text, "utf8"); - if (bytes > REPLAY_FINGERPRINT_MAX_BYTES) return false; - out.push(text); - return true; - }; - const walk = (value: unknown, depth: number): boolean => { - if (depth > REPLAY_FINGERPRINT_MAX_DEPTH) return false; - if (value === null || typeof value !== "object") return push(JSON.stringify(value) ?? "null"); - if (Array.isArray(value)) { - if (!push("[")) return false; - for (const element of value) { - if (!walk(element, depth + 1)) return false; - if (!push(",")) return false; - } - return push("]"); - } - if (!push("{")) return false; - for (const key of Object.keys(value as Record).sort()) { - if (!push(JSON.stringify(key))) return false; - if (!walk((value as Record)[key], depth + 1)) return false; - if (!push(",")) return false; - } - return push("}"); - }; - return walk(item, 0) ? out.join("") : null; -} - -/** Non-empty provider-issued `id`/`call_id` on an item, else null. */ -function providerIssuedIdentity(item: unknown): string | null { - if (!item || typeof item !== "object" || Array.isArray(item)) return null; - const record = item as { id?: unknown; call_id?: unknown }; - for (const candidate of [record.id, record.call_id]) { - if (typeof candidate === "string" && candidate.trim().length > 0) return candidate; - } - return null; -} - -/** - * Number of leading stored items the client already carries verbatim, or 0. - * - * Requires an exact ordered run: every stored item must match the client input item at the - * same index. Any not-comparable item aborts to 0 — skipping just that item could align two - * different occurrences and manufacture a false positive, and a false positive here deletes - * real conversation history. - * - * Known gap (FU-2): stored input can contain proxy-injected guidance the client never saw, - * and ids repaired after recording. Those sessions do not match here and expand as before. - */ -function clientCarriedPrefixLength(stored: readonly unknown[], clientInput: readonly unknown[]): number { - if (stored.length === 0 || clientInput.length < stored.length) return 0; - for (let index = 0; index < stored.length; index += 1) { - const storedPrint = replayItemFingerprint(stored[index]); - if (storedPrint === null) return 0; - const clientPrint = replayItemFingerprint(clientInput[index]); - if (clientPrint === null || storedPrint !== clientPrint) return 0; - } - return stored.length; -} - /** Test-only: replay prepends skipped because the client already carried the history. */ export function replayOverlapSkipsForTests(): number { return replayOverlapSkips; @@ -1903,9 +888,9 @@ function pruneResponses(at = now()): void { // deleted only when even their bounded metadata cannot fit the override. while (storedResponseBytes > byteCap() && states.size > 0) { const oldestResident = [...states].find(([id, entry]) => entry.kind === "resident" - && pendingResponseSpillById.get(id)?.candidate !== entry); + && !spillQueueHoldsResidentCandidate(id, entry)); const hasPendingResident = !oldestResident && [...states].some(([id, entry]) => entry.kind === "resident" - && pendingResponseSpillById.get(id)?.candidate === entry); + && spillQueueHoldsResidentCandidate(id, entry)); if (hasPendingResident) break; const oldestId = oldestResident?.[0] ?? states.keys().next().value as string | undefined; if (!oldestId) break; @@ -1950,70 +935,12 @@ export function sweepExpiredResponseStates(at = now()): number { return removed; } -/** - * Periodic disk reclaim for abandoned atomic-write temps. - * - * `ensureLoaded` sweeps once per process, at load, BEFORE that process writes anything: - * every `schedulePersist` site is downstream of it. So a process that abandons a temp has - * already had its only look, the 15-minute grace hides the temp its predecessor's crash - * just produced, and `maxCleanups` caps a single pass below a large backlog. A restart - * loop therefore accumulates monotonically. Repeating the reclaim on a timer fixes all - * three: the grace expires into a later tick and the per-pass cap becomes a per-tick rate. - * - * Registered on the sweeper's LIVENESS tick, not the TTL tick: `sweepExpiredOnWrite` puts - * `sweepExpired` on hot write paths, and a directory scan does not belong there. - */ -export function reclaimAbandonedResponseStateTemps( - options: ResponseStateTempRecoveryOptions = {}, -): ResponseStateTempRecoveryResult { - const total: ResponseStateTempRecoveryResult = { - matched: 0, removed: 0, failed: 0, bytesRemoved: 0, eligible: 0, eligibleBytes: 0, truncated: false, - }; - // The try encloses responseStateSweepDirectories() deliberately: recoverStaleResponseStateTemps - // already swallows its own enumeration failures, so a catch around only that call would be - // unreachable. snapshotPath()/getConfigDir() are the paths that can genuinely throw. - try { - for (const dir of responseStateSweepDirectories()) { - const result = recoverStaleResponseStateTemps(dir, options); - total.matched += result.matched; - total.removed += result.removed; - total.failed += result.failed; - total.bytesRemoved += result.bytesRemoved; - total.eligible += result.eligible; - total.eligibleBytes += result.eligibleBytes; - // Truncation anywhere makes the whole total a prefix. - total.truncated ||= result.truncated; - } - } catch { - /* best-effort: disk reclaim must never destabilize the caller */ - } - return total; -} - -/** - * Report-only counterpart for `ocx doctor`: applies every selection gate and unlinks - * nothing. It runs the SAME predicate as the reclaim, so the report and the subsequent - * removal cannot disagree about which files are reclaimable. - */ -export function inspectAbandonedResponseStateTemps(): ResponseStateTempRecoveryResult { - return reclaimAbandonedResponseStateTemps({ dryRun: true }); -} - -/** Sweeper adapter: narrows the reclaim to the `() => number` the liveness tick expects. */ -export function sweepAbandonedResponseStateTemps(): number { - return reclaimAbandonedResponseStateTemps({ - maxEntries: PERIODIC_TEMP_MAX_ENTRIES, - maxCleanups: PERIODIC_TEMP_MAX_CLEANUPS, - deadlineMs: PERIODIC_TEMP_SCAN_DEADLINE_MS, - }).removed; -} - export function responseContinuationRetainedStoreSnapshot(): RetainedStoreSnapshot { let currentPendingBytes = 0; - for (const job of pendingResponseSpills) { - if (job.candidate && states.get(job.id) === job.candidate) currentPendingBytes += job.sizeBytes; + for (const job of spillQueueResidentCandidates()) { + if (states.get(job.id) === job.candidate) currentPendingBytes += job.sizeBytes; } - const detachedPendingBytes = Math.max(0, pendingResponseSpillBytes - currentPendingBytes); + const detachedPendingBytes = Math.max(0, spillQueuePendingBytes() - currentPendingBytes); const bytes = storedResponseBytes + detachedPendingBytes; const evictableBytes = Math.max(0, residentResponseBytes - currentPendingBytes); return { @@ -2390,8 +1317,7 @@ export function clearResponseStateMemoryForTests(): void { persistTimer = null; } pendingPersistPath = null; - for (const id of [...pendingResponseSpillById.keys()]) cancelPendingResponseSpill(id); - pendingResponseSpillById.clear(); + resetSpillQueueForTests(); states.clear(); storedResponseBytes = 0; residentResponseBytes = 0; @@ -2421,8 +1347,6 @@ export function clearResponseStateMemoryForTests(): void { export function clearResponseStateForTests(): void { for (const entry of states.values()) deleteOwnedSpills(entry); clearResponseStateMemoryForTests(); - reservedResponseSpillBytes = 0; - unreclaimableSpillPaths.clear(); try { unlinkSync(snapshotPath()); } catch { diff --git a/src/responses/state/replay-fingerprint.ts b/src/responses/state/replay-fingerprint.ts new file mode 100644 index 0000000000..759bd22eeb --- /dev/null +++ b/src/responses/state/replay-fingerprint.ts @@ -0,0 +1,80 @@ +/** Hard cap for canonicalizing ANY item. Past it, the item is not comparable. */ +const REPLAY_FINGERPRINT_MAX_BYTES = 8 * 1024; +/** Depth ceiling so a pathologically nested item cannot blow the canonicalizer. */ +const REPLAY_FINGERPRINT_MAX_DEPTH = 64; + +/** + * Canonical, order-stable fingerprint for one input item, or null when the item cannot be + * compared safely. + * + * Byte-counted DURING the walk rather than serialize-then-measure: a tool result can be + * megabytes and this runs on the request path, so the point of the cap is to stop early, + * not to discover afterwards that we should have. Object keys are sorted so two + * semantically identical items cannot differ by key order alone. + * + * The cap applies to EVERY item. An `id`/`call_id` is additional occurrence evidence, never + * a substitute for content equality, so an over-cap identified tool item is non-comparable + * exactly like an over-cap message. + */ +function replayItemFingerprint(item: unknown): string | null { + const out: string[] = []; + let bytes = 0; + const push = (text: string): boolean => { + bytes += Buffer.byteLength(text, "utf8"); + if (bytes > REPLAY_FINGERPRINT_MAX_BYTES) return false; + out.push(text); + return true; + }; + const walk = (value: unknown, depth: number): boolean => { + if (depth > REPLAY_FINGERPRINT_MAX_DEPTH) return false; + if (value === null || typeof value !== "object") return push(JSON.stringify(value) ?? "null"); + if (Array.isArray(value)) { + if (!push("[")) return false; + for (const element of value) { + if (!walk(element, depth + 1)) return false; + if (!push(",")) return false; + } + return push("]"); + } + if (!push("{")) return false; + for (const key of Object.keys(value as Record).sort()) { + if (!push(JSON.stringify(key))) return false; + if (!walk((value as Record)[key], depth + 1)) return false; + if (!push(",")) return false; + } + return push("}"); + }; + return walk(item, 0) ? out.join("") : null; +} + +/** Non-empty provider-issued `id`/`call_id` on an item, else null. */ +export function providerIssuedIdentity(item: unknown): string | null { + if (!item || typeof item !== "object" || Array.isArray(item)) return null; + const record = item as { id?: unknown; call_id?: unknown }; + for (const candidate of [record.id, record.call_id]) { + if (typeof candidate === "string" && candidate.trim().length > 0) return candidate; + } + return null; +} + +/** + * Number of leading stored items the client already carries verbatim, or 0. + * + * Requires an exact ordered run: every stored item must match the client input item at the + * same index. Any not-comparable item aborts to 0 — skipping just that item could align two + * different occurrences and manufacture a false positive, and a false positive here deletes + * real conversation history. + * + * Known gap (FU-2): stored input can contain proxy-injected guidance the client never saw, + * and ids repaired after recording. Those sessions do not match here and expand as before. + */ +export function clientCarriedPrefixLength(stored: readonly unknown[], clientInput: readonly unknown[]): number { + if (stored.length === 0 || clientInput.length < stored.length) return 0; + for (let index = 0; index < stored.length; index += 1) { + const storedPrint = replayItemFingerprint(stored[index]); + if (storedPrint === null) return 0; + const clientPrint = replayItemFingerprint(clientInput[index]); + if (clientPrint === null || storedPrint !== clientPrint) return 0; + } + return stored.length; +} diff --git a/src/responses/state/snapshot-codec.ts b/src/responses/state/snapshot-codec.ts new file mode 100644 index 0000000000..c5a53d1432 --- /dev/null +++ b/src/responses/state/snapshot-codec.ts @@ -0,0 +1,104 @@ +import type { + ResidentInput, + ResidentResponseState, + SpillFailedResponseState, + SpilledResponseState, + StoredResponseState, +} from "../state"; +import type { ResponseSpillRef } from "../spill-store"; +import type { OcxProviderContinuationState } from "../../types"; + +export interface SnapshotLoadStore { + replaceMapEntry(id: string, next: StoredResponseState, expected?: StoredResponseState): boolean; + stubSize(id: string, entry: Omit): number; + tombstone(id: string, createdAt: number): SpillFailedResponseState; + measureResidentEntry(id: string, entry: ResidentInput): ResidentResponseState | null; + admitOversizedCandidate(id: string, candidate: ResidentResponseState, expected?: StoredResponseState): void; + byteCap(): number; +} + +interface LegacySnapshotState { + createdAt?: unknown; + clientThreadId?: unknown; + items?: unknown; + providers?: OcxProviderContinuationState; + conversationId?: unknown; + cursorCheckpointUsable?: unknown; +} + +function isSpillRef(value: unknown): value is ResponseSpillRef { + if (!value || typeof value !== "object" || Array.isArray(value)) return false; + const ref = value as ResponseSpillRef; + return ref.version === 1 + && typeof ref.fileName === "string" + && /^[0-9a-f]{64}$/.test(ref.digest) + && Number.isSafeInteger(ref.payloadBytes) + && ref.payloadBytes >= 0; +} + +export function loadSnapshotEntry(id: string, value: unknown, store: SnapshotLoadStore): void { + if (!value || typeof value !== "object" || Array.isArray(value)) return; + const rec = value as LegacySnapshotState & { kind?: unknown; spill?: unknown }; + if (typeof rec.createdAt !== "number" || !Number.isFinite(rec.createdAt)) return; + const clientThreadId = typeof rec.clientThreadId === "string" && rec.clientThreadId.trim().length > 0 + ? rec.clientThreadId.trim() + : undefined; + // A malformed boundary degrades to "never skip" rather than to a bad index: an untrusted + // snapshot must not be able to authorize dropping conversation history. + const anchorFor = (itemCount: number): number | undefined => { + const raw = (rec as { providerOutputStart?: unknown }).providerOutputStart; + return Number.isSafeInteger(raw) && (raw as number) >= 0 && (raw as number) <= itemCount + ? raw as number + : undefined; + }; + if (rec.kind === "spill") { + if (!isSpillRef(rec.spill)) return; + const base: Omit = { + kind: "spill", + createdAt: rec.createdAt, + ...(clientThreadId ? { clientThreadId } : {}), + // Item count is unknown until materialization, so accept any non-negative integer + // here; the spill payload validator re-checks it against the real array. + ...(anchorFor(Number.MAX_SAFE_INTEGER) !== undefined ? { providerOutputStart: anchorFor(Number.MAX_SAFE_INTEGER) } : {}), + ...(rec.providers ? { providers: rec.providers } : {}), + spill: rec.spill, + }; + store.replaceMapEntry(id, { ...base, sizeBytes: store.stubSize(id, base) }); + return; + } + if (rec.kind === "spill-failed") { + store.replaceMapEntry(id, store.tombstone(id, rec.createdAt)); + return; + } + if (rec.kind !== undefined && rec.kind !== "resident") return; + if (!Array.isArray(rec.items)) return; + const providers = rec.providers ?? (typeof rec.conversationId === "string" + ? { + cursor: { + conversationId: rec.conversationId, + ...(typeof rec.cursorCheckpointUsable === "boolean" + ? { checkpointUsable: rec.cursorCheckpointUsable } + : {}), + }, + } + : undefined); + const resident = store.measureResidentEntry(id, { + createdAt: rec.createdAt, + ...(clientThreadId ? { clientThreadId } : {}), + items: rec.items, + ...(anchorFor(rec.items.length) !== undefined ? { providerOutputStart: anchorFor(rec.items.length) } : {}), + ...(providers ? { providers } : {}), + }); + if (!resident) { + store.replaceMapEntry(id, store.tombstone(id, rec.createdAt)); + return; + } + // Same admission boundary as live writes: an oversized snapshot row goes + // straight to spill (or tombstone above the payload ceiling) instead of + // entering the resident map and demoting unrelated rows on the first prune. + if (resident.sizeBytes > store.byteCap()) { + store.admitOversizedCandidate(id, resident, undefined); + return; + } + store.replaceMapEntry(id, resident); +} diff --git a/src/responses/state/spill-failure.ts b/src/responses/state/spill-failure.ts new file mode 100644 index 0000000000..fbba526590 --- /dev/null +++ b/src/responses/state/spill-failure.ts @@ -0,0 +1,118 @@ +export const spillCounters = { + writes: 0, writeFailures: 0, readFailures: 0, + aclRetryReturnedTimeouts: 0, aclTimeoutMemoRefusals: 0, +}; + +export type ResponseSpillWriteFailureCode = + | "EACLRETRYEXHAUSTED" + | "ETIMEDOUT" + | "EACCES" + | "ENOSPC" + | "EFBIG" + | "EIO" + | "ECAPACITY" + | "ELOOP" + | "EUNKNOWN"; + +export type ResponseSpillWriteStatus = "initial" | "healthy" | "degraded"; + +export type ResponseSpillWriteFailureOrigin = + | "retry_returned_timeout" + | "timeout_memo_refusal"; + +interface ResponseSpillWriteHealth { + consecutiveFailures: number; + lastFailureCode: ResponseSpillWriteFailureCode | null; + lastFailureOrigin: ResponseSpillWriteFailureOrigin | null; + lastFailureAt: number | null; + lastSuccessAt: number | null; +} + +export const spillWriteHealth: ResponseSpillWriteHealth = { + consecutiveFailures: 0, + lastFailureCode: null, + lastFailureOrigin: null, + lastFailureAt: null, + lastSuccessAt: null, +}; + +/** + * Collapse filesystem/runtime errors into a fixed privacy-safe diagnostic union. + * Messages and paths are deliberately ignored: this projection is returned by the + * authenticated memory endpoint, and a nested `cause` can contain a username or + * workspace path even when the public wrapper does not. + */ +function classifySpillWriteFailure(error: unknown): ResponseSpillWriteFailureCode { + let cursor = error; + for (let depth = 0; depth < 4 && cursor && typeof cursor === "object"; depth += 1) { + const record = cursor as { code?: unknown; cause?: unknown }; + const code = typeof record.code === "string" ? record.code.toUpperCase() : ""; + switch (code) { + case "EACLRETRYEXHAUSTED": return "EACLRETRYEXHAUSTED"; + case "ETIMEDOUT": return "ETIMEDOUT"; + case "EACCES": + case "EPERM": return "EACCES"; + case "ENOSPC": + case "EDQUOT": return "ENOSPC"; + case "EFBIG": return "EFBIG"; + case "EIO": return "EIO"; + case "ECAPACITY": return "ECAPACITY"; + case "ELOOP": return "ELOOP"; + } + cursor = record.cause; + } + return "EUNKNOWN"; +} + +/** The spill writer preserves ACL errors in cause; only a fixed memo marker is diagnostic. */ +export function spillAclMemoRefusalOrigin(error: unknown): "timeout_memo_refusal" | null { + let cursor = error; + for (let depth = 0; depth < 4 && cursor && typeof cursor === "object"; depth += 1) { + const record = cursor as { code?: unknown; aclFailureOrigin?: unknown; cause?: unknown }; + if ((record.code === "ETIMEDOUT" || record.code === "EACLRETRYEXHAUSTED") + && record.aclFailureOrigin === "timeout_memo_refusal") { + return "timeout_memo_refusal"; + } + cursor = record.cause; + } + return null; +} + +export function noteSpillWriteSuccess(): void { + spillCounters.writes += 1; + spillWriteHealth.consecutiveFailures = 0; + spillWriteHealth.lastSuccessAt = Date.now(); +} + +export function noteSpillWriteFailure( + error: unknown, + override?: ResponseSpillWriteFailureCode, + retryOrigin: ResponseSpillWriteFailureOrigin | null = null, +): void { + const code = override ?? classifySpillWriteFailure(error); + const origin = code === "ETIMEDOUT" || code === "EACLRETRYEXHAUSTED" + ? spillAclMemoRefusalOrigin(error) ?? retryOrigin + : null; + spillCounters.writeFailures += 1; + spillWriteHealth.consecutiveFailures += 1; + spillWriteHealth.lastFailureCode = code; + spillWriteHealth.lastFailureOrigin = origin; + spillWriteHealth.lastFailureAt = Date.now(); + // Count terminal publications, not ACL calls or a transient first attempt. + if (origin === "retry_returned_timeout") spillCounters.aclRetryReturnedTimeouts += 1; + else if (origin === "timeout_memo_refusal") spillCounters.aclTimeoutMemoRefusals += 1; +} +/** + * Admission-boundary observability (test-visible). directSpills: oversized + * candidates routed straight to durable spill without a resident stay or + * unrelated demotion. oversizedDrops: candidates above the single-spill + * payload ceiling, tombstoned instead of retained. snapshotOversizedRefusals: + * snapshot files refused before parse. + */ +export const admissionCounters = { directSpills: 0, oversizedDrops: 0, snapshotOversizedRefusals: 0 }; + + +/** Test-only: admission-boundary counters (proves the new paths fire). */ +export function responseAdmissionCountersForTests(): Readonly { + return admissionCounters; +} diff --git a/src/responses/state/spill-queue.ts b/src/responses/state/spill-queue.ts new file mode 100644 index 0000000000..a99b2eb3da --- /dev/null +++ b/src/responses/state/spill-queue.ts @@ -0,0 +1,665 @@ +import { existsSync } from "node:fs"; +import { + cleanupSupersededResponseSpillPublication, + createResponseSpillPublicationControl, + deleteResponseSpill, + markResponseSpillPublicationSuperseded, + MAX_RESPONSE_SPILL_PAYLOAD_BYTES, + prospectiveResponseSpillBytes, + responseSpillPayloadCap, + type ResponseSpillPublicationControl, + type ResponseSpillRef, + writeResponseSpillDurably, + writeResponseSpillDurablyAsync, +} from "../spill-store"; +import { enforceAppOwnedMemoryBudget } from "../../lib/app-owned-memory"; +import { + admissionCounters, + noteSpillWriteFailure, + noteSpillWriteSuccess, + spillAclMemoRefusalOrigin, + type ResponseSpillWriteFailureCode, + type ResponseSpillWriteFailureOrigin, +} from "./spill-failure"; +import type { ResidentResponseState, StoredResponseState } from "../state"; + +const RESPONSE_SPILL_SHUTDOWN_BUDGET_MS = 5_000; +const RESPONSE_SPILL_SHUTDOWN_FALLBACK_RESERVE_MS = 4_000; +const RESPONSE_SPILL_ASYNC_ACL_ATTEMPT_BUDGET_MS = 30_000; + +export interface SpillQueueStore { + swapResidentForSpill(id: string, expected: ResidentResponseState, ref: ResponseSpillRef): boolean; + replaceWithSpillFailure(id: string, expected?: StoredResponseState, options?: { deferSpillUnlink?: boolean }): void; + deleteEntry(id: string, options?: { deleteSpill?: boolean }): void; + deferSupersededSpill(ref: ResponseSpillRef | undefined): void; + replaceMapEntry(id: string, next: StoredResponseState, expected?: StoredResponseState): boolean; + currentEntry(id: string): StoredResponseState | undefined; + residentEntries(): Array<[string, StoredResponseState]>; + recomputeOldestResident(): void; + schedulePersist(): void; + pruneResponses(): void; + accountedResponseSpillBytes(): number; + spillByteCap(): number; + enforceSpilledResponseBudget(): number; + terminalizationMaxPasses(): number; +} + +let store: SpillQueueStore | null = null; + +export function bindSpillQueueStore(next: SpillQueueStore): void { + store = next; +} + +function requireStore(): SpillQueueStore { + if (!store) throw new Error("spill-queue store is not bound"); + return store; +} + +/** + * Windows keeps the candidate replayable while required ACL hardening runs off the event loop. + * Pending bytes are pinned, not evictable; cap them below the process-owned 512 MiB ceiling so an + * icacls outage cannot turn the serialized queue into an unbounded resident backlog. + */ +const MAX_PENDING_RESPONSE_SPILL_BYTES = MAX_RESPONSE_SPILL_PAYLOAD_BYTES; + +interface PendingResponseSpill { + id: string; + candidate: ResidentResponseState | null; + supersededSpill?: ResponseSpillRef; + directAdmission: boolean; + running: boolean; + cancelled: boolean; + released: boolean; + sizeBytes: number; + /** Peak on-disk bytes reserved for this publication; released exactly once on settle. */ + reservedBytes: number; + publicationControl: ResponseSpillPublicationControl; +} + +const pendingResponseSpills = new Set(); +const pendingResponseSpillById = new Map(); +let pendingResponseSpillBytes = 0; +/** + * On-disk bytes a queued publication is about to occupy but has not yet installed into + * `states`. + * + * `spilledResponseBytes()` walks installed spills and deferred unlinks — files that + * already exist. It cannot see one that `writeResponseSpillDurablyAsync` is in the + * middle of creating, and on Windows that middle can last as long as `icacls` takes. + * Without a reservation the cap holds only when writes are fast, which is not a cap. + * + * The reserved figure is the PEAK footprint, not the payload: publication can fall back + * from hard-linking to an exclusive copy, and during that fallback the destination copy + * and the temp file exist simultaneously. Reserving one envelope would leave the overshoot + * intact at half its magnitude. + * + * Ownership is single: a job holds its reservation from queue until + * `releasePendingResponseSpill`, which every exit from the publication path reaches + * through the `finally` in `runPendingResponseSpill` and through cancellation of a + * not-yet-running job. A leaked reservation is monotonic — it would ratchet the usable + * cap toward zero — so the release must stay on the settlement path rather than in a + * parallel bookkeeping pass. + */ +let reservedResponseSpillBytes = 0; +/** + * Paths a failed cleanup left on the volume, with the bytes each one occupies. + * + * A failed unlink leaves a real file behind, so the cap has to keep seeing it. But a + * never-decremented total would be phantom debt: a Windows lock that clears a moment + * later, or the async writer's own retry, can remove the file while the charge stays + * forever — and with 256 MiB payloads two conservative charges consume the whole default + * cap, after which nothing can spill for the life of the process. + * + * So the debt is per PATH, priced at what that path actually holds, and settled the + * moment the path is gone. `reconcileUnreclaimableSpillPaths` re-checks on every read of + * the accounted total, which is the same tick that would otherwise refuse an admission. + */ +const unreclaimableSpillPaths = new Map(); + +function chargeUnreclaimableSpillPath(path: string | null | undefined, bytes: number): void { + if (!path || bytes <= 0) return; + unreclaimableSpillPaths.set(path, bytes); +} + +/** Drop charges for paths that have since disappeared; returns the surviving total. */ +function reconcileUnreclaimableSpillPaths(): number { + let total = 0; + for (const [path, bytes] of [...unreclaimableSpillPaths]) { + if (existsSync(path)) total += bytes; + else unreclaimableSpillPaths.delete(path); + } + return total; +} + +/** + * Peak on-disk footprint of publishing this candidate: temp plus destination copy. + * + * Measured from the production serializer rather than from `candidate.sizeBytes`. The + * resident measurement omits the `version` field the published envelope carries, so + * pricing an admission by it undercounts and lets a request sitting exactly at the cap + * still exceed it. Falls back to the resident figure only when serialization fails, which + * is the same condition that will fail the publication itself. + */ +function publicationFootprintBytes(id: string, candidate: ResidentResponseState): number { + const exact = prospectiveResponseSpillBytes(id, spillPayloadForResident(candidate)); + return (exact ?? candidate.sizeBytes) * 2; +} +let responseSpillPublicationTail: Promise = Promise.resolve(); +let responseSpillShutdownBudgetOverride: { totalMs: number; fallbackReserveMs: number } | null = null; +let responseSpillShutdownTerminalizationPassLimitOverride: number | null = null; +let responseSpillAsyncAclAttemptBudgetOverride: number | null = null; + +function releasePendingResponseSpill(job: PendingResponseSpill): void { + if (job.released) return; + job.released = true; + pendingResponseSpillBytes = Math.max(0, pendingResponseSpillBytes - job.sizeBytes); + reservedResponseSpillBytes = Math.max(0, reservedResponseSpillBytes - job.reservedBytes); + pendingResponseSpills.delete(job); + if (pendingResponseSpillById.get(job.id) === job) pendingResponseSpillById.delete(job.id); + job.candidate = null; +} + +export function cancelPendingResponseSpill(id: string): ResponseSpillRef | undefined { + const job = pendingResponseSpillById.get(id); + if (!job) return undefined; + pendingResponseSpillById.delete(id); + job.cancelled = true; + markResponseSpillPublicationSuperseded(job.publicationControl); + const superseded = job.supersededSpill; + // Ownership TRANSFERS to the caller. Leaving the ref on the cancelled job would let the + // accounting walk count the same physical file twice — once here and once on the + // replacement — and an overcount evicts live continuations to make room for bytes that + // are not there. + delete job.supersededSpill; + // A queued job has not captured the candidate in an async frame yet, so release it now. + // A running job retains its accounting until settlement and will discard its stale file. + if (!job.running) releasePendingResponseSpill(job); + return superseded; +} + +function isAclTimeout(error: unknown): boolean { + return !!error && typeof error === "object" && "code" in error + && String((error as { code?: unknown }).code) === "ETIMEDOUT"; +} + +function spillPayloadForResident(candidate: ResidentResponseState): Parameters[1] { + return { + createdAt: candidate.createdAt, + ...(candidate.clientThreadId ? { clientThreadId: candidate.clientThreadId } : {}), + items: candidate.items, + ...(candidate.providerOutputStart !== undefined ? { providerOutputStart: candidate.providerOutputStart } : {}), + ...(candidate.providers ? { providers: candidate.providers } : {}), + }; +} + +async function runPendingResponseSpill(job: PendingResponseSpill): Promise { + if (job.cancelled || !job.candidate) return; + job.running = true; + const candidate = job.candidate; + let ref: ResponseSpillRef | null = null; + let exhaustedAclRetry = false; + let aclRetryFailureOrigin: ResponseSpillWriteFailureOrigin | null = null; + try { + const state = spillPayloadForResident(candidate); + try { + ref = await writeResponseSpillDurablyAsync(job.id, state, { + aclBudgetMs: responseSpillAsyncAclAttemptBudgetMs(), + publicationControl: job.publicationControl, + }); + } catch (error) { + if (!isAclTimeout(error)) throw error; + // The ACL helper permits exactly one caller-owned recovery budget. The resident generation + // remains replayable during both attempts, so a transient timeout never becomes a tombstone. + try { + ref = await writeResponseSpillDurablyAsync(job.id, state, { + aclBudgetMs: responseSpillAsyncAclAttemptBudgetMs(), + retryTimedOutOnce: true, + publicationControl: job.publicationControl, + }); + } catch (retryError) { + exhaustedAclRetry = isAclTimeout(retryError); + // A returned timeout can also mean an exhausted budget before the next OS command. + aclRetryFailureOrigin = spillAclMemoRefusalOrigin(retryError) + ?? (exhaustedAclRetry ? "retry_returned_timeout" : null); + throw retryError; + } + } + if (ref.payloadBytes > responseSpillPayloadCap()) { + deleteResponseSpill(ref); + ref = null; + if (job.directAdmission) admissionCounters.oversizedDrops += 1; + throw Object.assign(new Error("Response spill payload exceeds replay ceiling"), { code: "EFBIG" }); + } + if (requireStore().currentEntry(job.id) !== candidate || job.cancelled) { + deleteResponseSpill(ref); + ref = null; + return; + } + if (requireStore().swapResidentForSpill(job.id, candidate, ref)) { + ref = null; + noteSpillWriteSuccess(); + if (job.directAdmission) admissionCounters.directSpills += 1; + requireStore().deferSupersededSpill(job.supersededSpill); + } + } catch (error) { + if (ref) deleteResponseSpill(ref); + if (requireStore().currentEntry(job.id) === candidate && !job.cancelled) { + noteSpillWriteFailure(error, exhaustedAclRetry ? "EACLRETRYEXHAUSTED" : undefined, aclRetryFailureOrigin); + requireStore().replaceWithSpillFailure(job.id, candidate); + requireStore().deferSupersededSpill(job.supersededSpill); + } + } finally { + const cancelled = job.cancelled; + releasePendingResponseSpill(job); + requireStore().recomputeOldestResident(); + if (!cancelled) { + requireStore().schedulePersist(); + requireStore().pruneResponses(); + enforceAppOwnedMemoryBudget(); + } + } +} + +export function queuePendingResponseSpill( + id: string, + candidate: ResidentResponseState, + options: { supersededSpill?: ResponseSpillRef; directAdmission?: boolean } = {}, +): void { + const inheritedSpill = cancelPendingResponseSpill(id) ?? options.supersededSpill; + if (pendingResponseSpillBytes + candidate.sizeBytes > MAX_PENDING_RESPONSE_SPILL_BYTES) { + noteSpillWriteFailure(null, "ECAPACITY"); + requireStore().replaceWithSpillFailure(id, candidate); + requireStore().deferSupersededSpill(inheritedSpill); + return; + } + // Enforce the disk cap BEFORE the temp or destination file is created. Deleting the + // overflow afterwards is not equivalent: on Windows the file can outlive the decision + // by as long as ACL hardening takes, which is the window the measured 6.8 GiB + // accumulated in. Reclaim first, and only refuse if the peak footprint still does not + // fit — an eviction pass can free a live continuation's worth of room. + const footprint = publicationFootprintBytes(id, candidate); + // The superseded generation this job is about to own is already off `states` and not + // yet on the job, so it is invisible to the walk. Price it here or admission decides + // against a total that is short by a whole envelope. + const inheritedBytes = inheritedSpill?.payloadBytes ?? 0; + if (requireStore().accountedResponseSpillBytes() + footprint + inheritedBytes > requireStore().spillByteCap()) { + requireStore().enforceSpilledResponseBudget(); + if (requireStore().accountedResponseSpillBytes() + footprint + inheritedBytes > requireStore().spillByteCap()) { + noteSpillWriteFailure(null, "ECAPACITY"); + requireStore().replaceWithSpillFailure(id, candidate); + requireStore().deferSupersededSpill(inheritedSpill); + return; + } + } + const job: PendingResponseSpill = { + id, + candidate, + ...(inheritedSpill ? { supersededSpill: inheritedSpill } : {}), + directAdmission: options.directAdmission === true, + running: false, + cancelled: false, + released: false, + sizeBytes: candidate.sizeBytes, + reservedBytes: footprint, + publicationControl: createResponseSpillPublicationControl(), + }; + pendingResponseSpills.add(job); + pendingResponseSpillById.set(id, job); + pendingResponseSpillBytes += job.sizeBytes; + reservedResponseSpillBytes += job.reservedBytes; + requireStore().recomputeOldestResident(); + responseSpillPublicationTail = responseSpillPublicationTail + .then(() => runPendingResponseSpill(job), () => runPendingResponseSpill(job)); +} + +export function replaceWithPendingResponseSpill( + id: string, + candidate: ResidentResponseState, + expected: StoredResponseState | undefined, + options: { directAdmission?: boolean } = {}, +): boolean { + const inheritedSpill = pendingResponseSpillById.get(id)?.supersededSpill + ?? (expected?.kind === "spill" ? expected.spill : undefined); + if (!requireStore().replaceMapEntry(id, candidate, expected)) return false; + queuePendingResponseSpill(id, candidate, { + ...(inheritedSpill ? { supersededSpill: inheritedSpill } : {}), + directAdmission: options.directAdmission === true, + }); + return true; +} + +/** Test-only: settle every serialized Windows spill publication. */ +export async function flushPendingResponseSpillsForTests(): Promise { + await drainResponseSpillPublications(); +} + +/** Test-only: observe ordinary queue settlement without invoking shutdown fallback. */ +export async function awaitResponseSpillPublicationTailForTests(): Promise { + await responseSpillPublicationTail; +} + +/** Test-only: observe the bounded queue without exposing payloads. */ +export function pendingResponseSpillMetricsForTests(): { count: number; bytes: number } { + return { count: pendingResponseSpills.size, bytes: pendingResponseSpillBytes }; +} + +/** Test-only: shorten the shutdown drain/fallback budget (null restores production values). */ +export function setResponseSpillShutdownBudgetForTests( + budget: { totalMs: number; fallbackReserveMs: number } | null, +): void { + responseSpillShutdownBudgetOverride = budget; +} + +/** Test-only: shorten the ordinary async whole-attempt ACL budget. */ +export function setResponseSpillAsyncAclAttemptBudgetForTests(budgetMs: number | null): void { + responseSpillAsyncAclAttemptBudgetOverride = budgetMs; +} + +function responseSpillAsyncAclAttemptBudgetMs(): number { + return responseSpillAsyncAclAttemptBudgetOverride ?? RESPONSE_SPILL_ASYNC_ACL_ATTEMPT_BUDGET_MS; +} + +/** Test-only: lower the hard terminalization pass guard (null restores production). */ +export function setResponseSpillShutdownTerminalizationPassLimitForTests(limit: number | null): void { + responseSpillShutdownTerminalizationPassLimitOverride = limit; +} + +function responseSpillShutdownTerminalizationPassLimit(): number { + return responseSpillShutdownTerminalizationPassLimitOverride + ?? requireStore().terminalizationMaxPasses(); +} + +function responseSpillShutdownBudget(): { totalMs: number; fallbackReserveMs: number } { + return responseSpillShutdownBudgetOverride ?? { + totalMs: RESPONSE_SPILL_SHUTDOWN_BUDGET_MS, + fallbackReserveMs: RESPONSE_SPILL_SHUTDOWN_FALLBACK_RESERVE_MS, + }; +} + +function awaitResponseSpillTailUntil(observed: Promise, deadline: number): Promise { + const remaining = deadline - Date.now(); + if (remaining <= 0) return Promise.resolve(false); + return new Promise(resolve => { + let finished = false; + const finish = (settled: boolean): void => { + if (finished) return; + finished = true; + clearTimeout(timer); + resolve(settled); + }; + const timer = setTimeout(() => finish(false), remaining); + observed.then(() => finish(true), () => finish(true)); + }); +} + +function installShutdownFallbackSpill( + job: PendingResponseSpill, + candidate: ResidentResponseState, + aclBudgetMs: number, +): void { + let ref: ResponseSpillRef | null = null; + // Supersession released this job's reservation, but the synchronous write below is the + // largest publication of the shutdown path and has its own link-then-copy fallback + // holding a temp and a destination at once. Re-reserve for its duration so the cap is + // not blind exactly where the drain does its heaviest work, and settle in `finally` so + // every return, throw and mismatch releases it. + const footprint = publicationFootprintBytes(job.id, candidate); + reservedResponseSpillBytes += footprint; + try { + // Supersession released this job, so its superseded generation is no longer visible + // to the accounting walk — but the file is still on the volume until + // `deferSupersededSpill` or a delete takes it. Price it here or the fallback decides + // against a total short by that whole envelope, which is exactly the gap that lets + // `debt + footprint <= cap < old + debt + footprint` publish over budget. + const supersededBytes = job.supersededSpill?.payloadBytes ?? 0; + // The drain must not publish over the cap either. Reclaim first; if the footprint + // still does not fit — which is what unreclaimable cleanup debt looks like — the + // honest close-out is a tombstone, not another file on a volume that is already + // over budget. `replaceWithSpillFailure` is the same fail-closed ending the budget + // exhaustion path uses, so replay reports `spill_failed` and the client resends. + if (requireStore().accountedResponseSpillBytes() + supersededBytes > requireStore().spillByteCap()) { + requireStore().enforceSpilledResponseBudget(); + if (requireStore().accountedResponseSpillBytes() + supersededBytes > requireStore().spillByteCap()) { + if (requireStore().currentEntry(job.id) === candidate) { + noteSpillWriteFailure(null, "ECAPACITY"); + requireStore().replaceWithSpillFailure(job.id, candidate); + requireStore().deferSupersededSpill(job.supersededSpill); + } + throw Object.assign(new Error("Response spill shutdown fallback exceeds the durable disk cap"), { code: "ENOSPC" }); + } + } + ref = writeResponseSpillDurably(job.id, spillPayloadForResident(candidate), { aclBudgetMs }); + if (ref.payloadBytes > responseSpillPayloadCap()) { + deleteResponseSpill(ref); + ref = null; + if (job.directAdmission) admissionCounters.oversizedDrops += 1; + throw Object.assign(new Error("Response spill payload exceeds replay ceiling"), { code: "EFBIG" }); + } + if (requireStore().currentEntry(job.id) !== candidate) { + deleteResponseSpill(ref); + ref = null; + return; + } + if (requireStore().swapResidentForSpill(job.id, candidate, ref)) { + ref = null; + noteSpillWriteSuccess(); + if (job.directAdmission) admissionCounters.directSpills += 1; + requireStore().deferSupersededSpill(job.supersededSpill); + } + } catch (error) { + if (ref) deleteResponseSpill(ref); + if (requireStore().currentEntry(job.id) === candidate) { + noteSpillWriteFailure(error); + requireStore().replaceWithSpillFailure(job.id, candidate); + requireStore().deferSupersededSpill(job.supersededSpill); + } + throw error; + } finally { + reservedResponseSpillBytes = Math.max(0, reservedResponseSpillBytes - footprint); + } +} + +function terminalizeShutdownFallbackCandidate( + job: PendingResponseSpill, + candidate: ResidentResponseState, + failureCode: ResponseSpillWriteFailureCode = "ETIMEDOUT", +): void { + if (requireStore().currentEntry(job.id) !== candidate) return; + noteSpillWriteFailure(null, failureCode); + requireStore().replaceWithSpillFailure(job.id, candidate); + requireStore().deferSupersededSpill(job.supersededSpill); +} + +function pendingShutdownFallbackCandidates(): Array<{ + job: PendingResponseSpill; + candidate: ResidentResponseState; +}> { + return [...pendingResponseSpills] + .map(job => ({ job, candidate: job.candidate })) + .filter((entry): entry is { job: PendingResponseSpill; candidate: ResidentResponseState } => !!entry.candidate); +} + +function supersedeShutdownFallbackBatch( + pending: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, + failures: Error[], +): void { + for (const { job } of pending) { + job.cancelled = true; + markResponseSpillPublicationSuperseded(job.publicationControl); + } + for (const { job } of pending) { + const cleanupFailure = cleanupSupersededResponseSpillPublication(job.publicationControl); + if (cleanupFailure) { + failures.push(cleanupFailure); + // Cleanup failed, so an async temp or destination is STILL on the volume. Releasing + // the reservation would un-account a file that exists, and the fallback write that + // follows reserves only its own footprint — three envelopes on disk priced as two. + // + // Charge the surviving PATHS rather than a flat two envelopes: `clearOwnedPath` + // nulls whichever it managed to remove, so one failure is one file, not two. The + // charge is settled automatically once the path disappears, which a retried unlink + // or a released Windows lock can still do. + const perPath = Math.max(1, Math.floor(job.reservedBytes / 2)); + chargeUnreclaimableSpillPath(job.publicationControl.tempPath, perPath); + chargeUnreclaimableSpillPath(job.publicationControl.destinationPath, perPath); + } + releasePendingResponseSpill(job); + } +} + +function stopAtShutdownTerminalizationPassLimit( + pending: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, + failures: Error[], +): void { + failures.push(Object.assign(new Error("Response spill shutdown terminalization pass limit exceeded"), { code: "ELOOP" })); + supersedeShutdownFallbackBatch(pending, failures); + for (const { job, candidate } of pending) { + terminalizeShutdownFallbackCandidate(job, candidate, "ELOOP"); + } + for (const [id, state] of requireStore().residentEntries()) { + if (state.kind !== "resident") continue; + noteSpillWriteFailure(null, "ELOOP"); + requireStore().replaceWithSpillFailure(id, state); + } + requireStore().recomputeOldestResident(); + requireStore().pruneResponses(); + enforceAppOwnedMemoryBudget(); +} + +function terminalizeExhaustedShutdownFallback( + initial: Array<{ job: PendingResponseSpill; candidate: ResidentResponseState }>, + failures: Error[], +): void { + let pending = initial; + let passes = 0; + const passLimit = responseSpillShutdownTerminalizationPassLimit(); + // Every pass replaces each captured resident with a tombstone. Pruning may expose + // another finite batch, but resident count strictly decreases until none can requeue. + while (pending.length > 0) { + if (passes >= passLimit) { + stopAtShutdownTerminalizationPassLimit(pending, failures); + return; + } + passes += 1; + supersedeShutdownFallbackBatch(pending, failures); + for (const { job, candidate } of pending) { + failures.push(Object.assign(new Error("Response spill shutdown fallback budget exhausted"), { code: "ETIMEDOUT" })); + terminalizeShutdownFallbackCandidate(job, candidate); + } + requireStore().recomputeOldestResident(); + requireStore().pruneResponses(); + enforceAppOwnedMemoryBudget(); + pending = pendingShutdownFallbackCandidates(); + } +} + +function fallbackPendingResponseSpills(reserveMs: number): Error[] { + const deadline = Date.now() + reserveMs; + const failures: Error[] = []; + for (;;) { + const pending = pendingShutdownFallbackCandidates(); + if (pending.length === 0) return failures; + if (Date.now() >= deadline) { + terminalizeExhaustedShutdownFallback(pending, failures); + return failures; + } + + supersedeShutdownFallbackBatch(pending, failures); + let reserveExhausted = false; + for (let index = 0; index < pending.length; index += 1) { + const { job, candidate } = pending[index]!; + if (requireStore().currentEntry(job.id) !== candidate) continue; + const remaining = deadline - Date.now(); + if (remaining <= 0) { + reserveExhausted = true; + for (const exhausted of pending.slice(index)) { + failures.push(Object.assign(new Error("Response spill shutdown fallback budget exhausted"), { code: "ETIMEDOUT" })); + terminalizeShutdownFallbackCandidate(exhausted.job, exhausted.candidate); + } + break; + } + try { + installShutdownFallbackSpill(job, candidate, remaining); + } catch (error) { + failures.push(error instanceof Error ? error : new Error("Response spill shutdown fallback failed")); + } + } + requireStore().recomputeOldestResident(); + requireStore().pruneResponses(); + enforceAppOwnedMemoryBudget(); + if (reserveExhausted || Date.now() >= deadline) { + terminalizeExhaustedShutdownFallback(pendingShutdownFallbackCandidates(), failures); + return failures; + } + } +} + +export async function drainResponseSpillPublications(): Promise { + const budget = responseSpillShutdownBudget(); + const fallbackReserveMs = Math.min(budget.totalMs, Math.max(1, budget.fallbackReserveMs)); + const drainDeadline = Date.now() + Math.max(0, budget.totalMs - fallbackReserveMs); + + for (;;) { + if (pendingResponseSpills.size === 0) return; + const observed = responseSpillPublicationTail; + const settled = await awaitResponseSpillTailUntil(observed, drainDeadline); + if (!settled) { + const failures = fallbackPendingResponseSpills(fallbackReserveMs); + if (failures.length > 0) { + throw new AggregateError(failures, "Response spill shutdown fallback incomplete"); + } + return; + } + if (observed === responseSpillPublicationTail) return; + } +} + +/** + * Byte accounting the facade's `accountedResponseSpillBytes` adds on top of the + * installed-spill walk: reserved publication footprint, files a pending job still + * owns through its superseded generation, and per-path cleanup debt that still + * exists on the volume. + */ +export function spillQueueAccounting(): { reservedBytes: number; jobOwnedBytes: number; unreclaimableBytes: number } { + let jobOwnedBytes = 0; + for (const job of pendingResponseSpills) { + if (job.supersededSpill) jobOwnedBytes += job.supersededSpill.payloadBytes; + } + return { + reservedBytes: reservedResponseSpillBytes, + jobOwnedBytes, + unreclaimableBytes: reconcileUnreclaimableSpillPaths(), + }; +} + +/** Resident candidates still owned by queued publications, for facade-side accounting. */ +export function spillQueueResidentCandidates(): Array<{ id: string; candidate: ResidentResponseState; sizeBytes: number }> { + const candidates: Array<{ id: string; candidate: ResidentResponseState; sizeBytes: number }> = []; + for (const job of pendingResponseSpills) { + if (job.candidate) candidates.push({ id: job.id, candidate: job.candidate, sizeBytes: job.sizeBytes }); + } + return candidates; +} + +/** Bytes pinned by queued jobs themselves (not their superseded generations). */ +export function spillQueuePendingBytes(): number { + return pendingResponseSpillBytes; +} + +/** True when this resident entry is the candidate a queued publication will install. */ +export function spillQueueHoldsResidentCandidate(id: string, state: ResidentResponseState): boolean { + return pendingResponseSpillById.get(id)?.candidate === state; +} + +/** Superseded generation a queued job will replace, if one is already parked on it. */ +export function spillQueueSupersededSpillFor(id: string): ResponseSpillRef | undefined { + return pendingResponseSpillById.get(id)?.supersededSpill; +} + +/** Test-only: release queued jobs and zero the queue-owned byte accounting. */ +export function resetSpillQueueForTests(): void { + for (const id of [...pendingResponseSpillById.keys()]) cancelPendingResponseSpill(id); + pendingResponseSpillById.clear(); + reservedResponseSpillBytes = 0; + unreclaimableSpillPaths.clear(); +} diff --git a/src/responses/state/temp-recovery.ts b/src/responses/state/temp-recovery.ts new file mode 100644 index 0000000000..5809d5a7e5 --- /dev/null +++ b/src/responses/state/temp-recovery.ts @@ -0,0 +1,257 @@ +import { opendirSync, lstatSync, unlinkSync } from "node:fs"; +import { uptime } from "node:os"; +import { dirname, join } from "node:path"; +import { getConfigDir, resolveWriteTarget } from "../../config"; + +const STALE_TEMP_GRACE_MS = 15 * 60 * 1_000; +const STALE_TEMP_MAX_ENTRIES = 4_096; +const STALE_TEMP_MAX_CLEANUPS = 512; +/** Absorbs `os.uptime()` granularity only. It is deliberately NOT the safety margin: + * the unconditional 15-minute grace above is (see the boot floor in the scan loop). */ +const BOOT_FLOOR_SKEW_MS = 60 * 1_000; +/** Per-tick budget for the periodic reclaim. Smaller than the startup budget because the + * periodic pass runs synchronously on the serving process's event loop every 60 s. */ +const PERIODIC_TEMP_MAX_ENTRIES = 512; +const PERIODIC_TEMP_MAX_CLEANUPS = 64; +/** Wall-clock ceiling for one periodic scan. An entry cap bounds syscalls, not time: on a + * network-mounted config dir each `lstat` can cost 10-20 ms, which would stall in-flight + * streams. Reclaim is idempotent, so a truncated tick simply resumes on the next one. */ +const PERIODIC_TEMP_SCAN_DEADLINE_MS = 25; +const RESPONSE_STATE_TEMP_NAME = /^responses-state\.json\.ocx\.(\d+)\.(\d+)\.tmp$/; + +export interface ResponseStateTempRecoveryResult { + matched: number; + removed: number; + failed: number; + bytesRemoved: number; + /** Entries that passed EVERY gate and would be reclaimed. In a dry run nothing is + * unlinked, so this is the only honest count to show an operator: `matched` is + * incremented before the file-type, age, boot-floor, and liveness gates. */ + eligible: number; + /** Total size of the `eligible` entries. */ + eligibleBytes: number; + /** The scan stopped on a budget (entry cap, cleanup cap, or deadline) rather than reaching + * the end of the directory, so the counts below describe a prefix of the backlog and not + * the backlog. `eligible > removed + failed` cannot express this: outside a dry run every + * eligible entry is unlinked or failed on the same iteration, so the two are always equal + * and a comparison between them is dead code. */ + truncated: boolean; +} + +interface ResponseStateTempRecoveryIO { + now: () => number; + /** Approximate epoch ms of the current boot; see the boot floor in the scan loop. */ + bootTime: () => number; + list: (dir: string) => Iterable; + inspect: (path: string) => { isFile: boolean; mtimeMs: number; size: number }; + isProcessAlive: (pid: number) => boolean; + unlink: (path: string) => void; +} + +export type ResponseStateTempRecoveryOptions = Partial & { + maxEntries?: number; + maxCleanups?: number; + /** Wall-clock ceiling for the scan, or null/undefined for no deadline (startup path). */ + deadlineMs?: number | null; + /** Report only: apply every gate, count what would be reclaimed, unlink nothing. */ + dryRun?: boolean; +}; + +function processIsAlive(pid: number): boolean { + if (pid === process.pid) return true; + try { + process.kill(pid, 0); + return true; + } catch (error) { + // EPERM means the process exists but cannot be signalled. Unknown platform errors + // are also protected; cleanup should prefer a false negative over touching a live writer. + return (error as NodeJS.ErrnoException).code !== "ESRCH"; + } +} + +const responseStateTempRecoveryIO: ResponseStateTempRecoveryIO = { + now: Date.now, + bootTime: () => Date.now() - uptime() * 1_000, + list: function* list(dir) { + const handle = opendirSync(dir); + try { + for (let entry = handle.readSync(); entry; entry = handle.readSync()) yield entry.name; + } finally { + handle.closeSync(); + } + }, + inspect: path => { + const stat = lstatSync(path); + return { isFile: stat.isFile() && !stat.isSymbolicLink(), mtimeMs: stat.mtimeMs, size: stat.size }; + }, + isProcessAlive: processIsAlive, + unlink: unlinkSync, +}; + +/** + * Recover only abandoned response-state atomic-write files. The exact basename, + * regular-file check, age gate, and PID liveness check protect unrelated/active files. + * Cleanup is capped and best-effort because continuation state is only a cache. Removal + * deliberately uses unlink only: path-based truncation could follow a replacement symlink. + */ +export function recoverStaleResponseStateTemps( + dir = getConfigDir(), + options: ResponseStateTempRecoveryOptions = {}, +): ResponseStateTempRecoveryResult { + const { + maxEntries = STALE_TEMP_MAX_ENTRIES, + maxCleanups = STALE_TEMP_MAX_CLEANUPS, + deadlineMs = null, + dryRun = false, + ...overrides + } = options; + const io = { ...responseStateTempRecoveryIO, ...overrides }; + const result: ResponseStateTempRecoveryResult = { + matched: 0, + removed: 0, + failed: 0, + bytesRemoved: 0, + eligible: 0, + eligibleBytes: 0, + truncated: false, + }; + const startedAt = io.now(); + // One probe per scan, not one per entry. A non-finite or future-dated boot is anomalous, and + // clamping it to "now" would be the WORST response: the floor would then retire the liveness + // probe for every file older than the skew, which is every file past the grace. Disable it + // instead -- an absent floor only costs a missed reclaim, never a wrong one. + const rawBoot = io.bootTime(); + const bootMs = Number.isFinite(rawBoot) && rawBoot <= startedAt ? rawBoot : Number.NEGATIVE_INFINITY; + let names: Iterable; + try { names = io.list(dir); } catch { return result; } + let iterator: Iterator; + try { iterator = names[Symbol.iterator](); } catch { return result; } + let scanned = 0; + // Every early exit runs through this. The production `list` is a generator that closes its + // directory handle in a `finally`, and a `finally` does NOT run when the consumer simply + // stops calling `next()` -- only `return()` resumes the generator to completion. Breaking + // out of the loop directly therefore leaked one directory handle per truncated scan, and the + // periodic reclaim truncates on purpose (entry cap, cleanup cap, deadline), so on a slow + // filesystem that is a leak per tick, forever. + const stopScan = (): ResponseStateTempRecoveryResult => { + try { iterator.return?.(); } catch { /* closing is best-effort; never fail a reclaim on it */ } + return result; + }; + for (;;) { + let next: IteratorResult; + try { next = iterator.next(); } catch { return result; } + if (next.done) break; + const name = next.value; + scanned += 1; + // A dry run performs no cleanups, so bounding it by the cleanup budget would truncate + // the very report an operator uses to size the problem. + if (scanned > maxEntries) { result.truncated = true; return stopScan(); } + if (!dryRun && result.removed + result.failed >= maxCleanups) { result.truncated = true; return stopScan(); } + if (deadlineMs !== null && io.now() - startedAt > deadlineMs) { result.truncated = true; return stopScan(); } + const match = RESPONSE_STATE_TEMP_NAME.exec(name); + if (!match) continue; + result.matched += 1; + const pid = Number(match[1]); + const sequence = Number(match[2]); + if (!Number.isSafeInteger(pid) || pid <= 0 || !Number.isSafeInteger(sequence) || sequence <= 0) continue; + const path = join(dir, name); + let file: ReturnType; + try { file = io.inspect(path); } catch { continue; } + if (!file.isFile || io.now() - file.mtimeMs < STALE_TEMP_GRACE_MS) continue; + // Boot floor. After a reboot the original writer's pid is routinely reused, which makes + // the liveness skip PERMANENT: the 15-minute grace above is a lower bound and never + // expires it, so the file is skipped on every future pass forever. A temp older than + // this boot cannot be owned by the pid we would probe, so the probe is vacuous and we + // retire it. This does NOT claim the file is provably dead: under a shared-volume + // container, suspend-excluding uptime, or a network config dir the computed boot can + // land after the real one. The unconditional 15-minute grace above remains the safety + // floor, and this process's own temps are never touched. + const predatesBoot = file.mtimeMs < bootMs - BOOT_FLOOR_SKEW_MS; + if (pid === process.pid) continue; + if (!predatesBoot && io.isProcessAlive(pid)) continue; + + result.eligible += 1; + result.eligibleBytes += file.size; + if (dryRun) continue; + + try { + io.unlink(path); + result.removed += 1; + result.bytesRemoved += file.size; + } catch (error) { + // Another proxy sharing this config dir may have won the race. A file that is already + // gone is reclaimed, not a failure -- reporting it as one would surface "in use or + // locked" to an operator for a file nobody holds. + if ((error as NodeJS.ErrnoException)?.code === "ENOENT") { + result.removed += 1; + continue; + } + // Locked files remain for a later startup. Do not truncate by path: a same-user + // replacement could turn that fallback into an arbitrary symlink-target write. + result.failed += 1; + } + } + return result; +} + +/** + * Literal config dir plus the snapshot's resolved dir. Atomic writes place their temp beside + * the RESOLVED target, so a symlinked snapshot (dotfiles-managed config dir) strands temps in + * the link's real directory where a scan of the literal dir would never see them. The two + * collapse to one when nothing is symlinked. + */ +function responseStateSweepDirectories(): Set { + const path = join(getConfigDir(), "responses-state.json"); + let resolvedDir = dirname(path); + try { + resolvedDir = dirname(resolveWriteTarget(path)); + } catch { + /* unresolvable link: sweep the literal dir only */ + } + return new Set([dirname(path), resolvedDir]); +} + +export function reclaimAbandonedResponseStateTemps( + options: ResponseStateTempRecoveryOptions = {}, +): ResponseStateTempRecoveryResult { + const total: ResponseStateTempRecoveryResult = { + matched: 0, removed: 0, failed: 0, bytesRemoved: 0, eligible: 0, eligibleBytes: 0, truncated: false, + }; + // The try encloses responseStateSweepDirectories() deliberately: recoverStaleResponseStateTemps + // already swallows its own enumeration failures, so a catch around only that call would be + // unreachable. snapshotPath()/getConfigDir() are the paths that can genuinely throw. + try { + for (const dir of responseStateSweepDirectories()) { + const result = recoverStaleResponseStateTemps(dir, options); + total.matched += result.matched; + total.removed += result.removed; + total.failed += result.failed; + total.bytesRemoved += result.bytesRemoved; + total.eligible += result.eligible; + total.eligibleBytes += result.eligibleBytes; + // Truncation anywhere makes the whole total a prefix. + total.truncated ||= result.truncated; + } + } catch { + /* best-effort: disk reclaim must never destabilize the caller */ + } + return total; +} + +/** + * Report-only counterpart for `ocx doctor`: applies every selection gate and unlinks + * nothing. It runs the SAME predicate as the reclaim, so the report and the subsequent + * removal cannot disagree about which files are reclaimable. + */ +export function inspectAbandonedResponseStateTemps(): ResponseStateTempRecoveryResult { + return reclaimAbandonedResponseStateTemps({ dryRun: true }); +} + +/** Sweeper adapter: narrows the reclaim to the `() => number` the liveness tick expects. */ +export function sweepAbandonedResponseStateTemps(): number { + return reclaimAbandonedResponseStateTemps({ + maxEntries: PERIODIC_TEMP_MAX_ENTRIES, + maxCleanups: PERIODIC_TEMP_MAX_CLEANUPS, + deadlineMs: PERIODIC_TEMP_SCAN_DEADLINE_MS, + }).removed; +} diff --git a/structure/catalog.md b/structure/catalog.md index fc7e65a9d5..e3976b185d 100644 --- a/structure/catalog.md +++ b/structure/catalog.md @@ -378,7 +378,7 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c ## Provider-scoped approval reviewer -`src/codex/catalog/sync.ts` resolves exact case-preserving provider/model reviewer selectors against the final catalog in both retained sync and `src/codex/convergence.ts`. Valid per-model selection wins over valid provider-wide selection, then the root selector supplies fallback. Native root stamps retain the observed original value and applied selector bound to their slug; removal restores the original only while the applied value is unchanged. The native provenance remains after restoration so an equal provider reviewer cannot trigger legacy reclassification on the next sync. Ambiguous legacy unmarked catalogs retain their existing heuristic cleanup. Provider stamps do not change routing or credentials. +`src/codex/catalog/auto-review.ts` resolves exact case-preserving provider/model reviewer selectors against the final catalog in both retained sync and `src/codex/convergence.ts`. Valid per-model selection wins over valid provider-wide selection, then the root selector supplies fallback. Native root stamps retain the observed original value and applied selector bound to their slug; removal restores the original only while the applied value is unchanged. The native provenance remains after restoration so an equal provider reviewer cannot trigger legacy reclassification on the next sync. Ambiguous legacy unmarked catalogs retain their existing heuristic cleanup. Provider stamps do not change routing or credentials. The [explicit model-capability contract](config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior. diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index 38a8221eb2..44faba2956 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -529,7 +529,7 @@ untouched. ## Z.ai quota destination ownership -`src/providers/quota.ts` uses one exact normalized-base mapping for both Z.ai quota +`src/providers/quota/vendor-probes-key.ts` uses one exact normalized-base mapping for both Z.ai quota eligibility and monitor selection. International root, coding Chat, Anthropic and Responses bases use `api.z.ai` with Bearer authentication. Existing BigModel CN root, coding Chat and Responses bases use `open.bigmodel.cn` with the raw key. Unsupported diff --git a/structure/providers/openai-tiers.md b/structure/providers/openai-tiers.md index 3407c03888..691a2fe7bf 100644 --- a/structure/providers/openai-tiers.md +++ b/structure/providers/openai-tiers.md @@ -468,7 +468,7 @@ Listener startup diagnostics follow [the runtime lifecycle contract](../runtime. ## Automatic pool plan exclusions -`src/codex/routing.ts` applies optional `codexPool.excludedPlans` to both candidate selection and existing active/affined accounts. An all-excluded pool returns no automatic candidate, including preview and configured-account fallback. Native main remains exempt and unknown plans remain eligible. Explicit account-qualified routes retain pause, credential and entitlement checks while bypassing only this automatic policy. +`src/codex/routing/selection.ts` applies optional `codexPool.excludedPlans` to both candidate selection and existing active/affined accounts. An all-excluded pool returns no automatic candidate, including preview and configured-account fallback. Native main remains exempt and unknown plans remain eligible. Explicit account-qualified routes retain pause, credential and entitlement checks while bypassing only this automatic policy. `src/codex/auth-api.ts` projects `selectionExcludedReason: "plan_excluded"` and `selectionExcludedPlan` from the routing config, even when a newer display-only WHAM plan could not be persisted. The dashboard and account CLI show the policy reason separately from credential health; renewal clears the derived fields. The automatic next-session action and badge are omitted for excluded rows. ## Paginated history writer boundary @@ -535,7 +535,7 @@ The history read API reports a median effective token estimate and interval samp ## Reset-first account ordering -`src/codex/routing.ts` supports Codex-only `accountPoolStrategy: "reset-first"`. For new shared-quota assignments it chooses the earliest future short/weekly reset after existing eligibility, priority and usage-threshold filtering; ties and absent/elapsed deadlines use the existing usage order. Seconds and milliseconds are normalized with `resetAtToMs`. Threshold zero disables usage filtering while retaining reset ordering. Monthly deadlines do not order this strategy. +`src/codex/routing/selection.ts` supports Codex-only `accountPoolStrategy: "reset-first"`. For new shared-quota assignments it chooses the earliest future short/weekly reset after existing eligibility, priority and usage-threshold filtering; ties and absent/elapsed deadlines use the existing usage order. Seconds and milliseconds are normalized with `resetAtToMs`. Threshold zero disables usage filtering while retaining reset ordering. Monthly deadlines do not order this strategy. Live bindings obey the cache-affinity release policy: `pool.cacheAffinity` is on by default, so threshold crossing alone retains a healthy account. A bound thread that does leave may move only onto an account with genuine quota headroom and strictly lower usage. Manual preference, scoped health and shared-cursor guards remain authoritative. Set the flag false to restore threshold rebinding of bound tasks. Independent `spark`/`reserve` quota scopes resolve reset-first to existing quota selection because shared reset timestamps do not describe those windows. The configured value stays unchanged. diff --git a/structure/runtime.md b/structure/runtime.md index e4a50ba3f8..49a2fc3f2f 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -339,7 +339,7 @@ Automatic Codex pool selection and account status share the [plan exclusion cont `src/web-search/loop.ts` makes at most one extra answer attempt after a clean forced-answer terminal with no visible output or tool call. The recovery has no tools and reuses gathered search results. Malformed calls fail before refusal/truncation passthrough, and well-formed recognized refusal/truncation terminals pass through unchanged, including empty or partial answers. The extra generation may incur provider usage. ## Scoped provider quota for Combo selection -`src/providers/quota.ts` publishes routing evidence only when a producer explicitly supplies its +`src/providers/quota/report-cache.ts` publishes routing evidence only when a producer explicitly supplies its inference-wide projection. A matching credential alone does not grant veto authority. Display-only account, model-group, search and legacy MCP windows remain visible but cannot exclude a provider. The private WeakMap binds provider name, adapter, destination and captured credential; neither diff --git a/structure/subagents.md b/structure/subagents.md index 69047b076b..b88dd48369 100644 --- a/structure/subagents.md +++ b/structure/subagents.md @@ -68,7 +68,7 @@ v2. An explicit attempt to enable the global flag while the hybrid pin is active ### What the five-model `spawn_agent` window is, and how V1 differs from V2 -`MAX_SPAWN_AGENT_MODEL_OVERRIDES = 5` (mirrored in `src/codex/catalog/sync.ts`) is **not** a +`MAX_SPAWN_AGENT_MODEL_OVERRIDES = 5` (mirrored in `src/codex/catalog/subagent-roster.ts`) is **not** a subagent concurrency limit and **not** an eligibility limit. Upstream uses it in exactly two places: the model list rendered into the `spawn_agent` tool description (`multi_agents_spec.rs:789`) and the "Available models:" suggestions in an unknown-model error diff --git a/tests/ci-workflows/file-size-ratchet.test.ts b/tests/ci-workflows/file-size-ratchet.test.ts new file mode 100644 index 0000000000..4b6cbdb611 --- /dev/null +++ b/tests/ci-workflows/file-size-ratchet.test.ts @@ -0,0 +1,212 @@ +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; + +/** + * Cycle 1 of 260914_godfile_round2. No file is split here. The gate is a bun + * test in the existing suite, not a new ci.yml job, because PR checkouts are a + * single refs/pull/N/merge commit at fetch-depth 1 and cannot see origin/dev. + * + * The scanner exports evaluate() so this file can feed it synthetic FileSize + * rows. Importing the module must not scan the repository: privacy-scan.ts runs + * on import and that pattern is forbidden here. + * + * Source-oracle reads go through tests/helpers/repo-root.ts (INV-TESTS-01). + */ +import { + GENERATED_PATHS, + THRESHOLD, + countLines, + evaluate, + isOffender, + isScannedPath, + loadBaseline, + scanRepo, + updateBaseline, + type Baseline, + type FileSize, +} from "../../scripts/file-size-ratchet"; +import { repoPath, repoRoot } from "../helpers/repo-root"; + +/** + * The ratchet must fail for the reason it claims. A single "repo is currently + * green" test would stay green if evaluate() started returning NEW_OK for a + * 2,000-line new file, as long as this tree had no such file today. + * + * Five pure cases plus one repository scan. Do not add a seventh test(): + * SHRANK already covers updateBaseline (lower, drop missing, never raise, + * seed only when asked). + */ +const emptyBaseline = (): Baseline => ({ generated: [], files: {} }); + +const linesOf = (count: number): string => { + const rows = Array.from({ length: count }, (_, i) => `line ${i}`); + return `${rows.join("\n")}\n`; +}; + +describe("file-size ratchet: countLines", () => { + test("NEW_OVERSIZED: baseline에 없고 2000줄 이상이면 실패", () => { + // The formula is the contract: split on \n, then drop the phantom cell that a + // trailing newline creates. wc -l disagrees on files that do not end in a newline, + // so the helper is asserted here instead of trusted from the scanner comments. + expect(countLines(linesOf(THRESHOLD))).toBe(THRESHOLD); + expect(countLines(linesOf(THRESHOLD - 1))).toBe(THRESHOLD - 1); + expect(countLines("")).toBe(1); + expect(countLines("a\nb")).toBe(2); + expect(countLines("a\nb\n")).toBe(2); + + const oversized: FileSize[] = [{ path: "src/new-god.ts", lines: THRESHOLD }]; + const under: FileSize[] = [{ path: "src/new-small.ts", lines: THRESHOLD - 1 }]; + const baseline = emptyBaseline(); + + expect(evaluate(oversized, baseline)).toEqual([ + { path: "src/new-god.ts", lines: THRESHOLD, verdict: "NEW_OVERSIZED" }, + ]); + expect(evaluate(under, baseline)).toEqual([ + { path: "src/new-small.ts", lines: THRESHOLD - 1, verdict: "NEW_OK" }, + ]); + expect(evaluate(oversized, baseline).filter(isOffender)).toHaveLength(1); + expect(evaluate(under, baseline).filter(isOffender)).toEqual([]); + }); +}); + +describe("file-size ratchet: caps", () => { + test("GREW: baseline 캡보다 길어지면 실패", () => { + // Grandfathered files may stay oversized, but they may not grow. Equality is + // UNCHANGED, not SHRANK; a test that only checked isOffender() would not notice + // if equality started reporting GREW. + const baseline: Baseline = { generated: [], files: { "src/config.ts": 4707 } }; + const grew = evaluate([{ path: "src/config.ts", lines: 4708 }], baseline); + const same = evaluate([{ path: "src/config.ts", lines: 4707 }], baseline); + + expect(grew).toEqual([{ path: "src/config.ts", lines: 4708, verdict: "GREW" }]); + expect(same).toEqual([{ path: "src/config.ts", lines: 4707, verdict: "UNCHANGED" }]); + expect(grew.filter(isOffender)).toHaveLength(1); + expect(same.filter(isOffender)).toEqual([]); + }); + + test("SHRANK: 줄면 통과하고 --update는 캡을 내리기만 한다", () => { + // --update is operator tooling, not a seventh test(). The seed path is the only + // way a 2,000+ file enters `files`; after that, a later --update without seed + // must not re-grandfather a new godfile, must not raise a cap, and must keep a + // shrunken former godfile so the facade cannot grow back. + const baseline: Baseline = { + generated: [], + files: { "src/keep.ts": 2100, "src/gone.ts": 2500, "src/small.ts": 800 }, + }; + const current: FileSize[] = [ + { path: "src/keep.ts", lines: 2099 }, + { path: "src/small.ts", lines: 800 }, + { path: "src/new-ok.ts", lines: 1200 }, + ]; + + expect(evaluate(current, baseline)).toEqual([ + { path: "src/keep.ts", lines: 2099, verdict: "SHRANK" }, + { path: "src/small.ts", lines: 800, verdict: "UNCHANGED" }, + { path: "src/new-ok.ts", lines: 1200, verdict: "NEW_OK" }, + ]); + expect(evaluate(current, baseline).filter(isOffender)).toEqual([]); + + // seed=false: lower keep, drop gone, do not add new-ok (it is under 2000 and + // must remain free to grow until 1999). small.ts stays at 800 even though it + // is under the threshold — a former godfile must not grow back. + const lowered = updateBaseline(current, baseline, false); + expect(lowered.files).toEqual({ "src/keep.ts": 2099, "src/small.ts": 800 }); + expect(lowered.files["src/gone.ts"]).toBeUndefined(); + expect(lowered.files["src/new-ok.ts"]).toBeUndefined(); + + // A later --update must never raise. If it did, ratchet:update would launder GREW. + const notRaised = updateBaseline( + [{ path: "src/keep.ts", lines: 3000 }], + { generated: [], files: { "src/keep.ts": 2099 } }, + false, + ); + expect(notRaised.files["src/keep.ts"]).toBe(2099); + + // seed=true is the first-commit path only (baseline file missing). Exempt + // generated paths stay out of files even at 9000 lines. Under-threshold files + // stay out so the 2,000 cap remains the policy for new modules. + const seeded = updateBaseline( + [ + { path: "src/old.ts", lines: 2500 }, + { path: "src/fresh.ts", lines: 1800 }, + { path: "gui/src/i18n/en.ts", lines: 9000 }, + ], + { generated: ["gui/src/i18n/en.ts"], files: {} }, + true, + ); + expect(seeded.files).toEqual({ "src/old.ts": 2500 }); + }); + + test("GENERATED: baseline.generated 경로는 커져도 통과", () => { + // Exact paths only. A sibling under cursor/gen/ that is not in generated[] is a + // new oversized file, even though a glob would have exempted the whole directory. + const path = "src/adapters/cursor/gen/agent_pb.ts"; + const baseline: Baseline = { + generated: [path], + files: { [path]: 100 }, + }; + const rows = evaluate([{ path, lines: 99_999 }], baseline); + expect(rows).toEqual([{ path, lines: 99_999, verdict: "GENERATED" }]); + expect(rows.filter(isOffender)).toEqual([]); + + const globWouldHaveCaught = evaluate( + [{ path: "src/adapters/cursor/gen/hand-written.ts", lines: 2500 }], + { generated: [path], files: {} }, + ); + expect(globWouldHaveCaught[0]?.verdict).toBe("NEW_OVERSIZED"); + }); +}); + +describe("file-size ratchet: scan filter", () => { + test("스캔제외: 화이트리스트 밖·제외 접두·bun.lock은 evaluate에 안 들어온다", () => { + // evaluate() never sees excluded paths; the filter is isScannedPath(). devlog/, + // assets, docs-site public/assets, gui/dist, bun.lock, and non-whitelist + // extensions (.mdx, .png) stay out. src/generated/model-metadata.ts is scanned: + // it is not on the 12-path exemption list, and if it crosses 2,000 it must fail. + // Whitelist hits. .yml and .json are in the contract list; .mdx is not. + expect(isScannedPath("src/config.ts")).toBe(true); + expect(isScannedPath("gui/src/pages/Models.tsx")).toBe(true); + expect(isScannedPath(".github/workflows/ci.yml")).toBe(true); + expect(isScannedPath("scripts/foo.sh")).toBe(true); + expect(isScannedPath("package.json")).toBe(true); + expect(isScannedPath("README.md")).toBe(true); + expect(isScannedPath("gui/src/styles.css")).toBe(true); + expect(isScannedPath(".github/scripts/issue-quality.test.cjs")).toBe(true); + expect(isScannedPath("scripts/foo.mjs")).toBe(true); + + // Prefix and exact exclusions. gui/dist without a trailing slash is listed + // in the contract alongside gui/dist/ children. + expect(isScannedPath("devlog/_plan/260914_godfile_round2/010.md")).toBe(false); + expect(isScannedPath("assets/banner.png")).toBe(false); + expect(isScannedPath("docs-site/public/favicon.png")).toBe(false); + expect(isScannedPath("docs-site/src/assets/og.png")).toBe(false); + expect(isScannedPath("gui/dist/index.js")).toBe(false); + expect(isScannedPath("gui/dist")).toBe(false); + expect(isScannedPath("bun.lock")).toBe(false); + expect(isScannedPath("docs-site/src/content/docs/index.mdx")).toBe(false); + expect(isScannedPath("src/generated/model-metadata.ts")).toBe(true); + }); +}); + +describe("file-size ratchet: repository", () => { + test("저장소 스캔: 커밋된 기준선 대비 offender가 없다", () => { + // Mirrors tests/ci-workflows/repo-hygiene.test.ts: git ls-files + expect([]). + // An empty scan would also equal [], so scanned.length > 0 is the non-vacuous + // guard. generated[] is the committed JSON, not the script constant used alone. + const baseline = loadBaseline( + readFileSync(repoPath("tests/fixtures/file-size-baseline.json"), "utf8"), + ); + expect(baseline.generated).toEqual([...GENERATED_PATHS]); + + const scanned = scanRepo(repoRoot()); + expect(scanned.length).toBeGreaterThan(0); + expect(scanned.some((file) => file.path.startsWith("devlog/"))).toBe(false); + expect(scanned.some((file) => file.path === "bun.lock")).toBe(false); + + const rows = evaluate(scanned, baseline); + expect(rows.filter(isOffender)).toEqual([]); + expect( + rows.filter((row) => row.verdict === "GENERATED").map((row) => row.path).sort(), + ).toEqual([...GENERATED_PATHS].slice().sort()); + }); +}); diff --git a/tests/codex-integration/codex-history-reachability.test.ts b/tests/codex-integration/codex-history-reachability.test.ts index 553049da4d..156709aefb 100644 --- a/tests/codex-integration/codex-history-reachability.test.ts +++ b/tests/codex-integration/codex-history-reachability.test.ts @@ -34,7 +34,7 @@ const PERMITTED_ROOTS = new Set(["codex/history-worker.ts"]); */ const INLINE_ALLOWED = new Set([ "codex/history-provider.ts", - "codex/inject.ts", + "codex/inject/restore.ts", "codex/internal/history-writer.ts", ]); diff --git a/tests/codex-integration/codex-inject-history-wording.test.ts b/tests/codex-integration/codex-inject-history-wording.test.ts index 9a681f120f..1c7584e6cc 100644 --- a/tests/codex-integration/codex-inject-history-wording.test.ts +++ b/tests/codex-integration/codex-inject-history-wording.test.ts @@ -8,7 +8,8 @@ import { } from "../../src/codex/inject"; import { repoPath } from "../helpers/repo-root"; -const injectSource = readFileSync(repoPath("src/codex/inject.ts"), "utf8"); +const injectSource = readFileSync(repoPath("src/codex/inject.ts"), "utf8") + + readFileSync(repoPath("src/codex/inject/restore.ts"), "utf8"); const doctorSource = readFileSync(repoPath("src/cli/doctor.ts"), "utf8"); const cliSource = readFileSync(repoPath("src/cli/index.ts"), "utf8"); const integrationGuide = readFileSync( diff --git a/tests/codex-integration/codex-retained-root-serialization.test.ts b/tests/codex-integration/codex-retained-root-serialization.test.ts index f97877ed90..ee03e51f4d 100644 --- a/tests/codex-integration/codex-retained-root-serialization.test.ts +++ b/tests/codex-integration/codex-retained-root-serialization.test.ts @@ -320,7 +320,7 @@ test("native restore cannot read-transform-write the catalog while another proce `); expect(restored.exitCode).toBe(0); expect(readFileSync(catalogPath, "utf8")).toBe(before); - const source = readFileSync(join(repoRoot, "src/codex/inject.ts"), "utf8"); + const source = readFileSync(join(repoRoot, "src/codex/inject/restore.ts"), "utf8"); const restoreRoot = source.slice(source.indexOf("const owningCodexHome"), source.indexOf("// Design B", source.indexOf("const owningCodexHome"))); expect(restoreRoot).toContain("withCatalogWriteSerialization(owningCodexHome"); expect(restoreRoot).toContain("restoreCodexCatalogWithPermit"); diff --git a/tests/config/config-save-boundary.test.ts b/tests/config/config-save-boundary.test.ts index 36dac68303..5b51e8182c 100644 --- a/tests/config/config-save-boundary.test.ts +++ b/tests/config/config-save-boundary.test.ts @@ -20,6 +20,7 @@ const GUARDED_FILES = [ "providers/api-keys.ts", // request-path + management key pool "providers/key-failover.ts", // 429 rotation, reached mid-turn with no user action "codex/routing.ts", // account auto-switch during a turn + "codex/routing/active-account.ts", // setActiveCodexAccount moved here in the routing split "codex/auth-api.ts", // runtime account/quota persistence "cli/claude-desktop.ts", // CLI against a running service "server/management-api.ts", diff --git a/tests/fixtures/file-size-baseline.json b/tests/fixtures/file-size-baseline.json new file mode 100644 index 0000000000..dff0815d02 --- /dev/null +++ b/tests/fixtures/file-size-baseline.json @@ -0,0 +1,63 @@ +{ + "generated": [ + "scripts/model-metadata.source.json", + "src/adapters/cursor/gen/agent_pb.ts", + "gui/src/i18n/de.ts", + "gui/src/i18n/en.ts", + "gui/src/i18n/fr.ts", + "gui/src/i18n/ja.ts", + "gui/src/i18n/ko.ts", + "gui/src/i18n/ru.ts", + "gui/src/i18n/tr.ts", + "gui/src/i18n/zh.ts", + "gui/src/i18n/zh-TW.ts", + "docs-site/src/data/frontier-benchmarks.json" + ], + "files": { + ".github/scripts/issue-quality.test.cjs": 2143, + "gui/src/pages/Models.tsx": 2792, + "gui/src/styles.css": 2958, + "src/adapters/openai-chat.ts": 2234, + "src/adapters/openai-responses.ts": 2627, + "src/bridge.ts": 2206, + "src/codex/auth-api.ts": 3134, + "src/codex/catalog/provider-fetch.ts": 2944, + "src/config.ts": 4799, + "src/providers/registry.ts": 3744, + "src/server/index.ts": 3400, + "src/server/responses/core.ts": 9360, + "tests/ci-workflows/ci-workflows.test.ts": 5628, + "tests/cli/cli-account.test.ts": 2313, + "tests/codex-integration/codex-auth-api.test.ts": 6549, + "tests/codex-integration/codex-auth-context.test.ts": 2496, + "tests/codex-integration/codex-catalog.test.ts": 7985, + "tests/codex-integration/codex-reset-credit-recovery.test.ts": 2135, + "tests/codex-integration/codex-routing.test.ts": 3443, + "tests/codex-integration/codex-shim.test.ts": 2388, + "tests/codex-integration/codex-v2-gate.test.ts": 2069, + "tests/providers/cursor/cursor-blob.test.ts": 3657, + "tests/providers/kiro/kiro-adapter.test.ts": 2050, + "tests/providers/kiro/kiro-stream.test.ts": 2258, + "tests/providers/provider-quota.test.ts": 3763, + "tests/responses/chat-completions-endpoint.test.ts": 3646, + "tests/responses/openai-responses-passthrough.test.ts": 4809, + "tests/responses/responses-compaction-routing.test.ts": 2776, + "tests/responses/responses-custom-tool-repair.test.ts": 2143, + "tests/responses/responses-state.test.ts": 3983, + "tests/responses/responses-undeclared-tool-guard.test.ts": 2379, + "tests/responses/ws-upstream.test.ts": 2004, + "tests/routing/subagent-fallback-handle-responses.test.ts": 2289, + "tests/server/config.test.ts": 3828, + "tests/server/management-provider-validation.test.ts": 5506, + "tests/server/server-auth.test.ts": 4589, + "tests/server/server-combo-failover-e2e.test.ts": 4166, + "tests/server/server-images.test.ts": 2755, + "tests/server/server-live.test.ts": 2253, + "tests/service/service.test.ts": 4106, + "tests/storage/storage-cleanup.test.ts": 2303, + "tests/usage/request-log.test.ts": 2075, + "tests/usage/usage-summary.test.ts": 2069, + "tests/web-search/web-search.test.ts": 2823, + "tests/windows/windows-secret-acl.test.ts": 2310 + } +} diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 348c5859bc..b4f994c4aa 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -535,6 +535,7 @@ "fastwire-policy.test.ts": "routing", "featherless-provider.test.ts": "providers", "fetch-header-timeout.test.ts": "server", + "file-size-ratchet.test.ts": "ci-workflows", "fixture-dir-uniqueness.test.ts": "ci-workflows", "flash-route-image-modalities.test.ts": "providers", "format-result.test.ts": "web-search", diff --git a/tests/providers/provider-quota.test.ts b/tests/providers/provider-quota.test.ts index b388889a94..0c9c8b210c 100644 --- a/tests/providers/provider-quota.test.ts +++ b/tests/providers/provider-quota.test.ts @@ -119,8 +119,8 @@ afterEach(() => { describe("fetchProviderQuotaReports", () => { test("provider quota probes have no direct Response.json calls", () => { - const source = readFileSync(repoPath("src/providers/quota.ts"), "utf8"); - expect(source).not.toMatch(/\.\s*json\s*\(/); + // Probes live in leaves now; the facade alone no longer holds one. + for (const p of ["quota.ts", "quota/vendor-probes-key.ts", "quota/vendor-probes-oauth.ts", "quota/antigravity.ts"]) expect(readFileSync(repoPath(`src/providers/${p}`), "utf8")).not.toMatch(/\.\s*json\s*\(/); }); test("quota JSON reading cancels a body that stalls before its first byte", async () => { diff --git a/tests/usage/quota-reset-detector.test.ts b/tests/usage/quota-reset-detector.test.ts index 3ce61d7b7b..f369db1a26 100644 --- a/tests/usage/quota-reset-detector.test.ts +++ b/tests/usage/quota-reset-detector.test.ts @@ -117,7 +117,7 @@ describe("quota reset detection", () => { }); test("sentinel reset clocks are ignored rather than read as 1970", () => { - // src/providers/quota.ts:279 and src/codex/quota.ts:192 disagree on whether 0 survives, + // src/providers/quota/account-cache.ts and src/codex/quota.ts disagree on whether 0 survives, // so the detector re-checks: a 0 deadline must not read as a long-passed one. expect(detect({ percent: 90, resetAt: 0 }, { percent: 88, resetAt: 0 })).toBeNull(); });