Skip to content

fix(opencodex): price usage by its recorded provider - #3676

Merged
steipete merged 1 commit into
steipete:mainfrom
Chipagosfinest:fix/provider-aware-pricing
Sep 16, 2026
Merged

steipete merged 1 commit into
steipete:mainfrom
Chipagosfinest:fix/provider-aware-pricing

Conversation

@Chipagosfinest

@Chipagosfinest Chipagosfinest commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

OpenCodex records its billing provider, but the previous pricing path used only the model name. An OpenRouter model such as openai/gpt-5.4 could be attributed to a Codex subscription or priced using the wrong provider. Missing token counts and an unpriced current day could also appear as zero cost.

Use the recorded provider for attribution and exact catalog prices, preserve custom-price precedence, and keep unknown costs distinct from genuinely free usage. Fresh opt-in loads refresh the existing public pricing cache. This changes estimates for existing logs; it adds no collection source or subscription route.

The maintainer revision also preserves independent cache counters, including cache-only usage. Caller-supplied overrides keep independent token-class arithmetic; app-level overrides and OpenAI catalog pricing retain their historical input conventions. New provider catalog prices count each recorded class once, and arithmetic overflow remains unpriced.

Validation: 281 focused tests across 12 suites passed, covering provider routing, catalog prices, custom overrides, cache-only/free/missing-rate cases, overflow, disk import, fan-out, refresh, store compatibility, and architecture gates. make check passed with zero violations after correcting test formatting. Independent code review found no actionable P0–P2 findings. Exact-head CI must finish successfully before merge.

Contributor: @Chipagosfinest. The batch changelog will be rewritten after landing with this fix and contributor credit.

A production CLI check also passed in a disposable macOS 15.7.7 VM with synthetic logs and a seeded pricing catalog, network access denied, and Keychain access disabled. Six real imported SQLite rows produced 346 tokens, four priced requests, two unpriced requests, and $0.000768 in known list-price estimates. A cache-only request priced at $0.00015; an exact free rate stayed zero; missing cache-write rates left today's cost omitted. Historical OpenAI pricing remained $0.000244. Guest fixtures and synthetic defaults were removed after verification. This proves local import/storage/pricing, without claiming live provider billing or network catalog refresh.

@clawsweeper

clawsweeper Bot commented Sep 16, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T06:28:28.400121Z 5fdd00d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 16, 2026
@clawsweeper

clawsweeper Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 16, 2026, 6:18 AM ET / 10:18 UTC (Revision 8).

ClawSweeper review

What this changes

Prices imported OpenCodex usage by its recorded provider, refreshes cached prices before fresh results, and preserves custom overrides and unknown costs.

Merge readiness

Ready for maintainer review

The fix remains necessary: current main and v0.60.3 retain model-only pricing for OpenCodex logs. The current head resolves the prior override-precedence findings, and no blocking defect was found.

Priority: P2
Reviewed head: 67f1d5d005ec65cc5ee92fc87e4a687aca6aa883

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused, well-covered correction with sufficient production CLI evidence and no remaining actionable findings.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The supplied macOS production CLI results exercise real log import, SQLite storage, and the changed pricing aggregator, demonstrating provider prices, cache-only charges, free rates, unknown costs, and historical compatibility; network catalog refresh is supported separately by focused tests.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The supplied macOS production CLI results exercise real log import, SQLite storage, and the changed pricing aggregator, demonstrating provider prices, cache-only charges, free rates, unknown costs, and historical compatibility; network catalog refresh is supported separately by focused tests.
Evidence reviewed 8 items Policy and patch scope: Read the full root AGENTS.md and checked Sources, Tests, docs, and .agents for applicable nested guidance; none was found. Reviewed all 21 introduced files against the pinned merge base. Provider isolation, focused validation, and Keychain safety informed the review; no builds or tests were executed in this read-only checkout.
Current-main necessity: Current main still passes entry.model to codexCostUSD without selecting prices using entry.provider, and substitutes zero for missing input/output counts. The branch supplies a distinct correction to this owned pricing path.
Latest-release comparison: The v0.60.3 source also retains the old model-only calculation, so the requested correction is not already present in the supplied latest release.
Findings None None.
Security None None.

How this fits together

CodexBar imports OpenCodex usage logs into a local SQLite cache and calculates spending estimates using custom prices and the public models.dev catalog. These estimates feed CLI JSON and supported subscription rows in the spend dashboard.

flowchart TD
  A[OpenCodex usage logs] --> B[Local usage cache]
  B --> C[Recorded provider and model]
  D[Custom prices] --> E[Price token classes]
  F[Cached public catalog] --> E
  C --> E
  E --> G[Known or unknown cost]
  G --> H[CLI and spend dashboard]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +289/-44 lines; tests +743/-25 lines Production growth supports shared provider resolution and refresh integration, with focused pricing and compatibility coverage.

Technical review

Best possible solution:

Retain provider-scoped estimates with existing override precedence, raw usage preservation, and explicit unknown costs.

Do we have a high-confidence way to reproduce the issue?

Yes: current main prices an OpenRouter record such as openai/gpt-5.4 through the model-only Codex calculation. This is source-established; the reviewer did not execute a reproduction.

Is this the best way to solve the issue?

Yes: the patch reuses existing catalog and override infrastructure while preserving legacy routes and stored usage, with focused compatibility coverage.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against f13aaf15d748.

Labels

Label justifications:

  • P2: Corrects spending estimates and subscription attribution for optional imported OpenCodex logs.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The supplied macOS production CLI results exercise real log import, SQLite storage, and the changed pricing aggregator, demonstrating provider prices, cache-only charges, free rates, unknown costs, and historical compatibility; network catalog refresh is supported separately by focused tests.
  • proof: sufficient: Contributor real behavior proof is sufficient. The supplied macOS production CLI results exercise real log import, SQLite storage, and the changed pricing aggregator, demonstrating provider prices, cache-only charges, free rates, unknown costs, and historical compatibility; network catalog refresh is supported separately by focused tests.

Evidence

What I checked:

  • Policy and patch scope: Read the full root AGENTS.md and checked Sources, Tests, docs, and .agents for applicable nested guidance; none was found. Reviewed all 21 introduced files against the pinned merge base. Provider isolation, focused validation, and Keychain safety informed the review; no builds or tests were executed in this read-only checkout. (AGENTS.md:1, 67f1d5d005ec)
  • Current-main necessity: Current main still passes entry.model to codexCostUSD without selecting prices using entry.provider, and substitutes zero for missing input/output counts. The branch supplies a distinct correction to this owned pricing path. (Sources/CodexBarCore/Vendored/OpenCodexUsage/OpenCodexUsageAggregator.swift:305, f13aaf15d748)
  • Latest-release comparison: The v0.60.3 source also retains the old model-only calculation, so the requested correction is not already present in the supplied latest release. (Sources/CodexBarCore/Vendored/OpenCodexUsage/OpenCodexUsageAggregator.swift:305, 9db44805dc1e)
  • Prior findings resolved: Recorded application-overlay keys are checked before resolving legacy routes, and the existing rates helper preserves bare-key precedence. Regression cases cover original aliases, recorded-provider overrides, partial/free overrides, and competing bare/provider-qualified keys. The previous completed review used this same head and reported no findings. (Tests/CodexBarTests/OpenCodexProviderPricingTests.swift:176, 67f1d5d005ec)
  • Upgrade compatibility: The native store accepts the preceding parser hash through its existing compatibility mechanism. The parameterized regression preserves stored snapshots and checkpoints without rebuilding; OpenCodex's separate SQLite schema remains unchanged and estimates are recomputed from raw entries. (Tests/CodexBarTests/CostUsageStoreTests.swift:1026, 67f1d5d005ec)
  • Production behavior evidence: The complete supplied PR body, captured under sourceRevision eb87bc7a0bde3f700aa0d93a243eeeccca3280924956fff891f0f770785941cd, reports a production CLI run in a disposable macOS 15.7.7 VM. Six imported SQLite rows produced 346 tokens, four priced and two unpriced requests, and $0.000768 in known estimates; cache-only usage yielded $0.00015, free usage stayed zero, missing cache-write prices omitted today's cost, and historical OpenAI pricing remained $0.000244. This exercises import/storage/aggregation through the changed CLI path. Network refresh was expressly outside that run's coverage. The scratch directory contained no additional media artifacts. (Sources/CodexBarCLI/CLICostCommand.swift:653, 67f1d5d005ec)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Olddonkey: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (7 earlier review cycles)
  • reviewed 2026-09-16T06:26:17.426Z sha 5fdd00d :: needs real behavior proof before merge. :: [P1] Preserve original application-overlay keys before normalizing aliases
  • reviewed 2026-09-16T06:43:30.071Z sha bdc8790 :: needs real behavior proof before merge. :: [P1] Preserve recorded-provider overlay keys before resolving legacy routes
  • reviewed 2026-09-16T06:54:03.009Z sha bdc8790 :: blocked before merge. :: [P1] Preserve recorded-provider overlay keys before resolving legacy routes
  • reviewed 2026-09-16T08:02:18.925Z sha 1efac91 :: blocked before merge. :: [P1] Preserve bare-key precedence when checking recorded overrides
  • reviewed 2026-09-16T08:13:48.062Z sha 62090a0 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-16T09:08:07.258Z sha 62090a0 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-16T09:59:26.204Z sha 67f1d5d :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Sep 16, 2026
Keep router model namespaces from changing subscription attribution, resolve exact provider prices, and preserve unknown costs when counts or required rates are absent. Refresh the existing public pricing cache for fresh opt-in loads.

Preserve independently recorded cache classes and existing caller/application override conventions, with cache-only, free, incomplete-rate, overflow, route, and historical compatibility regressions.

Co-authored-by: Alec Gutman <44984861+Chipagosfinest@users.noreply.github.com>
@steipete
steipete force-pushed the fix/provider-aware-pricing branch from 62090a0 to 67f1d5d Compare September 16, 2026 09:55
@steipete steipete changed the title Resolve OpenCodex estimates by billing provider and refresh pricing fix(opencodex): price usage by its recorded provider Sep 16, 2026
@steipete
steipete merged commit 5e15f7e into steipete:main Sep 16, 2026
9 checks passed
@steipete

Copy link
Copy Markdown
Owner

Landed in 5e15f7e. Thanks @Chipagosfinest!

Verified with make check (zero violations) and make test-fast FILTER='OpenCodexProviderPricingTests|ModelsDevPricingTargetResolverTests|OpenCodexUsageFanOutTests|OpenCodexRouteDispatcherTests|OpenCodexUsageParserTests|OpenCodexUsageOverflowTests|SpendDashboardOpenCodexPricingRefreshTests|CostUsageStoreTests|CostUsageCustomPricingTests|CostUsagePricingTests|ModelsDevPricingTests|ProviderArchitectureGatekeeperTests': 281 tests passed. Independent review is clean. Exact-head CI passed, including both macOS shards and Linux builds; main was pulled and verified clean after merge.

The actual rebuilt CLI also passed an offline, synthetic macOS 15.7.7 VM check through usage.jsonl import, six persisted SQLite rows, pricing, and JSON output: 346 tokens, four priced requests, two unpriced, and $0.000768 in known estimates. Cache-only usage retained its $0.00015 charge, an exact free rate stayed zero, and missing cache-write prices omitted today's cost. The sandbox denied provider/network and Keychain access; guest fixtures/defaults were removed afterward. This validates local estimates, without claiming live provider billing or a live catalog refresh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants