fix(costs): count cached input tokens once

[justinhelmer]

Switchboard reports the cost of agent runs. This change counts cached input once so estimates and spend gates use the correct price tier.

Why: OpenAI includes cache reads and writes in its input total. The meter kept that total as ordinary input, then charged both cache buckets again. A 146,505-token staging turn also entered the 272,000-token tier too early. See the failed staging acceptance.

Where to look

1. Normalize the provider usage Remove cache reads and writes from inclusive input totals before the shared price calculation. ⚠ Keep Anthropic input counts unchanged.
2. Prove the observed cost and tier boundary Price a cached prompt against literal expected dollars, and retain the real long-context tier.
3. State the metering contract Define disjoint input buckets and link the regression proof.
4. Keep invalid cache usage visible Preserve the overreported-cache flag on buffered and streamed turn spans without negative ordinary input.

Feedback wanted: Check inclusive input normalization on both OpenAI wire shapes and confirm that provider-reported cost still takes precedence.

Risk: A wrong cache split can misstate run costs or spend gates for proxied model calls. Revert this commit to restore the prior meter. Stored historical estimates are not rewritten.

Verified: 300 focused tests, root types, scoped lint/format, specs and hygiene passed. Fresh current-head native review, CI and live acceptance remain pending.

Decisions (1)

• Normalize at the wire boundary. The shared calculator and pi adapter already use disjoint input buckets. Preserve their arithmetic and normalize the proxy readers to the same contract.

Validation (5 criteria)

Criterion	Proof
Reproduce cached input double charging	Six assertions failed before the fix, including 146,505 inclusive input being retained as ordinary input and a $1.3075 result instead of $0.1475.
Meter complete streamed and buffered responses	npx vitest run src/core/modelProxy/usage.test.ts src/core/modelPricing.test.ts src/channels/modelProxy.test.ts: 292 passed; src/core/trace/workerTrace.test.ts: 8 passed.
Keep types, style and spec proofs valid	Root tsc, scoped ESLint/Prettier, specs:check, hygiene:check and title check passed.
Verify current-head native review and live release	Pending. No production deploy or historical data rewrite was performed for this change.
Report cache counts larger than inclusive input	Three assertions failed before the flag was added. Buffered and streamed proxy replies now record cacheOverReported:true and zero ordinary input.

For agents

The official OpenAI prompt caching guide defines cached and cache-write input inside the inclusive input total: https://developers.openai.com/api/docs/guides/prompt-caching . Installed pi openai-responses-shared.js and openai-completions.js already subtract both buckets. The original staging gate remains red; this PR does not override it or raise the $5 cap. A cheaper recurring HTTP staging smoke model is a separate change. The first native round approved e299580. The current head adds the Polylane observability suggestion and needs fresh review. cacheOverReported is a bounded boolean; no provider text or credentials enter telemetry.

Generated with Codex.

[polylane]

Note

Production impact unlikely.

Checked switchboard and its model proxy: 747,590 ok invocations and 150 exceptions over 72h, and POST /v1/responses is exercised. This change only re-buckets tokens on model.turn spans, and every consumer sums all four counters, so pricing and tier selection do not regress.

View the full analysis →

Also considered · 4 refuted

• Refuted · Token re-bucketing understates run cost and weakens the spend gate · Precedence puts a wire-reported cost above every token layer (priceTurn: provider > operator > registry), and the tier sum is unchanged, so the priced figure can only change where the provider itself reports no cost - where the fix corrects, not understates, the rate.
• Refuted · A consumer reads inputTokens as the inclusive prompt total · A search over every non-test consumer of inputTokens in src found only sums, display rows and the pricing module; no context-window, compaction or gate logic reads the input count alone, so a smaller ordinary-input number cannot change a decision.
• Refuted · The Anthropic wire is double-subtracted · Anthropic's input_tokens already excludes cache reads and creation (src/core/modelProxy/usage.ts:14-23 returns the counters untouched), so the Anthropic path keeps its previous arithmetic exactly.
• Refuted · The new span attribute breaks attribute validation on write · The key ships in the same change as the flag that emits it, typed boolean in ATTR_TYPE, and ATTR_KEYS derives from that table (attrs.ts:388); an unregistered key would have been dropped as invalid, not thrown.

Analysed against 5 Cloudflare Workers and 1 repository

• Cloudflare Workers: switchboard, switchboard-controlled-acceptance-source, switchboard-memory, switchboard-resident, switchboard-sandbox
• Repository: coreplanelabs/switchboard

_{Polylane analysed 54f2c7e for production impact. You can ask follow-ups by mentioning @polylane in a comment.}

Rate this review: 👍 helpful · 👎 not helpful · 😕 confusing

Previous verdicts (1)

Head	Verdict	Analysis
e299580	Production impact unlikely	analysis

[coreplane-switchboard]

LGTM: no issues found

Reviewed at e299580.

For agents

Rationale: No findings: both OpenAI wire shapes normalize inclusive input before pricing and tier selection; provider-reported USD precedence and Anthropic accounting are preserved.

Full review

No findings: source review confirms that both OpenAI formats subtract cache reads and writes before pricing and tier selection, while preserving provider-reported USD precedence and Anthropic accounting.

[github-actions]

Auto-approved: coreplane-switchboard[bot] reviewed this PR and posted an LGTM verdict (see its review). This repository opted in through its REVIEW_BOT_LOGIN and REVIEW_BOT_ID variables.

[justinhelmer]

Addressed in 54f2c7e. Both OpenAI readers retain cacheOverReported: true when cache input exceeds the inclusive total. The flag reaches model.turn on buffered and streamed replies; ordinary input stays nonnegative. The bounded flag supports a count of inconsistent turns without logging provider payloads. The new assertions failed before the change, then passed with 292 pricing/proxy tests and 8 trace tests. A fresh native review is requested for this head.

[coreplane-switchboard]

LGTM: no issues found

Reviewed at 54f2c7e.

For agents

Rationale: No findings: both OpenAI readers preserve nonnegative input and the cache anomaly flag reaches model.turn through buffered and streamed proxy paths.

Full review

No findings in the source review: both OpenAI readers keep ordinary input nonnegative, and buffered and streamed proxy paths retain cacheOverReported: true on model.turn.

[github-actions]

Auto-approved: coreplane-switchboard[bot] reviewed this PR and posted an LGTM verdict (see its review). This repository opted in through its REVIEW_BOT_LOGIN and REVIEW_BOT_ID variables.