Pass CLIProxyAPI usage and tok/s into OpenCode session telemetry

[RaviTharuma]

Feature

Map gateway-reported cost, token usage, and tokens-per-second into the host harness so a run shows real $ and tok/s, not catalog guesses.

Why

OmniRoute already emits (non-stream) X-OmniRoute-Response-Cost, X-OmniRoute-Tokens-In/Out, X-OmniRoute-Latency-Ms, X-OmniRoute-Model, X-OmniRoute-Provider, X-OmniRoute-Decision. CLIProxyAPI often has upstream usage and, on some builds, internal TPS — but plugins typically register models and stop there.

Harness cost UIs (OpenCode session cost, Pi usage, OMP status) therefore:

• price the requested model from models.dev / /api/pricing even after a combo/credential failover
• omit tok/s, or compute it client-side from deltas (wrong on buffered streams and tool pauses)

Live generation speed cannot come from models.dev. It has to ride on the completion.

Requested behavior

1. On each completion (stream final chunk included), read gateway usage/cost/tok/s headers or usage fields and write them into the host’s usage/cost/speed slots.
2. If the gateway updates the routed model mid-request (combo / credential failover), update the host’s displayed model id for that turn.
3. Prefer gateway numbers over catalog cost × token estimates when both exist.
4. If the gateway has no tok/s yet, do not invent one from tokens / latency_ms (that includes TTFT). Show — until the gateway field exists.
5. Keep catalog sync (models.dev / /v1/models / /api/pricing) for picker metadata; this issue is per-request telemetry.

Related gateway asks (please link rather than re-implement scraping):

• OmniRoute: tok/s + streaming cost headers
• CLIProxyAPI: tok/s / usage on the inference response

Acceptance

• One streamed turn shows host cost + tok/s that match the gateway response.
• After failover, the turn is attributed to the model that answered.
• No extra HTTP calls to dashboards or log APIs.

[RaviTharuma]

Gateway-side request for usage + tok/s on the inference response (not only operator dashboards):

• feat(api): expose usage + tokens-per-second on inference responses for harness plugins router-for-me/CLIProxyAPI#5468

Plugin mapping requests (prefer gateway tok/s over local delta estimates):

• OpenCode: Pass CLIProxyAPI usage and tok/s into OpenCode session telemetry GRomR1/opencode-cliproxiapi-auth#11
• OpenCode: Pass CLIProxyAPI usage and tok/s into OpenCode session telemetry #5
• Pi (official): Pass CLIProxyAPI usage/cost and gateway tok/s into Pi (not local TPS guess only) router-for-me/pi-cliproxyapi-provider#24
• Pi: Pass CLIProxyAPI usage/cost and gateway tok/s into Pi 0xRichardH/pi-cliproxyapi-provider#14

[RaviTharuma]

Mapping contract (depends on the gateway field)

Do not invent tok/s as tokens_out / latency_ms. That includes TTFT and tool pauses.

Prefer, in this order:

1. Gateway usage.tokens_per_second / X-OmniRoute-Tokens-Per-Second / X-CLIProxy-Tokens-Per-Second on the final response or terminal stream event
2. Else show — for speed
3. Cost: prefer per-request X-OmniRoute-Response-Cost (OmniRoute) or equivalent CPA field when present; catalog /api/pricing / models.dev is only a fallback for picker estimates, never for the completed turn after combo failover
4. Attribute the turn to the winning X-OmniRoute-Model / X-OmniRoute-Provider (or CPA selected-auth / model), not the requested combo id

Gateway issues:

• OmniRoute: feat(api): emit tokens-per-second (and streaming cost) on inference responses diegosouzapw/OmniRoute#12616
• CLIProxyAPI: feat(api): expose usage + tokens-per-second on inference responses for harness plugins router-for-me/CLIProxyAPI#5468

Why this plugin is in the top 2

This is not a random pick. Ranking used actual source, not README claims: catalog sync, models.dev / pricing overlay, protocol choice, failover behavior, tests, activity, and whether per-request usage already exists.

Tests

Please add/adjust automated tests that:

• parse a mocked final stream/response with tok/s + cost + winning model
• refuse to compute tok/s from wall-clock latency
• keep previous catalog on failed sync (if this plugin owns catalog sync)

Follow this repo's existing test command and PR checklist. No private hostnames, LAN IPs, home paths, or API keys in fixtures.

[RaviTharuma]

Gateway implementation PRs (do not invent tok/s as tokens/total_latency):

• OmniRoute #12616 → feat(api): emit gateway-measured tokens-per-second excluding TTFT diegosouzapw/OmniRoute#12631 (release/v3.8.51)
• CLIProxyAPI #5468 → feat(usage): expose gateway-measured tokens_per_second on inference responses router-for-me/CLIProxyAPI#5474 (dev)

Contract: usage.tokens_per_second + response header, TTFT excluded, omit when TTFT unknown. Plugin mapping still blocked on merge; keep showing — until the field exists.