Skip to content

fix(vercel): fill non-language model costs from gateway pricing#3303

Open
rekram1-node wants to merge 1 commit into
devfrom
fix/vercel-non-language-cost
Open

fix(vercel): fill non-language model costs from gateway pricing#3303
rekram1-node wants to merge 1 commit into
devfrom
fix/vercel-non-language-cost

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Summary

  • Vercel sync previously only mapped chat-style input+output token pricing, so embed/image/video/audio models often had no [cost] at all.
  • Parse token-like pricing when available (including input-only for embeddings/speech/rerank, with output = 0).
  • For non-token shapes (per-image, per-second video) or missing pricing, write explicit zero costs and seed a leading header comment that survives sync refreshes until better non-token cost tracking exists.
  • Resynced 96 Vercel models so every gateway entry now has a cost block.

Test plan

  • bun test packages/core/test/sync.test.ts (including new embed/zeroed pricing cases)
  • bun run vercel:generate updates 96 models, then re-run is fully unchanged
  • bun validate succeeds
  • Confirm a sample embed model has parsed input cost and a sample image/video model has zeros + header

Parse token-like pricing for embeddings, speech, and similar models.
When Vercel only exposes non-token prices (image/video) or none, write
explicit zero costs with a persistent header note for future tracking.
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/vercel/models/zai/glm-4.6v-flash.toml, providers/vercel/models/perplexity/sonar.toml, perplexity/sonar-pro.toml, perplexity/sonar-reasoning-pro.toml (and cohere/rerank-v3.5.toml, rerank-v4-fast.toml, rerank-v4-pro.toml) - Check: Zeroed placeholder cost + "Vercel pricing for this model is not token-based" header must not be applied to token-based models. Why: These files declare token context windows (128_000–200_000) and, for glm-4.6v-flash/sonar-reasoning-pro, reasoning = true + tool_call — i.e. they are unambiguously token-priced LLMs. The seeded header ("not token-based … per-image, per-second video") is directly contradicted by the model's own [limit] context in the same file, and input = 0, output = 0 surfaces in api.json as "free," which is unsupported for premium search/reasoning LLMs and conflicts with their known per-token pricing. The contrast with voyage/rerank-2.5*.toml (same model class, but receiving real token input prices) shows the header conflates "Vercel API returned no pricing" with "non-token model." Action: Verify against the Vercel gateway API which of these are genuinely non-token-priced vs. merely missing API pricing. For token-based LLMs/rerankers, do not write the "not token-based" header and do not emit a false zero cost (preserve absent cost, or source the real price); reserve the zeroed placeholder for true per-image/per-second/per-request models.
  • [medium] [possible mistake] packages/core/src/sync/index.ts (header selection, diff lines 135-141) and packages/core/src/sync/providers/vercel.ts (diff lines 169-170) - Check: The seeded "Cost currently zeroed" header must be cleared when the placeholder condition no longer holds. Why: The new logic header = existingHeader || (translated.header ?? "") makes existing headers always win, and vercel.translateModel only sets header when costPlaceholder is true (else undefined). If the Vercel API later exposes real token pricing for a currently-zeroed model, costPlaceholder becomes false, translated.header is undefined, but existingHeader still returns the stale "Cost currently zeroed … not token-based" block — so the rewrite emits that header above a non-zero [cost], producing a self-contradictory file. There is no test covering header preservation/clearing across a placeholder→real transition. Action: Add a clearing path (e.g., have the provider return an explicit empty/sentinel header when not a placeholder and have the runner honor it, or make the header conditional on the current placeholder state rather than unconditionally preserving it) and add a test for the refresh-when-pricing-appears case.
  • [low] [possible mistake] PR body / providers/vercel/models/openai/text-embedding-3-large.toml, tts-1.toml, whisper-1.toml, voyage/voyage-3-large.toml, etc. - Check: Material factual pricing changes should cite a first-party source. Why: This PR sets concrete per-million-unit prices for ~30 embedding/TTS/whisper/rerank models (e.g. tts-1 input = 15, whisper-1 input = 0.0001, text-embedding-3-large input = 0.13) plus the zero-cost claims above, but the PR body cites no Vercel gateway endpoint or pricing page. TTS/Whisper pricing is per-character or per-minute on the upstream provider, so the unit mapped into cost.input (documented as "per million tokens") cannot be reviewed from the diff alone. Action: Add a citation to the Vercel AI Gateway models/pricing endpoint (or the relevant provider pricing page) stating what it supports, and clarify the billing unit for the speech/transcription entries so the mapped cost.input values can be verified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant