feat(cloudflare): sync full Workers AI catalog + mirror into AI Gateway

[michaeldwan]

1) Fix drift between Workers AI and AI Gateway

Cloudflare Workers AI models can be accessed directly, or through Cloudflare AI Gateway, but models.dev maintains them as two independent providers that have drifted apart for over a year.

Previously cloudflare-workers-ai kept Workers AI in sync each hour; AI Gateway had a hand-run shell pipeline under providers/cloudflare-ai-gateway/scripts/ that was never wired into the hourly workflow.

cloudflare-ai-gateway/models/workers-ai/@cf is now a symlink to cloudflare-workers-ai/models/@cf, so AIG's Workers AI catalog is the WAI catalog. The scripts/ + data/ pipeline is deleted, and the AIG README is rewritten to describe the symlink layout.

2) Sync the complete Workers AI model catalog

The WAI sync now covers Cloudflare's complete Workers AI catalog -- 39 new TOMLs for embeddings, ASR, TTS, translation, classification, and image gen. Previously it was only fetching text generation models due to the format=openrouter filter. Now it also fetches and reshapes models from the native ai/models/search endpoint into the same OpenRouterModel the pipeline already builds from. Where both APIs report the same model, openrouter wins, so the 22 existing chat entries stay byte-identical.

Any future WAI expansion (new task type, new model) lands in AIG on the next deploy through the same file.

Notes

The Cloudflare APIs don't provide all the data model TOMLs expose. I'll see if we can fix this on our (Cloudflare's) end:

• Output token limit and open-weights aren't in either API, so they're placeholders (0 / false) -- same fallback digitalocean.ts, chutes.ts, and wandb.ts use when a source doesn't report them. The old AIG shell pipeline this replaces incorrectly hardcoded context=128000 / output=16384 on every model.
• Unpriced tokens (audio-priced ASR/TTS, free models) use -1 (OpenRouter's own sentinel) rather than a fake 0, so price() doesn't invent cost. Audio-priced models get input_audio / output_audio applied in a follow-up pass.
• API endpoints require authentication and reflect models accessible to the authenticated user. We probably need a public unauthenticated API for use cases like this.

Sources

• Workers AI models API
• Workers AI catalog
• AI Gateway universal endpoint

[github-actions]

Action items

• [high] [violation] providers/cloudflare-workers-ai/models/@cf/openai/whisper-large-v3-turbo.toml:1 - Check: base_model must be used when a matching models/<provider>/<model>.toml entry exists. Why: models/openai/whisper-large-v3-turbo.toml exists with open_weights = true, limit.context = 448, limit.output = 448, and name = "Whisper Large v3 Turbo". The new WAI TOML is a full inline definition that duplicates all of this provider-agnostic metadata and fabricates incorrect values: open_weights = false (Whisper IS open-weight), limit.context = 0, limit.output = 0, and name = "@cf/openai/whisper-large-v3-turbo" (raw slug). The resolveCloudflareBaseModel heuristic failed to resolve the match because both whisper-large-v3 and whisper-large-v3-turbo satisfy the identity-token check, producing 2 matches instead of 1. Action: Use base_model = "openai/whisper-large-v3-turbo" and retain only provider-specific overrides (cost, attachment = true, release_date if different). Either fix the resolveCloudflareBaseModel heuristic to prefer exact filename matches over substring identity matches, or hand-add the base_model pointer (the sync runner preserves existing base_model on subsequent runs).

• [medium] [violation] packages/core/src/sync/providers/cloudflare-workers-ai.ts:applyAudioPricing - Check: cost.input_audio and cost.output_audio are documented as "Cost per million audio input/output tokens (USD)" per the README schema reference. Why: applyAudioPricing takes the native API's "per audio minute" price and puts it directly into input_audio, and takes "per 1k characters" or "per audio minute" and puts it into output_audio. These are different units from per-million-tokens. The resulting 9 new TOMLs (all ASR and TTS models: whisper, whisper-large-v3-turbo, nova-3, flux, aura-1, aura-2-en, aura-2-es, melotts, smart-turn-v2) contain audio pricing values that are not comparable with any other provider in the catalog (e.g. OpenAI input_audio = 32.00 is per-million-tokens; Cloudflare input_audio = 0.000338 is per-audio-minute). No TOML comment documents the unit discrepancy. Action: Either convert per-minute/per-character pricing to per-million-token pricing, or document the unit in a leading comment block on each affected TOML (the sync preserves leading comments), or clarify with maintainers whether input_audio/output_audio may carry non-per-million-token units.

• [medium] [possible mistake] providers/cloudflare-workers-ai/models/@cf/deepgram/flux.toml:2-3 - Check: Factual model data must be internally consistent. Why: The TOML has description = "Image model for prompt-driven generation, editing, and visual design workflows" and family = "flux" (the Black Forest Labs image-generation family), but the model's own modalities are input = ["audio"], output = ["text"] and it carries input_audio = 0.0077 — all indicating an ASR model, not an image model. The "flux" substring in the model name triggers false-positive image-model heuristics in describeModel (has(target, /\bflux\b/) → "Image model…") and inferFamily (matches "flux" in ModelFamilyValues). The native API fixture in the PR shows that the API provides an accurate description field per model, but reshapeNative discards it and buildOpenRouterModel synthesizes a generic description instead. Action: Override the description and family for @cf/deepgram/flux to reflect its ASR nature (e.g., family should not be "flux"; description should reference speech transcription). Consider preserving the native API's description field in reshapeNative so the generated TOMLs carry provider-accurate descriptions instead of heuristic ones.