Skip to content

providers.md

Latest commit

 

History

History
355 lines (267 loc) · 17.8 KB

providers.md

File metadata and controls

355 lines (267 loc) · 17.8 KB
Raw

Providers

Diffex supports subscription-based providers via OAuth and API key providers via environment variables or auth file. Built-in catalogs ship with Diffex; configured providers may refresh newer catalogs and cache them in ~/.diffex/agent/models-store.json for offline use.

Table of Contents

Subscriptions

Use /login in interactive mode, then select a provider:

  • ChatGPT
  • Claude Pro/Max
  • GitHub Copilot
  • xAI (Grok/X subscription)
  • OpenRouter (OAuth-minted API key billed from OpenRouter credits)

Use /logout to clear credentials. Tokens are stored in ~/.diffex/agent/auth.json and auto-refresh when expired. OpenRouter instead mints a user-controlled API key that does not expire automatically.

ChatGPT

Run /login, select ChatGPT, and continue in your browser to register Diffex and authorize ChatGPT plan usage. Existing installations using the inherited client registration must sign in again. The provider ID remains openai-codex for configuration and CLI selection.

Diffex stores the stable host identity and issued account registrations in ~/.diffex/agent/chatgpt/, alongside the active credentials in auth.json. DIFFEX_CODING_AGENT_DIR relocates both directories. Later sign-ins let you select a saved registration or add another ChatGPT account or workspace. If you decline plan usage, Diffex retains your validated sign-in and offers Enable ChatGPT plan usage on the next login. Remote machines can complete sign-in by pasting the full loopback redirect URL into Diffex.

Models come from the signed-in account's public model catalog, with an account-specific cache for offline startup. Inference uses the public Responses API with streaming and storage disabled. Availability depends on the selected account, workspace, and granted permissions. Token usage is reported without applying API-key token prices to subscription usage. Review app limits and credits in ChatGPT Settings - Usage.

When choosing a default, Diffex prefers gpt-5.6-terra if it is in the available ChatGPT catalog; otherwise it uses the first model in the account's catalog order. Available saved defaults and session models are preserved, not automatically upgraded.

At startup or session restoration, if a saved ChatGPT model is no longer available, Diffex reports the replacement and stays with ChatGPT while that account has available models. It falls back to another configured provider only when ChatGPT has none. These fallbacks do not rewrite your saved default; use /model to save a new choice.

Login selects and saves an available model when there is no current model or switching ChatGPT accounts makes the current ChatGPT model unavailable. If the newly authenticated provider has no models, login reports that instead of selecting a different provider. Logging in does not replace an available current choice or an unrelated custom model.

/logout stops the account's active requests, attempts remote session revocation, and clears the active tokens while retaining the host and registration for later sign-in. If remote revocation cannot be confirmed, Diffex reports that local sign-out completed and directs you to disconnect the app in ChatGPT Settings. An unusable refresh token clears the local token set and requires a new sign-in with the saved registration. Temporary network failures preserve credentials for retry.

See OpenAI's registration guide and models and inference guide.

Claude Pro/Max

Anthropic subscription auth is active for Claude Pro/Max accounts. Third-party harness usage draws from extra usage and is billed per token, not against Claude plan limits.

GitHub Copilot

  • Press Enter for github.com, or enter your GitHub Enterprise Server domain
  • If you get "model not supported", enable it in VS Code: Copilot Chat → model selector → select model → "Enable"

xAI (Grok/X subscription)

  • Run /login xai, then select Use a subscription
  • XAI_API_KEY remains available through Use an API key

OpenRouter

  • Run /login openrouter, then select Sign in with OpenRouter to open the OpenRouter PKCE authorization flow
  • The authorization creates a user-controlled OpenRouter API key billed from your OpenRouter credits
  • On remote/headless machines (e.g. over SSH) the browser cannot reach the loopback callback; paste the final redirect URL (or the authorization code) into the login prompt instead
  • OPENROUTER_API_KEY remains available through Use an API key

API Keys

Environment Variables or Auth File

Use /login in interactive mode and select a provider to store an API key in auth.json, or set credentials via environment variable:

export ANTHROPIC_API_KEY=sk-ant-...
diffex
Provider Environment Variable auth.json key
Anthropic ANTHROPIC_API_KEY anthropic
Ant Ling ANT_LING_API_KEY ant-ling
Azure OpenAI Responses AZURE_OPENAI_API_KEY azure-openai-responses
OpenAI OPENAI_API_KEY openai
DeepSeek DEEPSEEK_API_KEY deepseek
NVIDIA NIM NVIDIA_API_KEY nvidia
Google Gemini GEMINI_API_KEY google
Amazon Bedrock AWS_BEARER_TOKEN_BEDROCK amazon-bedrock
Mistral MISTRAL_API_KEY mistral
Groq GROQ_API_KEY groq
Cerebras CEREBRAS_API_KEY cerebras
Cloudflare AI Gateway CLOUDFLARE_API_KEY (+ CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_GATEWAY_ID) cloudflare-ai-gateway
Cloudflare Workers AI CLOUDFLARE_API_KEY (+ CLOUDFLARE_ACCOUNT_ID) cloudflare-workers-ai
xAI XAI_API_KEY xai
OpenRouter OPENROUTER_API_KEY openrouter
Vercel AI Gateway AI_GATEWAY_API_KEY vercel-ai-gateway
ZAI Coding Plan (Global) ZAI_API_KEY zai
ZAI Coding Plan (China) ZAI_CODING_CN_API_KEY zai-coding-cn
OpenCode Zen OPENCODE_API_KEY opencode
OpenCode Go OPENCODE_API_KEY opencode-go
Hugging Face HF_TOKEN huggingface
Fireworks FIREWORKS_API_KEY fireworks
Together AI TOGETHER_API_KEY together
Baseten BASETEN_API_KEY baseten
Kimi For Coding KIMI_API_KEY kimi-coding
MiniMax MINIMAX_API_KEY minimax
MiniMax (China) MINIMAX_CN_API_KEY minimax-cn
Qwen Token Plan (existing catalog) QWEN_TOKEN_PLAN_API_KEY qwen-token-plan
Qwen Token Plan (Individual) QWEN_TOKEN_PLAN_API_KEY qwen-token-plan-individual
Qwen Token Plan (China) QWEN_TOKEN_PLAN_CN_API_KEY qwen-token-plan-cn
Xiaomi MiMo XIAOMI_API_KEY xiaomi
Xiaomi MiMo Token Plan (China) XIAOMI_TOKEN_PLAN_CN_API_KEY xiaomi-token-plan-cn
Xiaomi MiMo Token Plan (Amsterdam) XIAOMI_TOKEN_PLAN_AMS_API_KEY xiaomi-token-plan-ams
Xiaomi MiMo Token Plan (Singapore) XIAOMI_TOKEN_PLAN_SGP_API_KEY xiaomi-token-plan-sgp

Reference for environment variables and auth.json keys: const envMap in packages/ai/src/env-api-keys.ts.

Auth File

Store credentials in ~/.diffex/agent/auth.json:

{
  "anthropic": { "type": "api_key", "key": "sk-ant-..." },
  "ant-ling": { "type": "api_key", "key": "..." },
  "openai": { "type": "api_key", "key": "sk-..." },
  "deepseek": { "type": "api_key", "key": "sk-..." },
  "nvidia": { "type": "api_key", "key": "nvapi-..." },
  "google": { "type": "api_key", "key": "..." },
  "opencode": { "type": "api_key", "key": "..." },
  "opencode-go": { "type": "api_key", "key": "..." },
  "together": { "type": "api_key", "key": "..." },
  "qwen-token-plan":  { "type": "api_key", "key": "sk-sp-..." },
  "qwen-token-plan-individual": { "type": "api_key", "key": "sk-sp-..." },
  "qwen-token-plan-cn": { "type": "api_key", "key": "sk-sp-..." },
  "xiaomi": { "type": "api_key", "key": "..." },
  "xiaomi-token-plan-cn":  { "type": "api_key", "key": "..." },
  "xiaomi-token-plan-ams": { "type": "api_key", "key": "..." },
  "xiaomi-token-plan-sgp": { "type": "api_key", "key": "..." }
}

qwen-token-plan-individual uses the same international endpoint and QWEN_TOKEN_PLAN_API_KEY as qwen-token-plan, but limits the picker to the models documented for Individual subscriptions. The existing provider keeps its broader catalog for backward compatibility. When using auth.json, store the credential under the provider you select; an environment variable is shared by both international providers.

The file is created with 0600 permissions (user read/write only). Auth file credentials take priority over environment variables.

API key credentials can also include provider-scoped environment values. These values are used before process environment variables when resolving the credential key, provider/model headers, and provider configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, DIFFEX_CACHE_RETENTION, and HTTP_PROXY/HTTPS_PROXY.

{
  "cloudflare-ai-gateway": {
    "type": "api_key",
    "key": "$CLOUDFLARE_API_KEY",
    "env": {
      "CLOUDFLARE_API_KEY": "...",
      "CLOUDFLARE_ACCOUNT_ID": "account-id",
      "CLOUDFLARE_GATEWAY_ID": "gateway-id"
    }
  }
}

Use this when Diffex should use different provider settings than the project shell environment.

Key Resolution

The key field supports command execution, environment interpolation, and literals:

  • Shell command: "!command" at the start executes the whole value as a command and uses stdout (cached for process lifetime) 
    { "type": "api_key", "key": "!security find-generic-password -ws 'anthropic'" }
{ "type": "api_key", "key": "!op read 'op://vault/item/credential'" }
  • Environment interpolation: "$ENV_VAR" or "${ENV_VAR}" uses the value of the named variable. Interpolation works inside larger literals. 
    { "type": "api_key", "key": "$MY_ANTHROPIC_KEY" }
{ "type": "api_key", "key": "${KEY_PREFIX}_${KEY_SUFFIX}" }
    $FOO_BAR is the variable FOO_BAR; use ${FOO}_BAR when BAR is literal text. Missing environment variables make the value unresolved.
  • Escapes: "$$" emits a literal "$"; "$!" emits a literal "!" without triggering command execution. 
    { "type": "api_key", "key": "$$literal-dollar-prefix" }
{ "type": "api_key", "key": "$!literal-bang-prefix" }
  • Literal value: Used directly. Plain uppercase strings such as MY_API_KEY are literals; use $MY_API_KEY for environment variables. 
    { "type": "api_key", "key": "sk-ant-..." }
{ "type": "api_key", "key": "public" }

OAuth credentials are also stored here after /login and managed automatically.

Cloud Providers

Azure OpenAI

export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_BASE_URL=https://your-resource.ai.azure.com
# also supported: https://your-resource.cognitiveservices.azure.com
# also supported: https://your-resource.openai.azure.com
# root endpoints are auto-normalized to /openai/v1
# or use resource name instead of base URL
export AZURE_OPENAI_RESOURCE_NAME=your-resource

# Optional
export AZURE_OPENAI_API_VERSION=2024-02-01
export AZURE_OPENAI_DEPLOYMENT_NAME_MAP=gpt-4=my-gpt4,gpt-4o=my-gpt4o

Amazon Bedrock

Use /login amazon-bedrock to store a Bedrock API key, or configure one of the ambient AWS credential sources below:

# Option 1: AWS Profile
export AWS_PROFILE=your-profile

# Option 2: IAM Keys
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...

# Option 3: Bearer Token
export AWS_BEARER_TOKEN_BEDROCK=...

# Optional region (defaults to us-east-1)
export AWS_REGION=us-west-2

Also supports ECS task roles (AWS_CONTAINER_CREDENTIALS_*) and IRSA (AWS_WEB_IDENTITY_TOKEN_FILE).

diffex --provider amazon-bedrock --model us.anthropic.claude-sonnet-4-20250514-v1:0

Prompt caching is enabled automatically for Claude models whose ID contains a recognizable model name (base models and system-defined inference profiles). For application inference profiles (whose ARNs don't contain the model name), set AWS_BEDROCK_FORCE_CACHE=1 to enable cache points:

export AWS_BEDROCK_FORCE_CACHE=1
diffex --provider amazon-bedrock --model arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123

If you are connecting to a Bedrock API proxy, the following environment variables can be used:

# Set the URL for the Bedrock proxy (standard AWS SDK env var)
export AWS_ENDPOINT_URL_BEDROCK_RUNTIME=https://my.corp.proxy/bedrock

# Set if your proxy does not require authentication
export AWS_BEDROCK_SKIP_AUTH=1

# Set if your proxy only supports HTTP/1.1
export AWS_BEDROCK_FORCE_HTTP1=1

Cloudflare AI Gateway

CLOUDFLARE_API_KEY can be set via /login. The account ID and gateway slug can be set as environment variables or in the API key credential's env object in auth.json.

export CLOUDFLARE_API_KEY=...           # or use /login
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=...        # create at dash.cloudflare.com → AI → AI Gateway
diffex --provider cloudflare-ai-gateway --model "claude-sonnet-4-5"

Routes to OpenAI, Anthropic, and Workers AI through Cloudflare AI Gateway. Workers AI uses the Unified API (/compat) and prefixed model IDs (workers-ai/@cf/...). OpenAI uses the OpenAI passthrough route (/openai) with native OpenAI model IDs such as gpt-5.1. Anthropic uses the Anthropic passthrough route (/anthropic) with native Anthropic model IDs such as claude-sonnet-4-5.

AI Gateway authentication uses CLOUDFLARE_API_KEY as cf-aig-authorization. Upstream authentication can be one of:

Mode Request auth Upstream auth
Workers AI Cloudflare token only Cloudflare-native
Unified billing Cloudflare token only Cloudflare handles upstream auth and deducts credits
Stored BYOK Cloudflare token only Cloudflare injects provider keys stored in the AI Gateway dashboard
Inline BYOK Cloudflare token plus upstream Authorization header The request supplies the upstream provider key

For normal Diffex usage, prefer unified billing or stored BYOK. Inline BYOK requires configuring an additional upstream Authorization header for the Cloudflare AI Gateway provider, for example via a models.json provider/model override.

Cloudflare Workers AI

CLOUDFLARE_API_KEY can be set via /login. CLOUDFLARE_ACCOUNT_ID can be set as an environment variable or in the API key credential's env object in auth.json.

export CLOUDFLARE_API_KEY=...           # or use /login
export CLOUDFLARE_ACCOUNT_ID=...
diffex --provider cloudflare-workers-ai --model "@cf/moonshotai/kimi-k2.6"

Diffex automatically sets x-session-affinity for prefix caching discounts.

Google Vertex AI

Uses Application Default Credentials:

gcloud auth application-default login
export GOOGLE_CLOUD_PROJECT=your-project
export GOOGLE_CLOUD_LOCATION=us-central1

Or set GOOGLE_APPLICATION_CREDENTIALS to a service account key file.

llama.cpp

Diffex supports the llama.cpp router server. Configure it with /login llama.cpp, manage loaded models with /llama, and select a loaded model with /model.

See llama.cpp for server setup, model directory layout, environment variables, and command usage.

Custom Providers

Via models.json: Add Ollama, LM Studio, vLLM, or any provider that speaks a supported API (OpenAI Completions, OpenAI Responses, Anthropic Messages, Google Generative AI). See models.md.

Via extensions: For providers that need custom API implementations or OAuth flows, create an extension. See custom-provider.md and examples/extensions/custom-provider-gitlab-duo.

Resolution Order

When resolving credentials for a provider:

  1. CLI --api-key flag
  2. auth.json entry (API key or OAuth token)
  3. Environment variable
  4. Custom provider keys from models.json

Removed Providers

Radius is removed and must not be used in development, testing, evaluation, or production. It is absent from subscription and API-key login menus, provider discovery, and model catalogs. Custom registrations under the radius provider ID are rejected.

Existing Radius selections explain that support was removed and require a replacement provider. Remove providers.radius and any provider with "oauth": "radius" from ~/.diffex/agent/models.json. These entries invalidate the custom configuration file until removed; built-in providers remain available. Replace Radius defaults in global or project settings.json, including defaultProvider, defaultModel, and any Radius entries in enabledModels. Resume an old Radius session with an explicit --model provider/model to choose its replacement.

Saved Radius credentials and cached models remain on disk but cannot restore access or trigger token refresh, catalog discovery, or inference. Use /logout to delete a saved Radius credential locally without contacting the former service, or remove its entry from auth.json manually. Remove the unused Radius entry from models-store.json and unset the former RADIUS_API_KEY environment variable. Configure another supported provider through /login, then select its model with /model.