Repository navigation
One self-hosted model description for Codex and Claude Code workers - #13
Merged
Merged
Conversation
The provider description added in 0.4.0 was shaped around Codex. It now lives in src/model-provider.mjs with an `apis` list (responses, chat, anthropic) and one mapping per worker program: - Codex: config.toml for cloud workers, --config overrides for local ones - Claude Code: Anthropic Messages API at the server root, bearer key, every model alias on the served model, real context window and compaction point, no nonessential traffic or per-turn token reminder Local lite workers and factory-lite accept the description and its key. Cloud lite workers keep running Codex and refuse a provider without the Responses API. Fixes #11.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
0.4.0's
plan.modelProviderwas shaped around Codex. This makes it one description of a self-hosted endpoint that each worker program maps to its own settings.src/model-provider.mjs:validateModelProvider, with a newapislist (responses,chat,anthropic). It defaults to['responses'], so 0.4.0 descriptions keep working.providerPrograms(provider), which reports which worker programs can use an endpoint.codexProviderConfig(theconfig.tomlcloud workers already used, byte-identical) andcodexProviderOverrides(--configentries for local workers).claudeProviderEnvironment. It sends Claude Code to the Anthropic Messages API at the server root, with the key as a bearer token and every model alias on the served model. It sets the real context window and compaction point, turns off telemetry, update checks and error reports, and turns off the per-turn token reminder. That reminder is an extra system message that templates move to the front of the prompt, which defeats the server's prompt cache./v1/messages; checked in its source (vllm/entrypoints/anthropic/api_router.py).liteWorkerCommand/startLiteWorkeracceptmodelProvider,providerKeyandproviderIndexfor both Codex and Claude Code. The key only goes into the program's environment, never into arguments.factory-lite ... --model-provider endpoint.jsonreads the key from the variable the file names.Fixes #11
Local CI (AGENTS.md)
Same steps as
.github/workflows/public-sdk.yml(the focused list is read from the workflow file), on a cleangit archiveexport of 567251b with freshnpm ci:node:22.17.0-bookwormnode:24.22.2-bookwormNo model or cloud call was made.