You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Self-hosted model: a deploy Modal never schedules waits silently; Claude Code settings keep the caller's session variables #33
Found on Nov 11 while Greenswarm started a run on a self-hosted Qwen through FACTORY 0.7.1 (Greenswarm issues flujo-app/greenswarm#318 and #324).
1. A deploy that Modal never schedules waits silently
On a Modal "starter" plan a function with a request time limit of 45 minutes or more is never given an H100. The qwen profile asks for 60 (request_timeout_minutes), nemotron too. Modal says so in the app log ("... is waiting to be scheduled on a GPU_H100 worker. Your workspace's concurrency limits are reached (H100 unavailable on starter plan)"), but ModalSelfHostedModel.deploy() only sees a held request and waits up to 45 minutes per replica, then reports an uncertain outcome. Measured with test functions of the same size: 10 to 30 minutes start in 3 to 4 seconds, 45 and 60 minutes never start.
Wanted: while verifying, read the app's log for the scheduler's "waiting to be scheduled" line and stop with that text; and a default request limit in the profiles that the common plans accept (30 minutes worked).
2. Claude Code settings for a self-hosted provider leave the caller's Claude session variables in place
claudeProviderEnvironment() sets the base address and the key and clears ANTHROPIC_API_KEY and CLAUDE_CODE_OAUTH_TOKEN. When the caller itself runs inside a Claude session (a desktop or SDK session), the child also inherits that session's variables (CLAUDE_CODE_SDK_HAS_HOST_AUTH_REFRESH, CLAUDE_CODE_ENTRYPOINT, CLAUDECODE, USE_STAGING_OAUTH, ...). Claude Code 2.1.321 then offers the session's own login to the endpoint instead of the endpoint key: HTTP 401, and a login is sent to a server it was not meant for. An older Claude Code (2.1.289) hangs instead.
Greenswarm now clears every inherited variable whose name starts with CLAUDE or ANTHROPIC_ or contains OAUTH before applying FACTORY's settings (swarm/lib/endpoint.mjs, sessionVariablesCleared). Wanted here: the same in claudeProviderEnvironment() (null for those names), so every caller gets it.
3. Small: a stand-in endpoint for tests has to speak https
validateModelProvider() refuses http://127.0.0.1:<port>. That is right for real endpoints; for tests a loopback address could be allowed. Greenswarm's stand-in makes its own certificate meanwhile.
Released in 0.7.2 (PR #34): the request limit defaults to 30 minutes (the cause of the silent wait on the starter plan), and Claude Code settings clear the caller's Claude session variables. Still open here: the deploy cannot yet tell a scheduler that never starts the GPU from a slow cold start (reading Modal's waiting message). Point 3 (loopback http for test stand-ins) is also still open.
Also found on the Nemotron deploy of Nov 12 and fixed in 0.7.3 (PR #35): the model's own code and tokenizer files came from the Hub's default branch, not the pinned revision; vLLM now gets --code-revision and --tokenizer-revision.
Activity
Self-hosted models: a request limit Modal's starter plan schedules, r…
flujo-app commentedon Nov 12, 2026
Released in 0.7.2 (PR #34): the request limit defaults to 30 minutes (the cause of the silent wait on the starter plan), and Claude Code settings clear the caller's Claude session variables. Still open here: the deploy cannot yet tell a scheduler that never starts the GPU from a slow cold start (reading Modal's waiting message). Point 3 (loopback http for test stand-ins) is also still open.
flujo-app commentedon Nov 12, 2026
Also found on the Nemotron deploy of Nov 12 and fixed in 0.7.3 (PR #35): the model's own code and tokenizer files came from the Hub's default branch, not the pinned revision; vLLM now gets --code-revision and --tokenizer-revision.