Repository navigation
Self-hosted models: a request limit the starter plan schedules, reuse of a filled weights volume, no Claude session variables; release 0.7.2 - #34
Merged
Conversation
…euse of an earlier weights volume, no Claude session variables in a worker's Claude Code; release 0.7.2 - request_timeout_minutes defaults to 30: the starter plan never schedules a GPU function with a limit of 45 minutes or more, and the deployment and every request then wait without an error (#33). - weights_volume / compile_volume: use an existing volume (for example one an earlier deployment filled) instead of creating the app's own; retire never deletes it. - claudeProviderEnvironment clears every inherited Claude session variable by name pattern (inherited, default this process), so a Claude Code child started from inside a Claude session uses the endpoint key, not the session's login (#33).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #33 (point 2, and the cause of point 1; what is left of point 1 stays in #33).
What changes
request_timeout_minutesnow defaults to 30. Modal's starter plan never schedules a GPU function whose request limit is 45 minutes or more: the deployment and every request wait without an error (measured Nov 11: 10 to 30 minutes start in seconds, 45 and 60 never). Both profiles inherit the new default; a plan that allows more can still set it. The limit is validated (1 to 1440) and shown indescribe().weights_volume/compile_volume: an existing volume (for example one an earlier deployment filled) holds the weights or vLLM's compile caches. The app uses it instead of creating its own volume, andretirenever deletes it.describe()names the volumes and says which ones the app owns.claudeProviderEnvironment(provider, { inherited })clears every inherited Claude session variable by name pattern (CLAUDE*,ANTHROPIC_*,*OAUTH*), so a Claude Code child started from inside a Claude session uses the endpoint key instead of offering the session's login (HTTP 401).inheriteddefaults to this process; the local lite worker passes its ownenv, and the cloud Claude driver passes{}because it builds its environment from scratch (so its ownCLAUDE_CONFIG_DIRand the like are never cleared).Still open from #33: the deploy still cannot tell a scheduler that never starts the GPU from a slow cold start. With the 30-minute default the known cause is gone; reading the scheduler's message is left in #33.
Tests
test/model-provider.test.mjs,test/selfhost-modal.test.mjs,test/cloud-lite-claude.test.mjs,test/cloud-lite-provider.test.mjs45/45; the whole focused list ofpublic-sdk.yml198/200, the two failures arenode:sqlitebackupmissing in Node 22.13 (needs 22.17), unrelated to this change; Python settings tests 11/11.