refuse local media jobs when the vLLM container is holding the GPU - #4808
Merged
Conversation
…4766) A serving vLLM Qwen3.8-27B container holds ~23 GB of a 24 GB RTX 3090, and unloadResidentModels() cannot see it — it evicts loopback Ollama / LM Studio only. Every GPU-heavy local job therefore proceeded, allocated, and died inside its model load with an out-of-memory error naming neither vLLM nor a remedy, sending the operator to debug the wrong subsystem. prepareLocalMemory() now carries an additive blockers array from a new detectGpuBlockers(): image, video, image-to-3D and LoRA-training jobs fail fast with prose that names the container, its project directory and the \`docker compose --profile single stop\` that frees the card. PortOS detects and refuses; it never stops the container, mirroring its existing refusal to start one — nothing would restart it, an attached agent session dies with it, and a cold start is ~5-7 minutes. The probe is skipped entirely when no vllmBacked provider is enabled and on any non-CUDA host, so the common path gains no latency. A probe that fails, times out, or throws means the container is not serving and the job proceeds: \"couldn't check\" never collapses into \"is blocking\".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
unloadResidentModels()cannot see it — it evicts loopback Ollama / LM Studio only. Every GPU-heavy local job proceeded, allocated, and died inside its model load with an OOM naming neither vLLM nor a remedy.detectGpuBlockers()inserver/lib/localMemory.jsprobes an enabledvllmBackedprovider's local endpoint and surfaces the result as an additiveblockersarray on theprepareLocalMemory()report.blockers, with prose naming the container, its project directory, and thedocker compose --profile single stopthat frees the card.vllmBackedprovider and on any non-CUDA host. Per the sentinel rule, a probe that fails, times out, or throws means "not serving" → the job proceeds; only an endpoint that actually answers blocks (including a 401/403, which is a container up behindVLLM_API_KEY).jobsentry and stepwise temp dir the way the busy branch already does, instead of only releasing the claim.Test plan
cd server && npm test— 1565 files, 32,773 tests green.server/lib/localMemory.test.js(17 cases): no probe with no enabledvllmBackedprovider (and no CUDA probe either), no probe when it is disabled, no probe on a non-CUDA host, still probes on anunknownCUDA verdict, blocker when the endpoint answers, blocker on a 401, no blocker when the probe fails / times out / throws, no blocker for a vLLM on another machine, one probe per endpoint preferring the credentialed record, no blocker when the provider store cannot be read, and the message naming the container / project dir / stop command.server/services/videoGen/local.test.jsasserts a blocked render rejects with the stop-command prose, never reachesspawnDetached, and releases the heavy claim.prepareLocalMemorymocks updated to carryblockers, so a schema drift fails rather than silently passing.Closes #4766