Guided install for the vLLM Qwen3.8-27B stack (clone, build, prepare) - #4818
Merged
Conversation
…wen3.8-27B stack (#4767) Getting the vLLM stack running meant leaving PortOS entirely: read docs/features/qwen38-rtx3090.md, clone the upstream compose project, hand-write five .env settings, build, prepare, come back and paste an API key. Four of those settings are load-bearing and each one fails in a way that points somewhere other than its cause — a wrong tool-call parser answers normally and silently never calls a tool; a missing WSL2 pin-memory flag turns into a quiet compose crash-loop; the wrong allocator config reads as an OOM that isn't one. The checklist now offers "Clone, build & prepare vLLM (Qwen3.8-27B) (~30 GB), then start" — the same name-the-payload consent shape MTPLX's weights download already used, so the click itself is the consent. It clones, writes the .env, builds the image, runs prepare, and starts the container, streaming every line. Re-running skips whatever already landed. install() still refuses. Docker, the NVIDIA Container Toolkit and WSL2 are host-level decisions with driver requirements PortOS cannot judge; this provisions the project on a host that is already capable of running it. Two judgment calls worth knowing about: - WSL2 detection counts win32, not just a matching /proc/version. PortOS commonly runs as a native Win32 process driving Docker Desktop, whose engine IS a WSL2 VM with no /proc/version to read — keying only on the file would omit the two mandatory WSL2 variables on exactly the host that needs them. - On Windows the action refuses unless VLLM_QWEN_PROJECT_DIR is set. The default ~/qwen-serving resolves to a Windows home, and 20 GB of weights read from the WSL2 VM across a 9p share is a mistake that costs 20 GB to undo. An existing .env is never overwritten: only keys it does not already mention are appended, and the key that ends up IN EFFECT is the one written onto the seeded vLLM providers — so an operator's own VLLM_API_KEY, GPU_UTIL or DFLASH_TOKENS all survive. The key is stored and never logged. Supporting work rolled in: the provisioning action is a table (PROVISION_STEPS) rather than a hardcoded pull-start, actions declare what they cover (installs/starts/provisions) instead of the checklist matching substrings of their names, the payload carries `provisions` so the client stops re-deriving it, and the orchestration lives in its own manager module beside mtplxServerManager/llamaServerManager rather than inside the registry table. readRuntimeWeights now skips runtimes this platform cannot run, so a Mac with Docker no longer sweeps for an RTX 3090 project every minute. The manual sequence stays in the feature doc as the fallback and the explanation. The WSL2 memory ceiling stays manual too — raising it needs `wsl --shutdown`, which would take down a PostgreSQL container this install may be using — so PortOS detects it and names the values instead.
…M provisioning tests (#4767) - `setupHint` now documents that its parameter is one of the `installs` / `starts` / `provisions` axes the action table declares. - Assert the Docker daemon's own words reach the operator, not just PortOS's "not answering" wrapper — the underlying message is what says where to look. - When a provider write fails, assert the generated key really is in `.env`, so the "paste it yourself" instruction the run emits is actually followable. - A `repin()` helper for the two tests that change platform mid-test.
…4767) #4778 landed `qwenAgentParsers.js` while this branch was open, and its `vllmExtraArgs()` exists for exactly this transport — the `.env` line the guided install writes. Read the spelling from that table instead of retyping it, so a correction there reaches the install without a second edit. Its guard test enforces this; the two assertions now compare against the table's own answer rather than a second copy of the string.
atomantic
force-pushed
the
claim/issue-4767
branch
from
August 22, 2026 04:46
f01703f to
5579eaf
Compare
This was referenced Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SETUP_ROWS.vllm.install()always refused with a pointer to the docs, so getting this stack running meant leaving PortOS entirely: clone the upstream compose project, hand-write five.envsettings, build, prepare, come back and paste an API key. Four of those settings are load-bearing and each fails in a way that points somewhere other than its cause — the wrong tool-call parser answers normally and silently never calls a tool; a missing WSL2 pin-memory flag becomes a quiet compose crash-loop; the wrong allocator config reads as an OOM that isn't one.The readiness checklist now offers "Clone, build & prepare vLLM (Qwen3.8-27B) (~30 GB), then start" — the same name-the-payload consent shape MTPLX's weights download already used, so the click itself is the consent. It clones, writes the
.env, builds the image, runsprepare, and starts the container, streaming every line. Re-running skips whatever already landed.install()still refuses. Docker, the NVIDIA Container Toolkit and WSL2 are host-level decisions with driver requirements PortOS cannot judge; this provisions the project on a host already capable of running it.Two judgment calls
win32, not just a matching/proc/version. The issue's scope named the feature doc'sgrep -qi microsoft /proc/versiontest, which is right for an operator typing inside the distro. PortOS commonly runs as a native Win32 process driving Docker Desktop, whose engine is a WSL2 VM with no/proc/versionto read — keying only on the file would omit the two mandatory WSL2 variables on exactly the host that needs them, producing the silent crash-loop they exist to prevent.VLLM_QWEN_PROJECT_DIRis set. The default~/qwen-servingresolves to a Windows home, and ~20 GB of weights read from the WSL2 VM across a 9p share is a mistake that costs 20 GB to discover and 20 GB to undo. The refusal names the UNC-path fix. Setting the override is itself the operator's explicit placement decision, so it only fires when nothing was configured.Safety properties
.envis never overwritten — only keys it does not already mention are appended, and the key that ends up in effect is the one written onto the seeded providers, so an operator's ownVLLM_API_KEY/GPU_UTIL/DFLASH_TOKENSall survive.wsl --shutdownwould take down a PostgreSQL container the install may be using.Supporting work rolled in
PROVISION_STEPS) rather than a hardcodedpull-start, and rows declareprovision: { action, run }explicitly — no implicit default that would mislabel a new row's step as a model download.installs/starts/provisions) instead of the checklist matching substrings of their names. The oldsetup.action?.includes('pull')silently skipped any runtime whose provisioning step isn't a download.provisions, so the client stops re-deriving it from the action name.services/vllmQwenManager.jsbesidemtplxServerManager/llamaServerManager, keeping the registry a table.readRuntimeWeightsskips runtimes this platform can't run — a Mac with Docker no longer sweeps for an RTX 3090 project every minute..envwrites go throughatomicWrite(it carries a bearer token).The manual sequence stays in
docs/features/qwen38-rtx3090.mdas the fallback and the explanation.Test plan
cd server && npm test— 32803 passing. The 13 failures are pre-existing on an untouchedmaincheckout: 12 inimageTo3d/trellis2NormalBake.test.js(shells out to a Python venv absent on this machine) and 1 inaskService.test.js, both verified failing onmain.cd client && npm test— the 11a11yConventions.test.jsfailures likewise reproduce on an untouchedmain.server/services/vllmQwenManager.test.js(24 cases — clone/build/prepare ordering,.envpreservation, key propagation and secrecy, WSL2 variables, the Windows placement refusal, docker-not-answering, cancellation never reaching a build),server/lib/vllmQwenProvision.test.js, plusvllmProjectSetupStateand client button cases.cd client && npm run lint— clean.Not validated end-to-end on real hardware yet — the handler and its refusals are unit-tested, but the ~9.5 GB build and ~20 GB prepare have not been exercised on the 3090 box from an empty project directory. That run is what the issue names as the unblocking step.
Closes #4767