Repository navigation
Conversation
bf532fa to
21de06d
Compare
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
21de06d to
d35b6cc
Compare
`rocm serve` protects a vLLM endpoint (public bind, --require-api-key, `rocm remote serve`) by setting VLLM_API_KEY. vLLM checks that key only under a fixed set of path prefixes (/v1 and a few others), while the server also answers routes outside them that run or expose the model, such as /invocations, /tokenize, /detokenize and /metrics. Those routes answered clients that did not present the key. A keyed vLLM launch now also passes `--middleware rocm_cli_vllm_auth.RequireEndpointApiKey`: a pure-ASGI, standard-library middleware compiled into the engine adapter, written atomically under the engine's data directory at launch and prepended to the server's PYTHONPATH (composed over the inherited, removed or explicitly set value, so the user's own PYTHONPATH and the runtime's are kept). It requires the bearer key, compared in constant time, on every HTTP and WebSocket route except GET/HEAD /health and CORS preflights, and raises instead of constructing when the key is empty, so a server that loaded it never answers a request unchecked. Being path-agnostic, it also covers routes a later vLLM adds. The module's copyright comment is exempted from the xtask ASCII-output contract, as install.sh's is; the rest of the file is ASCII. The launch command is built by one function, vllm_server_command, so a keyed launch can be asserted to carry --middleware, VLLM_API_KEY, the middleware directory ahead of the runtime's own PYTHONPATH entry and the selected GPU, and an unkeyed launch none of the key protection. Because /metrics is now covered, `rocm dash` cannot read engine metrics for a key-protected vLLM server and shows a persistent scrape warning, and `rocm bench load`, which sends no key, cannot benchmark one. The README says so, and that the every-route guarantee is the vLLM engine's: AIM runs the same vLLM server without this check, so routes such as /invocations, /tokenize and /metrics still answer there without the key. The manual-testing, testing and architecture docs describe the check; manual-testing sections after the new one are renumbered, and the three places that cited one of them by number now cite it by title. Tested by driving the shipped middleware per route under python3 with a stdlib ASGI harness (including CORS-preflight look-alikes and a WebSocket /health), by asserting the launch argv and environment, and end to end by serve-61 (`@id:serve-vllm-key-guards-every-route`), a `@nightly` GPU scenario that runs in the nightly runs on Instinct GPUs, which serve with vLLM (the MI300X job in nightly.yml and the shared GPU queue's Instinct lanes). A real keyed vLLM server refuses keyless calls on those routes, accepts keyed ones, and still answers /health. The e2e step passes the key through the environment, with a new time-bounded run helper, so it stays out of the recorded command log. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
d35b6cc to
13a9aa6
Compare
juhovainio
left a comment
There was a problem hiding this comment.
I read this diff and found nothing blocking. The every-route key check for vLLM matches what the PR body describes, the tests cover the new routes, and the diff is clean of internal references. CI is green apart from a CodeQL job that was still running when I looked. This is a diff-only read; I did not build or run anything.
Two small documentation points are left inline. Beyond those, a few things for you to decide or confirm:
- Please confirm the middleware behaves as intended on the vLLM releases we pin; I did not check that.
/metricsnow needs the key, sorocm dashloses engine metrics on keyed vLLM servers. The README says so, which seems fine./healthstays open, and AIM stays unprotected until #1007.- The manual-testing renumber leaves two
## 12.headings, inherited from two## 11.headings. - The launch wiring in
spawn_vllm_serveris covered only by the nightly GPU scenario (serve-61), so it will get no pre-merge coverage until those runners are back.
| passing `--api-key`, so the secret stays out of your shell history and the | ||
| process table. | ||
|
|
||
| With the vLLM engine, an endpoint bound to a public interface |
There was a problem hiding this comment.
Consider adding that a key-protected vLLM service that is already running keeps its old launch (without the every-route check) until it is restarted. rocm serve reuses it because the middleware is not part of the reuse match, so after upgrading, users should restart keyed vLLM services.
There was a problem hiding this comment.
Agreed that it's real: managed_service_reuse_refusal in apps/rocm/src/service_launch/launch_transaction.rs matches on the recipe, requires_api_key and device policy, so a keyed vLLM service started before the upgrade is reused without the middleware until it is restarted. I've left the README unchanged in this PR so the approved diff stays as reviewed; the PR description carries it as an upgrade note ("Restart key-protected vLLM services after upgrading") for the release notes. The latest push (fb7fa46) only merges main and changes no behaviour.
| /// primitive only carries path-valued env overrides, so the key never reaches it. | ||
| /// vLLM enforces auth on every platform (`VLLM_API_KEY`), and loopback binds | ||
| /// (`public_bind == false`) need no key — both pass. `is_windows` is a parameter | ||
| /// The vllm engine enforces auth wherever it runs (`VLLM_API_KEY` plus the |
There was a problem hiding this comment.
"wherever it runs" conflicts with docs/testing.md, which now says Linux and WSL. vLLM serving is Linux/WSL-only, so I'd say that here (this check exempts vllm on every platform, and the middleware tests are cfg(linux)).
There was a problem hiding this comment.
Not changed. The sentence names the vllm engine, and that engine only runs on Linux and WSL: engines/vllm/src/runtime.rs refuses native Windows, and docs/testing.md says "vLLM (Linux and WSL, ...)" in the same paragraph. "Wherever it runs" is true as written; this function exempts vllm on every platform because there is no platform where it starts without the key check. fb7fa46 (merge of main) leaves it as is.
juhovainio
left a comment
There was a problem hiding this comment.
come# Review — no blocking findings
Full review of the whole change (base 9e50eac5a03df41a2f23e73dac3a7ec470ea78a7 to head 13a9aa617dd757558866c758d956763a6ae420f2).
No ticket named as fixed/closed by this change; the description references issue #1007 as an explicit out-of-scope follow-up (extending the same protection to the AIM engine), and the diff does not touch engines/aim/, consistent with that scoping.
Blocking
None
Non-blocking
- Already-running key-protected vLLM services silently keep their pre-upgrade launch (no every-route middleware), and nothing shipped tells an operator to restart them —
apps/rocm/src/service_launch/launch_transaction.rs:329,README.md:1316
rocm serve's reuse check isif require_api_key && !existing.requires_api_key— a bare boolean "did the existing record ask for a key," with no awareness of whether that process has this PR's middleware. README.md, docs/testing.md and docs/manual-testing.md all document the general reuse-matching rule but never mention restarting a keyed vLLM service after upgrading rocm-cli; only the PR description's own "Upgrade note (not in the docs)" states it. Confidence 85 · architectural · Fix: add one sentence near README.md:1316 (or the new "vLLM Endpoint Key Coverage" doc section) telling operators to restart already-running key-protected vLLM services after upgrading. - A doc comment says the vLLM engine "enforces auth wherever it runs," overstating against this PR's own Linux/WSL-only scoping elsewhere —
apps/rocm/src/service_launch/endpoint_auth.rs:101
docs/testing.md's new text and the middleware's own Rust test (#[cfg_attr(not(target_os = "linux"), ignore)]) both scope this to Linux/WSL. Not functionally wrong — vLLM is already rejected on native Windows elsewhere (engines/vllm/src/runtime.rs:176) — just inconsistent wording. Confidence 80 · mechanical · Fix: say "wherever it runs (Linux and WSL)". _exempt'sscope["type"] != "http"guard has no test proving it necessary —engines/vllm/src/rocm_cli_vllm_auth.py:96
For a websocket scope,_is_cors_preflightand theUNAUTHENTICATED_ROUTESmethod check already fail closed (a websocket scope carries no"method"key), so removing this line changes no real ASGI behavior; the Rust test harness's own comment acknowledges this exact gap. Not a bypass, just redundant defense-in-depth no test independently exercises. Confidence 75 · mechanical · Fix: drop the now-provably-redundant line, or comment that it's deliberate belt-and-suspenders.
Decisions for the author
- Whether
rocm serve's reuse check should become middleware-aware (forcing a restart) instead of relying on a manual one — tradeoff. Auto-restarting a live inference server the moment a keyed relaunch is requested is invasive (drops in-flight requests, defeats "reuse" for availability) and arguably riskier than documenting the manual step; the current choice is reasonable, but worth confirming deliberate given the finding above.
Positive signals
_digest()hashes both sides to a fixed-length SHA-256 digest beforehmac.compare_digest, deliberately removing the timing leak from comparing variable-length secrets directly.- The Rust test drives the actual shipped file content (via
include_str!) under a realpython3subprocess rather than a reimplemented mock — verified this myself by extracting the file and running an equivalent harness directly; every claimed behavior held (every non-exempt route 401s without the key, the CORS-preflight exemption requires both headers plusOPTIONS, the health exemption is GET/HEAD-only and path-exact, a WebSocket closes before the handshake). vllm_server_commandconsolidates middleware/env/PYTHONPATH wiring into one function with both a keyed and an unkeyed test asserting presence and absence together — exactly the shape that catches a one-sided mutation (e.g. add--middlewarebut forgetVLLM_API_KEY).- The e2e step assertions (
assert_other_routes_refused, etc.) panic loudly on a request/connection failure rather than silently miscounting it as a 401, so the nightly scenario cannot pass vacuously if the server never came up.
Deployment notes
/metricsnow requires the key, sorocm dashcannot read engine metrics from a key-protected vLLM server (shows a persistentvllm scrape: ... 401warning); documented in README.rocm bench loadsends no API key and so cannot benchmark a key-protected endpoint (pre-existing limitation, now documented).- A key-protected vLLM service already running before an upgrade keeps answering without the every-route check until restarted (see the first non-blocking finding above).
What this covered
Read completely: engines/vllm/src/rocm_cli_vllm_auth.py and engines/vllm/src/endpoint_auth.rs (by an independent fan-out worker, and separately by me directly, including extracting and running the shipped middleware under python3); engines/vllm/src/process.rs, engines/vllm/src/lib.rs, apps/rocm/src/service_launch/endpoint_auth.rs, xtask/src/ascii_output_contract.rs, all five changed tests/e2e-cucumber files, and README.md/docs/architecture.md/docs/manual-testing.md/docs/testing.md in full, plus surrounding code for impact analysis (apps/rocm/src/service_launch/launch_transaction.rs, spawn.rs, crates/rocm-dash-daemon/src/runner.rs). Commits 9e50eac5a03df41a2f23e73dac3a7ec470ea78a7…13a9aa617dd757558866c758d956763a6ae420f2.
Fan-out: one of four planned independent workers completed (the middleware + Rust-harness group). The other three groups (process.rs/service_launch/xtask; the e2e-cucumber group; the docs group) could not be spawned after two attempts each — concurrent-subagent capacity was exhausted by unrelated concurrent work for the whole run — so I reviewed those three groups myself directly instead of dropping them; they are not independent passes, though every file in them was read in full. Agent-instruction adherence and cross-file consistency were folded into those direct passes (checked against this repo's doc-sync, architecture-doc, and commit-message rules) rather than run as separate dedicated passes; a "prior changes on these files" pass found nothing to check (the touched files are new or have no relevant prior review history). CI was confirmed via the GitHub API: all checks on the current head commit are green (CodeQL's "neutral" conclusion is its normal non-finding state).
Reconciled, in part: after finishing the above, I read the one existing submitted review (APPROVE) and its two inline comments. Both of its points — the undocumented restart-after-upgrade gap, and the "wherever it runs" wording — match two of my findings above, which I had reached independently before reading them, and verified further against the code myself. No other discussion existed to reconcile against.
🤖 Automated by an AI agent
| passing `--api-key`, so the secret stays out of your shell history and the | ||
| process table. | ||
|
|
||
| With the vLLM engine, an endpoint bound to a public interface |
There was a problem hiding this comment.
Non-blocking: a key-protected vLLM service already running before an upgrade keeps its old launch (no every-route middleware) until it is restarted — rocm serve's reuse check (apps/rocm/src/service_launch/launch_transaction.rs:329, if require_api_key && !existing.requires_api_key) is a bare boolean "did the existing record ask for a key" test with no awareness of whether that process actually has this middleware. Nothing in README.md, docs/testing.md or docs/manual-testing.md tells an operator to restart a keyed vLLM service after upgrading — only the PR description's own "Upgrade note (not in the docs)" mentions it. Suggest adding one sentence here (or in the new "vLLM Endpoint Key Coverage" manual-testing section) telling operators to restart already-running key-protected vLLM services after upgrading.
There was a problem hiding this comment.
Same point as the thread above, and the same answer: confirmed against managed_service_reuse_refusal, kept out of this PR's docs so the approved diff is unchanged, and stated as an upgrade note in the PR description. fb7fa46 is a merge of main only.
| /// primitive only carries path-valued env overrides, so the key never reaches it. | ||
| /// vLLM enforces auth on every platform (`VLLM_API_KEY`), and loopback binds | ||
| /// (`public_bind == false`) need no key — both pass. `is_windows` is a parameter | ||
| /// The vllm engine enforces auth wherever it runs (`VLLM_API_KEY` plus the |
There was a problem hiding this comment.
Non-blocking: "enforces auth wherever it runs" overstates against this change's own Linux/WSL-only scoping elsewhere (docs/testing.md's new text, and the #[cfg_attr(not(target_os = "linux"), ignore)] comment on the middleware's Rust test). Not functionally wrong — vLLM is already rejected on native Windows elsewhere (engines/vllm/src/runtime.rs:176), so there's no real gap — but the wording is inconsistent with the rest of the PR. Suggest "wherever it runs (Linux and WSL)" to match.
There was a problem hiding this comment.
Same as the thread above: left as is, because the vllm engine runs only on Linux and WSL (engines/vllm/src/runtime.rs rejects native Windows), so "wherever it runs" already means that. No change in fb7fa46, which only merges main.
| return b"origin" in names and b"access-control-request-method" in names | ||
|
|
||
| def _exempt(self, scope): | ||
| if scope["type"] != "http": |
There was a problem hiding this comment.
Non-blocking (test-coverage nitpick): this scope["type"] != "http" guard isn't independently exercised by any mutation — for a websocket scope, _is_cors_preflight and the UNAUTHENTICATED_ROUTES method lookup already fail closed (a websocket scope carries no "method" key), so removing this line changes no real ASGI behavior. The test harness's own comment in engines/vllm/src/endpoint_auth.rs acknowledges this exact gap. Not a bypass — just redundant defense-in-depth that no test proves necessary. Either drop it or leave a comment noting it's deliberate belt-and-suspenders rather than load-bearing.
There was a problem hiding this comment.
Correct, it isn't load-bearing today: a websocket scope has no method, so the /health and CORS-preflight checks already fail closed without it. It's deliberate belt-and-braces, so _exempt stays HTTP-only even if a future ASGI server puts a method on websocket scopes. The harness comment in engines/vllm/src/endpoint_auth.rs records that this guard is equivalent under current behaviour. Not changed in this PR; fb7fa46 only merges main.
Brings the approved branch current with main without rewriting it. Both sides added a section to docs/testing.md at the same place; both are kept. Main's new manual-testing section on `rocm fix` steps that do not finish is renumbered 17 -> 18, after this branch's inserted section 7 shifted the ones before it. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Review — no blocking findings
Full review of the whole change (base bfff5216b50ba997e380a90daf3d356dfb44e3f0 to head fb7fa46eb33ab58ec01ab26d16a3fc4b080e6e58).
No ticket closed by this change. #1007 is a draft follow-up pull request (not a plain issue) that depends on this PR's branch and extends the same every-route key middleware to the AIM engine — the description's own claim that AIM is out of scope here, tracked separately, is accurate.
Blocking
None
Non-blocking
- The manual-testing renumber carries forward a pre-existing duplicate-heading bug instead of fixing it —
docs/manual-testing.md:1770,docs/manual-testing.md:1835
The base branch already has two## 11.headings ("AIM (Docker) Engine Verification" and "Background Automation Helper"); inserting the new section 7 here shifts both to## 12., still duplicated. The diff directly edits both exact lines already, so fixing the duplication would cost nothing extra. Confidence 85 · mechanical · Fix: renumber "Background Automation Helper" (and everything after it) to## 13.onward.
Decisions for the author
- Three points a reviewer already raised inline, which the author declined to change — tradeoff, confirmed independently rather than re-litigated:
- An already-running key-protected vLLM service keeps its pre-upgrade launch (no every-route middleware) until restarted, since
rocm serve's reuse check (managed_service_reuse_refusalinapps/rocm/src/service_launch/launch_transaction.rs) matches onrequires_api_key, not on whether the middleware is installed, and none of README.md/docs/testing.md/docs/manual-testing.md state the restart requirement (only the PR description's "Upgrade note" does). Traced the reuse-check myself and confirm the gap is real; the author's choice to track it as a release note rather than expand this PR's own diff is reasonable. - The
ensure_public_bind_engine_supporteddoc comment "the vllm engine enforces auth wherever it runs" reads as a platform-universal claim next to docs/testing.md's explicit "Linux and WSL" wording for the same fact. Confirmedengines/vllm/src/runtime.rsrefuses native Windows unconditionally, so the sentence is true as written (vLLM never runs anywhere the check wouldn't apply) even though the phrasing invites misreading. rocm_cli_vllm_auth.py's_exempthas ascope["type"] != "http"guard before the CORS/health checks; traced it myself and confirmed a websocket scope has no"method"key, so_is_cors_preflightand theUNAUTHENTICATED_ROUTESlookup already fail closed without this guard — it is deliberate, provably-redundant defense-in-depth today, not a live gap.
- An already-running key-protected vLLM service keeps its pre-upgrade launch (no every-route middleware) until restarted, since
Positive signals
_digest()hashes both sides to a fixed-length SHA-256 digest beforehmac.compare_digest, removing the timing leak from comparing variable-length secrets directly — traced by hand and confirmed by an independent pass.write_modulestages into aNamedTempFilein the target directory andpersists it, so a concurrent launch can never observe a half-written middleware module.- The Rust test harness drives the actual shipped file content (
include_str!) under a realpython3subprocess, not a reimplemented mock, and covers per-route 401s,root_pathstripping edge cases, CORS-preflight look-alikes, and WebSocket closure before handshake. vllm_server_commandconsolidates the middleware/env/PYTHONPATH wiring into one function with both a keyed and an unkeyed test asserting presence and absence of all three protections together — the shape that catches a one-sided mutation.- The new e2e step assertions (
assert_other_routes_refused, etc.) panic loudly on a request/connection failure rather than silently miscounting it as a 401, so the scenario cannot pass vacuously if the server never came up.
Deployment notes
/metricsnow requires the key, sorocm dashcannot read engine metrics from a key-protected vLLM server (documented in README).rocm bench loadsends no API key and so cannot benchmark a key-protected endpoint (pre-existing limitation, now documented).- A key-protected vLLM service already running before an upgrade keeps answering without the every-route check until restarted (see the first "Decisions for the author" item above; not in the shipped docs).
What this covered
Read completely: engines/vllm/src/rocm_cli_vllm_auth.py, engines/vllm/src/endpoint_auth.rs and engines/vllm/src/lib.rs (by an independent fan-out worker, and separately by me directly, tracing the auth/exemption logic by hand given this change's security sensitivity); engines/vllm/src/process.rs, apps/rocm/src/service_launch/endpoint_auth.rs and xtask/src/ascii_output_contract.rs (by a second independent worker, plus apps/rocm/src/service_launch/launch_transaction.rs and engines/vllm/src/runtime.rs by me for impact analysis); all five changed tests/e2e-cucumber files (by a third independent worker); README.md, docs/architecture.md, docs/manual-testing.md and docs/testing.md in full, by me directly. Commits bfff5216…fb7fa46e.
Fan-out: four groups were planned; three ran as independent workers. The fourth (docs) failed to spawn twice on a concurrent-subagent limit, so per the protocol for a worker that never returns, I read and judged that group myself directly instead of dropping it — it is not an independent pass. Agent-instruction adherence was checked against this repo's OSS-contribution AGENTS.md (user-observable behaviour needs a Gherkin scenario; a new test is shown to fail against the code it covers; platform-gated code gets a test on its gate) and is satisfied by the new serve-61 scenario and the Rust harness that runs the shipped module for real. History and prior-changes passes did not run: these files are new or have no prior review history to check.
CI was confirmed via the GitHub API: all 40 checks on the current head are success, neutral (CodeQL) or an intentionally-skipped job.
A process note on blindness: Step 0's mandatory check for a concurrent claim required reading the PR's existing reviews before this pass began, which exposed their text early. Every finding above was independently re-derived and verified against the source (traced by hand or by an independent worker) rather than imported from that text, and the three "Decisions for the author" items were cross-checked against the author's own replies to those threads, which this review's independent tracing confirms are accurate.
🤖 Automated by an AI agent
| back to another engine. | ||
|
|
||
| ## 11. Background Automation Helper | ||
| ## 12. Background Automation Helper |
There was a problem hiding this comment.
Non-blocking: this renumber (inserting the new section 7 and shifting everything after it by one) carries forward a pre-existing duplicate-number bug rather than fixing it. The base branch already had two ## 11. headings ("AIM (Docker) Engine Verification" and "Background Automation Helper"); after this diff they are both ## 12. (line 1770 and this line, 1835). Since the diff directly edits both of these exact headings to renumber them, this would be a one-line fix to take while already touching the lines: bump "Background Automation Helper" (and everything after it) to ## 13. onward.
Why
rocm serveprotects a vLLM endpoint with an API key when it binds publicly (--allow-public-bind), when--require-api-keyis given, and underrocm remote serve. It did this only by settingVLLM_API_KEY. vLLM checks that key only under a fixed set of path prefixes (/v1and a few others), while the same server also answers routes outside them that run or expose the model, such as/invocations,/tokenize,/detokenizeand/metrics. Those routes answered clients that did not present the key, so a key-protected endpoint was only partly protected.What changes
A keyed vLLM launch now also passes
--middleware rocm_cli_vllm_auth.RequireEndpointApiKey.engines/vllm/src/rocm_cli_vllm_auth.py) uses only the Python standard library and works at the raw ASGI layer, so it does not depend on the Starlette version a given vLLM release ships. It requiresAuthorization: Bearer <key>, compared in constant time, on every HTTP and WebSocket route. The only exceptions areGET/HEAD /health(the liveness probe) and real CORS preflights. Because it does not look at path prefixes, it also covers routes a later vLLM release adds.VLLM_API_KEYvLLM's own check reads. The middleware raises instead of constructing when that key is empty, so a server that loaded it never answers a request unchecked. The key still never reaches argv.include_str!) and written atomically under the engine's data directory at launch. That directory is prepended to the server'sPYTHONPATH, and anyPYTHONPATHthe runtime set or the user inherited is kept after it. The launch command is now built by one function,vllm_server_command, so the keyed and unkeyed launches can be asserted directly.--middlewareargument, no module onPYTHONPATH.#copyright line is added to the xtask ASCII-output exemptions, asinstall.sh's is, because a Python comment is never printed. The rest of the file is ASCII.User-visible side effects, documented in the README:
/metricsnow needs the key too, sorocm dashcannot read engine metrics from a key-protected vLLM server. It leaves that server's KV-cache, queue, throughput and latency figures empty and keeps showing avllm scrape: ... status 401 Unauthorizedwarning.rocm bench loadsends no API key, so it cannot benchmark a key-protected endpoint. That predates this change; the README now says so./invocations,/tokenizeand/metricswithout the key (see Scope).Upgrade note (not in the docs): a key-protected vLLM service that is already running keeps its old launch, without the middleware, until it is restarted.
rocm servereuses a running service whose recipe and key requirement match, and the middleware is part of neither. Restart key-protected vLLM services after upgrading.Docs: README,
docs/testing.md,docs/manual-testing.md(new section 7, "vLLM Endpoint Key Coverage"; later sections renumbered, and the three places that cited the "Pasting An Error Intorocm diagnose" section by number now cite it by title) anddocs/architecture.md(entry forendpoint_auth.rs). The doc comment onensure_public_bind_engine_supportednow names the middleware.Tests
engines/vllm/src/endpoint_auth.rs): the shipped module runs underpython3with a standard-library ASGI harness./invocations,/tokenize,/detokenize,/metrics,/ping,/v1/...and an unknown route all return 401.GET/HEAD /health(exact path, after strippingroot_path, which is stripped only from a path that starts with it) and real CORS preflights pass without the key. Preflight look-alikes are refused:Originonly,Access-Control-Request-Methodonly, and both headers on aPOST. A WebSocket to/healthis closed, and WebSockets without the key are closed before the handshake.PYTHONPATHis composed (inherited, removed or set by the command), that the module file is written and replaced atomically, and that the import path matches the shipped class.engines/vllm/src/process.rs): a keyed launch gets--middleware,VLLM_API_KEY, the middleware directory ahead of the runtime's ownPYTHONPATHentry, and the selected GPU. An unkeyed launch on the same runtime gets none of the key protection: no--middleware, noVLLM_API_KEY, and the runtime's ownPYTHONPATHunchanged.--enforce-eageris forwarded either way, and the key never appears in argv.serve-61(@id:serve-vllm-key-guards-every-route) inmodel_serving.feature, tagged@requires-gpu @requires-engine:vllm @nightly. A real vLLM server started with--require-api-keyrefuses keyless calls to/invocations,/tokenize,/detokenizeand/metricswhile still answering/health, and accepts all of those calls when the client presents the key. The step passes the key throughROCM_SERVE_API_KEY(a newrun_rocm_bounded_with_envharness helper keeps the serve call time-bounded), so the key stays out of the recorded command log.nightly.ymland the shared GPU queue's Instinct lanes (MI350P among them). It is not@merge-queue, because no merge-queue lane runs vLLM (the MI300X lane does not run onmerge_group). A failure on the nightly lanes does not block merges.gh workflow run e2e-selfhosted.yml --ref <branch> -f platform=app-dev-gpu -f name_filter='^serve-61 '. Aname_filterdispatch bypasses the harness's tag gates, soinclude_nightlyis not needed.Breaking the fix turns the tests red:
/v1paths fails the harness test on every listed route;--middlewareargument, movingprotect_endpointahead of the runtime environment (so the runtime'sPYTHONPATHreplaces the middleware directory), or skipping GPU selection fails the keyed launch-command test, and prepending anyPYTHONPATHentry to an unkeyed launch fails the unkeyed one;root_pathfrom a path that does not start with it, fails the harness.Not unit-tested: the call in
spawn_vllm_serverthat hands the resolved key tovllm_server_command, because that spawn path probes the host (OpenMPI, libatomic, libnuma) before building the command. Only the GPU scenario covers it.Verification (Linux, Strix Halo host with no vLLM lane)
cargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warnings: clean.cargo test -p rocm-engine-vllm --lib: 183 passed.cargo test --workspace --all-targets: 2336 passed, 1 failed in code this change does not touch:service_control::tests::stop_of_a_container_engine_is_unconfirmed_when_the_engine_stop_fails, because this host'sdockeris podman's emulation, which ignores the test'sDOCKER_HOST.cargo test -p e2e-cucumber --lib --test feature_naming --test skill_referenceandcargo xtask check-architecture-doc: pass.cargo xtask smoke: every step passes up torocmd sandbox examine, which needs a bubblewrap network namespace this host does not permit.Not verified:
serve-vllm-key-guards-every-route: this host has no vLLM lane.--middlewarebehaviour (the class is mounted withapp.add_middleware).Scope
The AIM engine also runs vLLM's server inside its container and sets only
VLLM_API_KEY. This change does not cover it; #1007 applies the same rule to AIM.tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows.docs/architecture.md.docs/architecture.mdhas the matching entry, and the prose of any entry whose file changed role still describes it.