Repository navigation
Conversation
…tic ROCm claim ROCMAI-550: on a GPU with no ROCm/HIP build (e.g. gfx90c), `rocm examine` and `rocm engines` kept claiming the embedded Lemonade engine runs "ROCm llama.cpp backend" even though it actually falls back to Vulkan. `rocm install sdk`'s refusal for such a GPU didn't name the actual next step, and neither `rocm serve`'s deployment summary nor `rocm services list` showed which backend/ device an engine actually selected. - add `GpuSupportLevel` (RocmHip/VulkanOnly/Unknown) in rocm-core, derived from the existing `normalize_therock_family`/`gfx_target_has_no_rocm_build` primitives - make the Lemonade engine note in `rocm examine` and `rocm engines` GPU-aware instead of a hardcoded ROCm claim - `rocm install sdk`'s unsupported-GPU errors now suggest `rocm serve <model>` - surface the engine's actual backend/device (read live via the existing `Healthcheck` RPC, never guessed) in `rocm serve`'s deployment summary and `rocm services list` Most of the ticket's `rocm examine`/`install sdk` complaints for gfx90c were already fixed by #648 (ROCMAI-542); this covers what was left. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
CodeQL flagged two assertions as cleartext logging of `api_key`. Those two use `base_summary()`, whose key is `None`, so the literal instances are false positives — but the pattern is not. `render_summary` prints the key by design, and `api_key_client_config_shown_only_when_present` sets a real one, so a failure there copied `endpoint-secret` into CI logs, which are retained and widely readable. Assert on the real text, print a redacted copy. Covers all twelve sites that interpolated a rendered summary, not just the two CodeQL picked. Production output is unchanged: the key is still shown once, and the test pinning that still asserts the exact string. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
|
CodeQL's two What CodeQL flagged ( What it was actually pointing at. Fix: assert on the real text, print a redacted copy. A small test-only Production output is untouched: the key is still shown once, and the test that pins the exact string still asserts Verified on Linux at |
There was a problem hiding this comment.
🟡 Changes recommended
Unknown GPUs remain misreported, authenticated Lemonade devices are omitted, and required observable-output coverage is incomplete.
7 open findings
Unknown GPU status incorrectly claims ROCm backend · New Healthcheck omits API key, hiding device status · New Remediation may launch vLLM instead of Lemonade/Vulkan · New Missing Gherkin coverage for backend wording changes · New Missing detected-host remediation scenario coverage · New Scenarios omit deployment summary device-row assertion · New Scenario does not validate the reported backend device · New
What changed in this PR
Updates GPU/backend reporting to reflect detected support and live engine state.
Changes:
- Adds GPU support classification and dynamic Lemonade messaging.
- Reports live backend/device details in serving output.
- Improves unsupported-SDK guidance and test coverage.
| File | Description |
|---|---|
crates/rocm-core/src/host_gpu.rs |
Adds GPU support tiers. |
crates/rocm-core/src/lib.rs |
Exports support APIs. |
apps/rocm/src/main.rs |
Adds dynamic notes and live device reporting. |
apps/rocm/src/serve_cmd.rs |
Fetches devices for deployment summaries. |
apps/rocm/src/serve_summary.rs |
Renders the device row. |
apps/rocm/src/therock.rs |
Adds serving remediation guidance. |
apps/rocm/src/providers.rs |
Updates summary construction. |
tests/e2e-cucumber/features/model_serving.feature |
Extends GPU serving scenarios. |
tests/e2e-cucumber/tests/e2e/serving_steps.rs |
Adds device-reporting assertion. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| match level { | ||
| rocm_core::GpuSupportLevel::VulkanOnly => { | ||
| "default embedded Lemonade server; this GPU has no ROCm/HIP support, so it runs on the Vulkan llama.cpp backend" | ||
| } | ||
| _ => "default embedded Lemonade server with ROCm llama.cpp backend", | ||
| } |
There was a problem hiding this comment.
You were right and this is now fixed in f54a8d2f. The earlier round added engines_inventory_does_not_claim_vulkan_for_an_unrecognized_gpu and its two siblings, which pinned that Unknown must not claim Vulkan — but they said nothing about the ROCm side, and the _ => catch-all kept printing "with ROCm llama.cpp backend". That is exactly the stale claim this PR exists to remove, surviving on precisely the hosts where the CLI knows least.
lemonade_engine_note (apps/rocm/src/main.rs:2748) now matches GpuSupportLevel exhaustively, with no catch-all, so a future variant has to be classified rather than silently inheriting the ROCm wording:
VulkanOnly— confirmed no ROCm build, names the Vulkan backend.RocmHip— keeps the ROCm wording.Unknown— claims no backend: "default embedded Lemonade server; runrocm examineto see which llama.cpp backend this GPU uses".
Not the Vulkan wording for Unknown either, for the reason the type exists: a detection gap is not a confirmed absence. Pointing at rocm examine keeps the row actionable, and matches what unsupported_requested_target_message already does on the other branch where the host is unknown.
The two Unknown tests now assert all three facts — no Vulkan claim, no ROCm claim, and a next step — at apps/rocm/src/examine_report_text.rs:2608 and apps/rocm/src/main.rs (codex_bridge_engine_inventory_does_not_claim_vulkan_for_an_unrecognized_gpu). Falsified on the Linux box by restoring the catch-all: both go red.
| let response: HealthcheckResponse = engine_request( | ||
| Some(paths), | ||
| engine, | ||
| EngineMethod::Healthcheck, | ||
| &HealthcheckRequest { | ||
| service_id: service_id.to_owned(), | ||
| }, | ||
| ) |
There was a problem hiding this comment.
Verified against the current code and fixed in f54a8d2f — this was still live, not stale.
Confirmed the mechanism you described: engines/lemonade/src/state.rs::inference_verified passes rocm_engine_protocol::resolve_endpoint_api_key() to its probe, which reads ROCM_SERVE_API_KEY_FILE from the environment. fetch_service_device supplied only AppPaths, so for a public or --require-api-key service the engine's own /v1/models probe answered 401, listed/ready came back false, the device degraded to "unknown", and the row vanished for exactly the services that are protected.
Fix: the key file is threaded per call, since the key is per service id.
engine_request_with_scoped_env(apps/rocm/src/engine_ipc.rs:152) takes anendpoint_key_filebeside the existingenv_root. For the built-in path it is scoped bywith_scoped_builtin_engine_envinside the sameBUILTIN_ENGINE_ENV_LOCKcritical section as the other vars, so the env-mutation contract (xtask/src/env_mutation_contract.rs) still holds; for an external plugin it is set on the childCommand.fetch_service_device(:305) passesendpoint_keys::endpoint_key_file_if_present(paths, service_id)—Nonefor a loopback service, which has no key file and needs none.engine_request_with_env_rootdelegates, so no existing call site changed.
Your pointer to the restart path was the right precedent: services_text.rs already re-threads the same file onto the engine child it spawns.
Not verified against a live protected server — the dev box has no usable GPU telemetry, so there is no managed service to probe there. This is wiring verified by compilation and the existing gates, not an observed 401 turning into a device row.
| the Vulkan backend can still be used. \ | ||
| Try `rocm serve <model>` — it runs on Lemonade's Vulkan backend instead." |
There was a problem hiding this comment.
Confirmed and fixed in f54a8d2f. @juhovainio and @nowycondro reached the same defect from the configured-default_engine side; your recipe-selection angle is the other half of it, and the fix covers both.
unsupported_host_gpu_message (apps/rocm/src/therock.rs:3607) now pins the engine, which is the first of your two suggestions:
Try
rocm serve <model> --engine lemonade- it runs on Lemonade's Vulkan backend instead.
I did not take the second (describe backend selection conditionally). Conditioning means resolve_family reading RocmCliConfig and the model recipe to say which engine would win — a config load and a recipe resolution added to an install-path error message, to produce a sentence that is still only correct for the inputs it happened to check. --engine out-ranks every source in select_serve_engine, so pinning makes the claim unconditional instead of conditional, which is the stronger guarantee for less machinery.
One correction to your description, for the record: a recipe's preferred_engines is ranked below the host-GPU preference in select_serve_engine, not above it — the override that genuinely out-ranks the host is the configured default_engine. Either way the bare command could not be promised, so the conclusion stands.
Test: unsupported_gpu_error_pins_the_engine_a_configured_default_could_override (therock.rs:8399), which asserts the pinned form and the absence of the bare one still carrying the promise — necessary because the pinned form contains the bare one as a prefix.
Your note about line 3479 refers to unsupported_requested_target_message; that one was already fixed in 16ccfb3a and now points at rocm examine instead of predicting a backend.
| /// The lemonade engine note shown in `rocm examine` and `rocm engines`, adjusted | ||
| /// for whether this host's detected GPU can actually use the ROCm llama.cpp | ||
| /// backend. Both call sites have a live `GpuSupportLevel` in scope; other |
There was a problem hiding this comment.
Judging this one partly answered and partly not reachable, with the reasons below rather than a silent skip.
Answered. The two notes are no longer unit-only in the sense you meant: engines_inventory_* (apps/rocm/src/examine_report_text.rs:2550-2628), examine_engine_inventory_* and codex_bridge_engine_inventory_* / engines_mcp_tool_* now drive the real renderers for all three support levels, including the Unknown expectation you asked for — which this push strengthened to assert that Unknown claims no backend rather than merely not claiming Vulkan (see the main.rs:2837 thread).
Not reachable as a scenario, and this is the honest part. A Gherkin assertion on the ROCm-versus-Vulkan wording needs a host whose GPU is detected and confirmed to have no ROCm build — a Renoir/Cezanne Vega APU. detect_host_gpu_summary reads real sysfs/WMI with no test override, and every lane reports a ROCm-capable GPU (platform=mi300x, Strix Halo, W7900). A scenario written for the Vulkan branch would be permanently skipped on every lane, which reads greener than it is.
What a lane can assert is the negative, and that is now pinned: runtime-19 checks that the --family gfx90c refusal points at rocm examine and does not contain rocm serve, on a runner that is ROCm-capable. Proving the absence of a false prediction is the half a lane can actually reach.
The device: row — the other observable this PR adds — is now asserted end-to-end, including its value and agreement between rocm serve and rocm services list; see the model_serving.feature and serving_steps.rs threads.
| assert!(message.contains("no ROCm SDK build"), "{message}"); | ||
| assert!(message.contains("`gfx90c`"), "{message}"); | ||
| assert!(message.contains("Vulkan"), "{message}"); | ||
| assert!(message.contains("rocm serve <model>"), "{message}"); |
There was a problem hiding this comment.
Standing by the existing answer here rather than adding a scenario, and stating the reason explicitly rather than leaving it in another thread: the full write-up is in the PR description section "Why the rocm install sdk remediation sentence is unit-covered and not a scenario", and in the reply to @juhovainio's thread on the same hunk.
Short version: unsupported_host_gpu_message is only reachable from resolve_family after the --family, ROCM_CLI_THEROCK_FAMILY and managed-runtime doors are all closed, and then only when detect_host_gfx_target() returns a target normalize_therock_family does not cover. That needs a real Renoir/Cezanne Vega APU on the runner, and detect_host_gfx_target reads sysfs/WMI with no test override. runtime-19 drives the --family branch precisely because that is the one door a lane can open — you are right that it does not prove the detected-host path, and nothing in CI can.
What did change in f54a8d2f is the remediation itself: it now names the engine, rocm serve <model> --engine lemonade, because a configured default_engine could otherwise make the sentence false (see @nowycondro's and @juhovainio's threads). The two unit tests you point at are joined by a third, unsupported_gpu_error_pins_the_engine_a_configured_default_could_override (apps/rocm/src/therock.rs:8399), which pins the engine-pinned form and the absence of the bare one — the bare command is a prefix of the pinned one, so a presence-only assertion could not tell them apart.
| When the user sends a chat completion request | ||
| Then the response contains a model reply | ||
| And the response identifies the correct model | ||
| And the services list reports the engine's actual backend device |
There was a problem hiding this comment.
Correct, and fixed in f54a8d2f. The scenarios ran rocm serve in a Given and then asserted only on a later services list, so the deployment summary's own device: row — half of what this PR adds — had no live assertion anywhere.
Two changes:
E2eWorldgainsserve_summary(tests/e2e-cucumber/tests/e2e.rs), set byserve_and_waitandserve_gpu_modelon the successful attempt. Deliberately notcli_output: that field belongs to each scenario's When step, and overwriting it from a Given would change what every existing Then on these scenarios reads.assert_services_list_reports_device(serving_steps.rs:1491) now cross-checks the two surfaces: if the summary printed adevice:row it must equal the oneservices listreports (they read the sameHealthcheckRPC), and if it did not, the summary must not have saidready— so the row's absence has to be explained by a launch that was still starting, not by the row being dropped.
The assertion is two-sided rather than unconditional on purpose. rocm serve only asks the engine once the launch already reported ready, and a managed launch can legitimately return while the weights are still loading — which is exactly why the Given then polls /v1/models. An unconditional "summary must contain device:" would have been flaky on a slow cold start rather than a real check.
Caveat on where this actually runs: serve-07 is xfail'd on lemonade+linux (EAI-7423), so on that lane no step of it is a live assertion. serve-05 (the vLLM canary) is what exercises this on every PR.
| assert!( | ||
| block | ||
| .lines() | ||
| .any(|line| line.trim_start().starts_with("device:")), |
There was a problem hiding this comment.
Fair, and fixed in f54a8d2f. The step checked only that a line starts with device:, so an empty, hardcoded or wrong value all passed.
assert_services_list_reports_device (tests/e2e-cucumber/tests/e2e/serving_steps.rs:1491) now parses the value and holds it to three independent facts:
- Shape.
parse_device_row(:1453) requires<backend> gpuor<backend> gpu <n>with nothing trailing and a numeric ordinal — the shape every engine reports. An empty or malformed value fails here. - Backend against the engine that served. The step reads the
engine:row from the same services-list block and checks the backend against it:lemonademay reportrocmorvulkan;vllmandaimare ROCm-only, sovulkanfrom either means the value did not come from the engine that served.unknownfails for all three — it is the engines' own "no state for this", whichfetch_service_deviceis supposed to drop rather than print. That is yourrocm gpu …vsvulkan gpu …distinction, enforced rather than assumed. - Agreement with the serve summary — see the thread on
model_serving.feature.
Note the shape check only became meaningful with the engine fix in this same push: engines/vllm used to report the literal "rocm_gpu", which parse_device_row rejects. It now reports rocm gpu <n> like Lemonade.
What I could not do is assert the expected backend for a given host — that needs a Vulkan-only GPU, and no lane has one. The check is "consistent with what this engine can select", not "equal to what this host must select".
juhovainio
left a comment
There was a problem hiding this comment.
Reviewed this with a review-code fan-out pass plus a separate Standards/Spec pass against Jira ROCMAI-550. The core design is good: GpuSupportLevel is derived from existing detection primitives rather than adding new probing, the device: row is read live from each engine's own Healthcheck RPC rather than guessed from the launch-time GPU plan, and the PR description is refreshingly honest about what it couldn't verify end-to-end (both GPU canaries fail on this box for pre-existing, unrelated environment reasons, clearly disclosed) and what it checked by hand instead.
Requesting changes on one gap: the rocm install sdk remediation text added in therock.rs (unsupported_host_gpu_message/unsupported_requested_target_message now suggest rocm serve <model> as the next step) is a second user-observable CLI text change in this same PR, but only the device: row change got a Gherkin scenario. AGENTS.md §3 asks for a scenario or an explicit justification for user-observable output changes - the PR's test plan explicitly discloses the gap for the e2e device scenario (good), but says nothing about this one at all, so it reads as simply uncovered rather than a disclosed, accepted gap.
One more thing worth a look, not blocking: builtin_codex_bridge_engine_inventory (via builtin_engine_inventory's static table) still unconditionally claims lemonade runs "with ROCm llama.cpp backend" - the exact stale claim this PR exists to fix. The new lemonade_engine_note helper's own doc comment calls this out by name as a deliberate scope choice (no live GPU probe in front of the codex-bridge inventory), so this may be intentional - but it's the kind of thing worth confirming explicitly rather than leaving implicit, since it's the same wording this PR is removing from two other call sites.
| The GPU is detected but is not supported by ROCm; \ | ||
| the Vulkan backend can still be used." | ||
| the Vulkan backend can still be used. \ | ||
| Try `rocm serve <model>` — it runs on Lemonade's Vulkan backend instead." |
There was a problem hiding this comment.
This is a second user-observable CLI text change in this PR (alongside the device: row), but it has no Gherkin scenario and no justification note in the PR description - unlike the device: row change, whose missing e2e coverage is explicitly disclosed in the test plan as a known, accepted gap. Right now this one just reads as uncovered. Worth either a scenario asserting rocm install sdk's refusal message on an unsupported GPU names rocm serve <model>, or an explicit note in the PR description on why unit-test coverage (which does exist, in the two tests just below this hunk) is sufficient here.
There was a problem hiding this comment.
Went with the PR-description note rather than a scenario, and the reason is stronger than "a scenario costs runtime": a scenario for this branch cannot run on any lane we have. Full write-up is now a dedicated section in the PR description ("Why the rocm install sdk remediation sentence is unit-covered and not a scenario"); the short version:
unsupported_host_gpu_message is reached from resolve_family only after all three earlier doors are closed — no --family, no ROCM_CLI_THEROCK_FAMILY, no managed runtime — and then only when detect_host_gfx_target() itself returns a target normalize_therock_family does not cover and gfx_target_has_no_rocm_build confirms. That needs a real Renoir/Cezanne Vega APU on the runner. Every self-hosted lane reports platform=mi300x os=linux, and detect_host_gfx_target reads real sysfs/WMI with no test override, so the branch cannot be made to fire in e2e. A scenario written for it would be permanently skipped, which reads greener than it is.
What a lane can drive is the sibling branch, and that is now asserted — including the negative you found in your other thread: runtime-19 checks the --family gfx90c refusal points at rocm examine and does not contain rocm serve, precisely because that lane is ROCm-capable. So the one door a lane can open is covered by proving the absence of the prediction rather than enforcing it.
Holding this sentence up: the two unit tests just below (therock.rs:7681, therock.rs:7807), plus the fact that the claim is not a bare string pin — gfx_target_has_no_rocm_build is what routed the user here, and selects_vulkan_when_rocm_unsupported in engines/lemonade/src/backend_alignment.rs is what makes llama.cpp actually pick vulkan once rocm reports unsupported.
One wording change you should know about, since the exact text is what those unit tests sit next to: I merged main into this branch (it was 36 commits behind) and that brought in #683's whole-tree ASCII output contract, which this branch predates. xtask's every_literal_that_can_reach_a_redirectable_sink_is_ascii flagged this literal — and only this one out of everything the branch adds — for the em dash. It is now an ASCII hyphen:
Try `rocm serve <model>` - it runs on Lemonade's Vulkan backend instead.
Both unit tests match on rocm serve <model>, so they are unaffected. Merge commit is daed78d5; full gate re-run there on MI300X: 4178 tests passed / 0 failed, both clippy gates clean, cargo fmt --all --check clean.
| /// places that print this note statically (the dashboard TUI, `rocmd`'s own | ||
| /// catalog) have no host probe in front of them and keep the unconditional | ||
| /// ROCm wording, since they describe the engine in the abstract. | ||
| const fn lemonade_engine_note(level: rocm_core::GpuSupportLevel) -> &'static str { |
There was a problem hiding this comment.
This helper's own doc comment discloses that builtin_codex_bridge_engine_inventory/builtin_engine_inventory (main.rs ~2726-2747) keep the unconditional "ROCm llama.cpp backend" wording for lemonade, since that call site has no live GPU probe in front of it. That may be the right scope call, but it's worth confirming explicitly - it's the exact stale claim about ROCm support this PR exists to fix in the other two call sites, just surfacing through a third one (the codex bridge inventory) that's silently left out.
There was a problem hiding this comment.
You were right that this needed to be decided explicitly rather than left in a doc comment, and on looking at it I don't think the scope call it described actually held. Fixed in 49e23e3 instead of justified.
The old comment claimed the bridge inventory could not be host-aware because it is "the static catalogue". That isn't what blocked it — both call sites can see the host:
build_codex_bridge_snapshot(main.rs:2706) already gathers theExamineSummarycarryingdetected_gfx_targetandcompatible_therock_family, the exact two inputsgpu_support_leveltakes. It now hoists that gather and passes the level down, so the engine rows cost no detection the snapshot wasn't already paying for.- The bridge's
enginestool (main.rs:8123) takes the same cheapdetect_host_gpu_summaryprobe thatrocm enginesalready uses, via a newhost_gpu_support_levelhelper.
So there was no call site here that genuinely could not answer the question, which is what made "left alone deliberately" the wrong reading. As you put it, this is the same stale ROCm claim ROCMAI-550 exists to remove, surfacing through a third user-facing surface — a bridge client renders summary to the operator.
Only the summary string moves. Field set, field names and the installed_binary: true beside it are untouched, so the bridge's JSON shape is unchanged — this isn't the contract change the old comment warned about.
Two places still print the note statically, and that is now stated as deliberate rather than implied: the dashboard TUI's engine-manager pane renders a hard-coded descriptive list with no probe behind it, and rocmd's catalog in apps/rocmd/src/common.rs answers for whichever host a client asks about, not the one it runs on. Either becoming host-aware is its own change.
Gates on the Linux box at 49e23e3: 439 crates compiled, 3852 tests passed / 0 failed, clippy --workspace --all-targets -D warnings clean, and clippy --locked -p e2e-cucumber --test e2e -D warnings clean.
…l is produced The gfx90c refusals in `therock.rs` gained a `rocm serve <model>` remediation sentence, a second user-observable output change in this PR with no scenario behind it. AGENTS.md §3 wants remediation advice asserted where the message is really printed, not only against the formatter. runtime-19 already opens the one door of the two a lane can reach -- naming the target with `--family` -- so the assertion goes there. The detected-host door needs a Vega APU no runner has and stays unit-covered, as its existing comment already records. Also names the codex-bridge engine inventory at both the static table and the bridge helper as a deliberate scope boundary, instead of only in `lemonade_engine_note`'s doc comment where a reader of the table would miss it. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
|
Thanks — both points checked against the code before acting. Pushed Blocking: the One correction to be fair to the test plan: it did list "unit tests for the I chose coverage over disclosure, because it turned out the gap was smaller than either of us assumed. §3 also wants remediation advice tied to the behaviour, not just the string, and that part genuinely cannot run here, so the step's doc comment names what does prove it rather than pinning the wording alone: That leaves one genuine gap, now disclosed in the test plan rather than inherited silently: Non-blocking: the codex-bridge inventory. Confirmed, and deliberate — with one correction to the premise. The On the substance, I considered making it support-aware — |
juhovainio
left a comment
There was a problem hiding this comment.
Re-reviewed from scratch at the current head (this PR has moved substantially since my last review, which GitHub auto-dismissed). This version delivers what ROCMAI-550 asks for: a GpuSupportLevel classification, a GPU-aware Lemonade engine note, an install-sdk remediation sentence, and live device reporting (via the Healthcheck RPC, not guessed) in the serve summary and services list. I grepped the repo for the stale "ROCm llama.cpp backend" claim this ticket is about; the only remaining occurrences are the 4 locations this PR's own new doc comments document as a deliberate, explained exclusion - nothing stale slipped through. CI is fully green across all 24 checks, nothing flaky triggered.
One real issue, left as an inline comment below: the new remediation sentence added to unsupported_requested_target_message makes an unconditional claim about which backend the host will use, but that function fires on a rejected requested override (a --family flag or a stale ROCM_CLI_THEROCK_FAMILY env value) independent of the host's actual GPU. Your own new test in this same diff (cli_family_overrides_a_stale_env_value_naming_an_unsupported_gpu) demonstrates exactly this kind of override/real-host mismatch - which means on a host that actually has a ROCm-capable GPU, this message would tell the user rocm serve runs on Vulkan when it would really run on ROCm/HIP. The identical sentence is correct in the sibling unsupported_host_gpu_message, since that path only fires when host detection itself found no ROCm build, so it's describing a confirmed fact about this host.
Non-blocking, for your awareness:
host_gpu.rs'sExamineSummary::rocm_support_linehand-rolls the same classification the newgpu_support_level()function now exists to express, instead of calling it - currently equivalent, but a future change to one could silently desync from the other.- README.md/docs/testing.md/docs/manual-testing.md aren't updated for the new
device:lines in the serve summary and services list text output. - The lemonade e2e device-reporting Then-step is appended after assertions that are xfailed on that lane (EAI-7423), so it doesn't actually run there yet; only the vLLM canary scenario reliably exercises the new device-reporting path in CI today.
render_services_jsondoesn't get the newdevicefield that the text view does, so--jsonand text output diverge in informational content.
| That GPU is not supported by ROCm; the Vulkan backend can still be \ | ||
| used. Nothing was installed and no other family was substituted." | ||
| used. Nothing was installed and no other family was substituted. \ | ||
| Try `rocm serve <model>` — it runs on Lemonade's Vulkan backend instead." |
There was a problem hiding this comment.
This sentence is added to the requested-target rejection path, which fires independent of the host's real GPU (e.g. a stale ROCM_CLI_THEROCK_FAMILY env value naming an unsupported target on a host that actually has a ROCm-capable GPU - exactly the scenario your own new test cli_family_overrides_a_stale_env_value_naming_an_unsupported_gpu exercises). On such a host, rocm serve <model> would actually run on ROCm/HIP, not Vulkan, making this an incorrect claim rather than a confirmed fact. The identical sentence in unsupported_host_gpu_message above is fine because that path only fires when host detection itself returned the no-build target. Worth either dropping this sentence from the requested-target message, or rephrasing it to not assert a specific backend when the host's actual support level is unknown at this point.
There was a problem hiding this comment.
Confirmed and fixed in 16ccfb3 — you were right, and the precondition difference is exactly as you described.
Checking it before changing anything: unsupported_requested_target_message is raised from the two guards at the top of resolve_family (the --family branch and the ROCM_CLI_THEROCK_FAMILY branch), both of which sit above detect_managed_therock_family and detect_host_gfx_target. No host probe has run at that point. unsupported_host_gpu_message is only reachable further down, after detect_host_gfx_target() returned a target that normalize_therock_family didn't cover and gfx_target_has_no_rocm_build confirmed — so there the sentence is a fact about this host.
What I chose: reworded rather than dropped, to Run rocm examine to see what this host itself supports.
I considered gating it on the real GpuSupportLevel, but getting one there means calling detect_host_gfx_target() inside a branch deliberately placed ahead of all detection — it would contradict the function's own doc comment ("nothing was detected here"), and the existing test comment on requesting_gfx90c_by_name_fails_instead_of_installing_another_family explicitly treats reaching detection as the failure signal. Dropping it outright was the safe option but loses the next step, so pointing at rocm examine seemed strictly better: true on every host, still actionable, and it's the command this PR just taught to report the real support level.
One thing your review surfaced that I think matters more than the wording: runtime-19 was asserting the bug. It drives --family gfx90c, so it goes through this exact function, and its new Then was checking for rocm serve <model> + Vulkan. The CI runner it passes on reports platform=mi300x os=linux gpu=true effective_engine=vllm — a fully ROCm-capable host. So the false claim was being pinned green on precisely the kind of machine where it's false. That step now asserts the rocm examine pointer and the negative (!output.contains("rocm serve")), so the lane now proves the absence of the prediction instead of enforcing it.
Evidence the test fails without the fix — reverted just the message string on the devbox, left the tests alone:
test therock::tests::requested_target_error_does_not_claim_which_backend_serving_will_use ... FAILED
test therock::tests::requested_target_error_names_where_the_target_came_from ... FAILED
the requested target says nothing about this host's GPU, so the refusal must not
predict what `rocm serve` would run on: ... Try `rocm serve <model>` — it runs on
Lemonade's Vulkan backend instead.
Restored, both pass. The new test also asserts the positive side on the sibling path (unsupported_host_gpu_message keeps rocm serve <model> + Vulkan), so it pins the asymmetry rather than banning the sentence outright — if someone later copies it back across, that test is what catches it.
`unsupported_requested_target_message` is raised from the `--family` and `ROCM_CLI_THEROCK_FAMILY` branches at the top of `resolve_family`, before any host probe runs, so the rejected target says nothing about the GPU in this machine. It had borrowed its sibling's closing sentence promising that `rocm serve <model>` runs on Lemonade's Vulkan backend -- true in `unsupported_host_gpu_message`, which is only reached once host detection itself returns a target with no ROCm build, but false here: a stale env value naming gfx90c refuses just the same on a host whose own GPU is fully ROCm-capable, and there serving would run on ROCm/HIP. Point at `rocm examine` instead. It is true on every host, it is still a next step, and it is the command this PR just taught to report the real support level. runtime-19 was asserting the wrong half of this: it drives `--family gfx90c`, so it was pinning the Vulkan-serving claim green on exactly the ROCm-capable runners where it was false. It now asserts the examine pointer and the negative. Also route `ExamineSummary::rocm_support_line` through `gpu_support_level` rather than re-deriving the same classification, and document the serve summary's `device:` row in README and the manual-testing GPU check. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
|
Thanks for the re-review — the blocking finding was correct and is fixed in 16ccfb3. Details on the fix and the falsification evidence are in-thread; summary plus the four non-blocking items here. Blocking — requested-target path claimed a backend it never checked. Verified the precondition difference at the source before touching it: both Non-blocking:
Verification (devbox,
No new assertion interpolates a rendered summary, so no new |
…tory The codex-bridge engine inventory kept the unconditional "with ROCm llama.cpp backend" wording that ROCMAI-550 removed from `rocm examine` and `rocm engines`, so a bridge client still rendered a ROCm claim to the operator on a host that cannot use it. Both call sites can answer the question: the snapshot already gathers the ExamineSummary carrying the two inputs, and the `engines` tool takes the same cheap host probe `rocm engines` uses. Only the `summary` string changes; the field set and the bridge's JSON shape are untouched. The dashboard TUI pane and rocmd's own catalog keep the static wording deliberately -- the pane has no probe behind it, and rocmd answers for whichever host a client asks about, not the one it runs on. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Resolves the one conflict, apps/rocm/src/main.rs, by taking main's post-#667 file (chat/MCP-bridge extraction removed 185 functions) and re-applying this branch's 23 hunks onto it. One hunk follows its function: `run_internal_mcp_call` now lives in apps/rocm/src/chat.rs, so the `engines` tool's call to `builtin_codex_bridge_engine_inventory` is updated there and `host_gpu_support_level` is imported from the crate root. Verified by construction: the `fn` name sets of main, this branch and the resolution comm out to resolved == main's set plus exactly the five functions this branch adds (`fetch_service_device`, `host_gpu_support_level`, `lemonade_engine_note` and the two new examine tests), with nothing of main's dropped. Also converts this branch's one new non-ASCII production literal to ASCII for the output contract #683 added, which this branch predates: the em dash in `unsupported_host_gpu_message`'s "Try `rocm serve <model>`" sentence becomes a hyphen, matching how #683 converted the rest of the tree. `xtask`'s `every_literal_that_can_reach_a_redirectable_sink_is_ascii` named that literal and only that one; everything else this branch adds with non-ASCII is a comment or lives under the exempt `tests/` prefix. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Reviewed the GPU-support-level messaging work (ROCm/HIP vs Vulkan-only vs Unknown) across host_gpu.rs, the codex-bridge/MCP snapshot, rocm engines/rocm examine, and the install-sdk refusal paths.
The core idea is sound: a 3-tier GpuSupportLevel replaces a static "ROCm llama.cpp backend" claim with one that reflects what the host actually supports, and it's backed by real e2e coverage (serve-05/06/07, runtime-19) plus an honest EAI-7423 xfail scoped correctly to effective_engine=lemonade, os=linux.
One thing worth flagging before this merges: the install-sdk refusal messages are not symmetric between the host-detected path (unsupported_host_gpu_message, adds "Try rocm serve <model>") and the explicit --family/env path (unsupported_requested_target_message, points at rocm examine instead) — but that's deliberate and well-documented/tested here (requested_target_error_does_not_claim_which_backend_serving_will_use), so just calling it out as intentional, not a problem.
Blocking:
- New GPU-aware behavior on the codex-bridge/MCP surface (
host_gpu_support_level, the new branch inbuiltin_codex_bridge_engine_inventory, and the"engines"MCP tool call site inchat.rs) has zero test coverage.append_examine_engine_inventory(rocm examine) got solid new tests for bothVulkanOnlyandRocmHip, but neitherrender_engine_inventory_text_with_paths(rocm engines) nor the codex-bridge path exercise the same branch directly — see inline comment.
Non-blocking:
- Stale doc reference in
host_gpu.rs— see inline comment. - The
lemonadenote-substitution logic (if *engine == "lemonade" { lemonade_engine_note(...) }) is now duplicated three times acrossrender_engine_inventory_text_with_paths,append_examine_engine_inventory, andbuiltin_codex_bridge_engine_inventory. Not blocking, but a shared helper would avoid the three call sites drifting independently later.
Ticket scope: Confirmed against ROCMAI-550 (epic ROCMAI-517) — the change matches the ticket's intent (stop asserting a static ROCm claim, report real support level instead). One review pass raised that docs/rocm-docs/, docs/ux-guidelines.md, doctor skill files, and --help text might still carry the old unconditional claim; a follow-up repo-wide grep came back clean on all of those. Flagging the discrepancy rather than silently picking one — worth a quick independent grep before merge if you want to be certain, but I did not find the stale string anywhere outside what's already fixed in this diff.
CI: The PR was just updated with a merge of main (resolving an earlier merge conflict cleanly — verified independently, chat.rs's call site already correctly passes host_gpu_support_level(Some(paths))). Most checks are still pending against the new merge commit as of this review; nothing has failed so far.
| /// `Unknown` deliberately covers both "nothing detected" and "a gfx target we | ||
| /// don't recognize" rather than being called `Unsupported` — see the doc | ||
| /// comment on [`gfx_target_has_no_rocm_build`] and | ||
| /// `GFX_TARGETS_WITHOUT_ROCM_SUPPORT` in `diagnose.rs` for why conflating |
There was a problem hiding this comment.
This doc comment points to GFX_TARGETS_WITHOUT_ROCM_SUPPORT in diagnose.rs, but that constant doesn't exist anywhere in the repo (checked via a full-repo grep). Either the name changed before this landed or it was never added — worth fixing so the doc doesn't send readers chasing a symbol that isn't there.
There was a problem hiding this comment.
Confirmed and fixed in bce633d6. GFX_TARGETS_WITHOUT_ROCM_SUPPORT appears exactly once in the repo — in this doc comment. diagnose.rs has no such constant (its consts are the KEYWORDS_* tables, MIN_SCORE_FOR_MATCH, SHM_MIN_BYTES, etc.), and nothing matching GFX_TARGETS* or *WITHOUT_ROCM* exists anywhere else either. It was never a real symbol, not a rename.
The sentence needed the second reference because gfx_target_has_no_rocm_build alone doesn't carry the "confirmed" half — that comes from its conjunction with normalize_therock_family. So rather than point at a different file, the doc now names the conjunct that actually establishes it:
Only that predicate's conjunction with [
normalize_therock_family] establishes the "confirmed" half; an unrecognized target satisfies neither conjunct, so it is a gap in what this catalog knows rather than a fact about the card.
Both links are now intra-module rustdoc links, so a future rename breaks the doc build instead of rotting silently.
| /// The same two inputs `rocm examine` derives it from, read through | ||
| /// `detect_host_gpu_summary` rather than the full examine gather — exactly what | ||
| /// `render_engine_inventory_text_with_paths` (i.e. `rocm engines`) already does. | ||
| fn host_gpu_support_level(paths: Option<&AppPaths>) -> rocm_core::GpuSupportLevel { |
There was a problem hiding this comment.
This function (and the new GPU-aware branch it feeds into builtin_codex_bridge_engine_inventory, plus the "engines" MCP tool call site in chat.rs) has no test coverage. append_examine_engine_inventory got examine_engine_inventory_names_vulkan_for_a_gpu_with_no_rocm_support / ..._keeps_rocm_note_for_a_rocm_capable_gpu for the equivalent rocm examine branch — this surface (and render_engine_inventory_text_with_paths's own rocm engines output) should get the same treatment before merge, since it's the same user-facing correction (no more asserting ROCm support on a Vulkan-only host) just reached through the MCP bridge instead of the CLI.
There was a problem hiding this comment.
You're right, and the gap was mine — I pushed 49e23e30 earlier today without tests. Fixed in bce633d6.
Both surfaces were untestable for the same reason: the branch's input is a live sysfs probe, and no CI runner has a GPU that is detected and confirmed to have no ROCm build. So each now takes the level the way append_examine_engine_inventory already did — the caller probes once, the renderer is handed the answer:
render_engine_inventory_text_for_host(paths, host_default_engine, level)— split out ofrender_engine_inventory_text_with_paths, which still does the onedetect_host_gpu_summarycall and derives both arguments from it, so the*marker and the note cannot disagree about the same host.internal_mcp_engines_tool_result(level)— split out of the"engines"match arm inchat.rs. Same text, sameengineskey, same entries.
Six new tests mirroring yours, plus three on the probe wiring:
| test | proves |
|---|---|
engines_inventory_names_vulkan_for_a_gpu_with_no_rocm_support |
rocm engines names the Vulkan backend, drops the ROCm claim, and leaves vLLM's row alone |
engines_inventory_keeps_rocm_note_for_a_rocm_capable_gpu |
a ROCm host keeps the ROCm wording and is not told it's on Vulkan |
engines_inventory_does_not_claim_vulkan_for_an_unrecognized_gpu |
Unknown is a detection gap, not a confirmed absence |
codex_bridge_engine_inventory_names_vulkan_… / …_keeps_rocm_note_… / …_does_not_claim_vulkan_… |
the same three on CodexBridgeEngine.summary, with installed_binary and vLLM's row asserted unchanged so the JSON shape is pinned too |
engines_mcp_tool_names_vulkan_… / …_keeps_rocm_note_… |
asserted on structuredContent.engines[].summary — the exact string an MCP client renders |
engines_mcp_tool_answers_for_the_host_it_runs_on |
the tool reaches the inventory through the live probe, not a level nailed to a constant |
gpu_support_level_for_host_gpu_reads_a_vulkan_only_summary_as_vulkan_only (+ two) |
covers host_gpu_support_level's derivation |
Falsified every one — five mutations on the devbox, each run confirmed non-zero test count:
engine_inventory_note'sengine == "lemonade"→false: 3 new tests red (engines_inventory_names_vulkan…,codex_bridge_…_names_vulkan…,engines_mcp_tool_names_vulkan…), plus yourexamine_engine_inventory_names_vulkan….lemonade_engine_note'sVulkanOnly =>→RocmHip =>: 6 new tests red — both directions on all three surfaces.VulkanOnly =>→Unknown =>: 4 new tests red, including both…_does_not_claim_vulkan_for_an_unrecognized_gpu, so theUnknownarm has teeth.gpu_support_level_for_host_gpuarguments transposed:gpu_support_level_for_host_gpu_reads_a_vulkan_only_summary_as_vulkan_onlyred. The twoOption<String>fields are positional and same-typed, so that mutation compiles clean and classifies exactly the Vulkan-only host asRocmHip— it was the one genuinely silent failure mode here."engines"arm hard-coded toGpuSupportLevel::VulkanOnly:engines_mcp_tool_answers_for_the_host_it_runs_onred.
On your duplication point: I did extract the helper — engine_inventory_note(engine, catalog_note, level) — and all three renderers now route through it. The tests deliberately drive the renderers and never that helper, per your #703 note; mutation 1 above is the proof that going through it is what they're actually checking.
One honest limitation: host_gpu_support_level(Some(paths)) at the call site is covered only by engines_mcp_tool_answers_for_the_host_it_runs_on, which can catch a hard-coded level that disagrees with the runner but not one that happens to render identically to it (Unknown and RocmHip share wording). The discriminating coverage for a real Vulkan-only host stays with the e2e lane.
Gates on the devbox, all green: cargo test --workspace --all-targets --no-fail-fast (rc 0, 35 suites, all 12 new tests present), cargo clippy --workspace --all-targets -- -D warnings (rc 0), cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings (rc 0), cargo fmt --all clean.
… and the bridge `append_examine_engine_inventory` got tests for both support levels when ROCMAI-550 landed; the other two surfaces carrying the same correction did not. `rocm engines` and the codex-bridge inventory (including its `engines` MCP tool) could have gone back to asserting ROCm support on a Vulkan-only host with nothing red. Both were untestable for the same reason: the branch's input is a live sysfs probe, and no CI runner has a GPU that is detected *and* confirmed to have no ROCm build. So each surface now takes the level the way `append_examine_engine_inventory` already did -- the caller probes once, the renderer is given the answer. `render_engine_inventory_text_for_host` and `internal_mcp_engines_tool_result` are those renderers; neither changes what is printed. The three copies of the lemonade substitution collapse into `engine_inventory_note`, so the surfaces cannot drift. The new tests drive the renderers, not that helper: asserting on it would restate the condition rather than prove a surface prints it. `gpu_support_level_for_host_gpu` reads the two `HostGpuSummary` fields once instead of at each call site. They are same-typed positional arguments, so a transposed call site would have classified a Vulkan-only host as `RocmHip` and still compiled; it now has a test. Also drops the dangling `GFX_TARGETS_WITHOUT_ROCM_SUPPORT` reference from the `GpuSupportLevel` doc comment -- no such constant exists in `diagnose.rs` or anywhere else, so the doc pointed readers at nothing. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
|
Addressed the blocking item and both non-blocking ones in On your third point — the independent grep you asked for. I ran it and it agrees with yours: no stale unconditional claim survives outside this diff.
Specifically on the four places the earlier pass worried about:
So: nothing to fix, second pair of eyes agrees. Also leaving Re-requesting your review. |
juhovainio
left a comment
There was a problem hiding this comment.
Re-reviewed at the current head (bce633d6) after the prior review (at daed78d5) was superseded by new commits.
Blocking
apps/rocm/src/therock.rs(unsupported_host_gpu_message): the new closing sentence always promises Lemonade's Vulkan fallback viarocm serve. Butselect_serve_enginelets a configureddefault_engineoverride (e.g.vllm) win regardless of host GPU support, so a user on an unsupported-ROCm GPU with such a config gets a message promising a Vulkan fallback thatrocm servewon't actually take. Please condition the message on the engine that will actually be selected, or soften it to not name a specific backend unconditionally.
Non-blocking
apps/rocm/src/main.rs(render_services_text/fetch_service_device): the live per-serviceHealthcheckRPC on eachrocm services listrender has no visible timeout bound. The in-code comment explains why this reads live instead of from the persistedManagedServiceRecord(avoiding staleness) — that part's fine — but worth confirming there's an upstream timeout inengine_request, otherwise a hung/slow third-party engine plugin could stall the whole listing.- The
gpu_support_levelderivation (rocm_core::gpu_support_level(detected_gfx_target, compatible_therock_family)) is duplicated verbatim betweenbuild_codex_bridge_snapshotandexamine_human_report; an analogous helper to the PR's owngpu_support_level_for_host_gpuwould avoid the duplicated positional-argument call. - Ticket scope (ROCMAI-550) nuance: the ticket's literal wording asks for a three-tier model ("full ROCm/HIP support, Vulkan only, or unsupported"), but this ships two confirmed tiers plus
Unknownfor detection gaps rather than a confirmed-unsupported verdict. Seems like a deliberate, defensible choice to avoid false-negative claims, but flagging since it's a wording deviation from the ticket. - The PR's description implies the stale "ROCm llama.cpp backend" claim has been fully purged; confirmed true in README/docs/rocm-doctor, but
apps/rocmd/src/common.rs:186(rocmd_engine_inventory) still hard-codes that exact phrase and is untouched by this PR. May be intentionally out of scope (rocmd answers for an arbitrary queried host, not necessarily the one it runs on) but it's a live remaining instance worth a look.
CI / mergeability
CI is green aside from the known Strix Halo Windows self-hosted download-infra flake (repo.amd.com 403) and its consolidated-report rollup failure — not a regression. Separately, the PR currently shows mergeable=CONFLICTING and will need a rebase/merge-conflict resolution regardless of this review.
| The GPU is detected but is not supported by ROCm; \ | ||
| the Vulkan backend can still be used." | ||
| the Vulkan backend can still be used. \ | ||
| Try `rocm serve <model>` - it runs on Lemonade's Vulkan backend instead." |
There was a problem hiding this comment.
This sentence is unconditional, but select_serve_engine can select a configured default_engine override (e.g. vllm) regardless of host GPU support. On an unsupported-ROCm GPU with such a config, rocm serve won't actually take the Vulkan/Lemonade path this message promises. Consider conditioning this on the engine that will actually be selected, or softening the wording.
There was a problem hiding this comment.
Confirmed and fixed in f54a8d2f — you and @nowycondro landed on the same defect from two directions, and you were both right. select_serve_engine (apps/rocm/src/serve_cmd.rs) ranks explicit --engine first, then configured_default, and only then the host preference, so a host that reaches unsupported_host_gpu_message can still have rocm serve <model> select vLLM.
I pinned the engine rather than conditioning the wording on config. apps/rocm/src/therock.rs:3607 now reads:
Try
rocm serve <model> --engine lemonade- it runs on Lemonade's Vulkan backend instead.
The reason for pinning over conditioning: resolve_family would have to read RocmCliConfig to condition, which adds a config load to an install-path error and still only covers the configured-default source — the recipe-preferred-engine source (which @copilot-pull-request-reviewer raised separately) would remain. An explicit --engine out-ranks every selection source in that function, so the sentence is true whatever the config and whatever the recipe says.
New test at apps/rocm/src/therock.rs:8399, unsupported_gpu_error_pins_the_engine_a_configured_default_could_override. It pins both halves — the engine-pinned command, and the absence of the bare one still carrying the promise. Falsified on the MI300X box by reverting the message to the bare command: that test and requested_target_error_does_not_claim_which_backend_serving_will_use both go red (the latter because it asserts the sibling's positive side).
| // actually reports right now. Only meaningful once the engine is ready | ||
| // to answer, same gate `rocm serve`'s summary uses. | ||
| if record.status == "ready" | ||
| && let Some(device) = fetch_service_device(paths, &record.engine, &record.service_id) |
There was a problem hiding this comment.
Non-blocking: this issues a live Healthcheck RPC per listed service with no visible timeout bound here. The comment above explains why it reads live rather than from the persisted record (good reasoning), but worth confirming engine_request enforces a timeout upstream — otherwise a slow/hung third-party engine plugin could stall the whole rocm services list render.
There was a problem hiding this comment.
Checked it, and the answer to "worth confirming engine_request enforces a timeout upstream" is no, it does not — so this needed a fix, not a confirmation. engine_request_with_env_root (apps/rocm/src/engine_ipc.rs) ends in a plain child.wait() with no deadline; there is no *_TIMEOUT anywhere in that module. I had half-written a doc comment claiming a timeout before I went and read it, which is a good argument for your asking.
What is bounded is the engine side, and only for the built-ins: rocm serve's three engines take the in-process fast path, and every endpoint probe they make is timeout-bounded (engines/vllm's HEALTHCHECK_TIMEOUT_MS, engines/lemonade's rocm_core::INFERENCE_PROBE_TIMEOUT). An external plugin is a spawned child and is genuinely unbounded.
Fixed in f54a8d2f by bounding the aggregate at the call site rather than per call — fetch_service_devices (engine_ipc.rs:365) under SERVICE_DEVICE_PROBE_BUDGET (:345, 10s). Once the budget is spent the remaining services simply get no device: row. render_services_text (services_text.rs:441) goes through it.
Two things I deliberately did not do, both stated in the doc comment rather than left implicit:
- Not parallelised.
engine_requestroutes the built-ins throughwith_scoped_builtin_engine_env, which takes a process-wide mutex and mutates process-globalROCM_CLI_*env vars for the duration. Concurrent probes would serialise on that lock anyway and racing them would be unsound. - The budget is checked between probes, not inside one. A single external plugin that never exits still blocks. Bounding that means a detached thread, and abandoning a worker that holds the env lock mid-probe would leak mutated process env into whatever ran next — worse than the problem. That is the pre-existing behaviour of every engine call site in the CLI and this does not make it worse.
nowycondro
left a comment
There was a problem hiding this comment.
Automated review (medium/high-effort pass). 3 findings posted inline, ranked by severity: (1) a correctness issue where unsupported_host_gpu_message's new "Try rocm serve <model>" sentence can be false when the user has a configured default_engine override; (2) a documentation/behavior mismatch where vLLM's actual device string (rocm_gpu) doesn't match the format the PR documents (rocm gpu 0/rocm gpu <n>); (3) a minor efficiency note that rocm services list now does live per-service network healthchecks where it previously only read local state. No other blocking issues found — the core GpuSupportLevel plumbing, the rocm serve/services list device-row gating, and the refactor of append_examine_engine_inventory/render_engine_inventory_text_for_host all look correct and consistently wired.
| The GPU is detected but is not supported by ROCm; \ | ||
| the Vulkan backend can still be used." | ||
| the Vulkan backend can still be used. \ | ||
| Try `rocm serve <model>` - it runs on Lemonade's Vulkan backend instead." |
There was a problem hiding this comment.
This sentence unconditionally promises that rocm serve <model> "runs on Lemonade's Vulkan backend", but this branch (unsupported_host_gpu_message) only confirms the host GPU has no ROCm build — it says nothing about which engine rocm serve will actually pick. Per select_serve_engine (apps/rocm/src/serve_cmd.rs:60-78), a configured_default_engine (set via rocm config set-default-engine, which is never validated against the host's GPU) outranks the host's engine preference. So a user who previously ran rocm config set-default-engine vllm (e.g. after moving a config from a ROCm box, or by mistake) would have rocm serve <model> attempt vllm on this Vulkan-only host and fail under the project's strict no-CPU-fallback policy — directly contradicting this message's promise.
Notably, the sibling message unsupported_requested_target_message in this same PR deliberately avoids making this exact claim (see its doc comment a few lines below) because it "never looked at this host" — but unsupported_host_gpu_message did look at the host's GPU, just not at the user's configured engine override, so the same class of false-promise risk applies here too. The new unit tests (therock.rs ~7677, ~7807) only pin the string "rocm serve <model>", not that running it actually succeeds via Lemonade under a configured-engine override.
Failure scenario: host is gfx90c (Vulkan-only, confirmed via detect_host_gfx_target), rocm_cli_config.default_engine == Some("vllm"). rocm install sdk (no --family) hits this branch and tells the user to "Try rocm serve <model>" promising Vulkan/Lemonade; rocm serve <model> instead selects vllm (configured default wins) and fails since there's no ROCm GPU.
There was a problem hiding this comment.
Confirmed and fixed in f54a8d2f. Your failure scenario is exact, including that rocm config set-default-engine never validates against the GPU — I checked select_serve_engine (apps/rocm/src/serve_cmd.rs) and configured_default does sit above the host-GPU preference, so rocm serve <model> on a gfx90c host with default_engine = "vllm" attempts vLLM and dies under the no-CPU-fallback policy while this message promised Lemonade.
apps/rocm/src/therock.rs:3607 now pins the engine:
Try
rocm serve <model> --engine lemonade- it runs on Lemonade's Vulkan backend instead.
An explicit --engine out-ranks every other selection source in that function, so this is true for every config and for a recipe whose preferred_engines would otherwise win — which closes the second half of the same hole that a config-conditional message would have left open.
Your point that the sibling message deliberately avoids the claim is why I did not just drop the sentence here: the asymmetry is real (this branch did look at the host) and worth keeping, it was only over-claiming the engine, not the backend. The doc comment above the function now says exactly that, and the new test unsupported_gpu_error_pins_the_engine_a_configured_default_could_override (therock.rs:8399) pins it against the configured-default regression specifically.
On your note that the old tests only pinned the string "rocm serve <model>": true, and that is why the new test asserts the negative too (no bare rocm serve <model> followed by the promise), not just the presence of the longer form — the longer form contains the shorter one as a prefix, so a presence-only assertion could not tell them apart.
| - The engine uses ROCm GPU execution. The deployment summary's `device:` row | ||
| reports what the engine actually selected, so check it by reading that row | ||
| rather than inferring it from the absence of a fallback warning: expect | ||
| `rocm gpu <n>`. `rocm services` shows the same `device` while it runs. |
There was a problem hiding this comment.
This tells the manual tester to expect rocm gpu <n> in the new device: row for a vLLM/ROCm run, matching the format documented in README.md:680 (rocm gpu 0) and the doc comment on DeploymentSummary.device (serve_summary.rs:59). But vLLM's own Healthcheck handler (engines/vllm/src/state.rs:60-66) hardcodes the device string as the literal constant "rocm_gpu" (underscore, no GPU index) whenever state.is_some():
let device = if state.is_some() {
"rocm_gpu"
} else {
"unknown"
};fetch_service_device (apps/rocm/src/main.rs) displays response.device verbatim with no reformatting. So for the vLLM engine — the only engine that reports a rocm* device — the actual printed line will read device: rocm_gpu, not device: rocm gpu 0 as this doc (and the README) describe. A tester following this doc literally would flag a false failure, or a reader would be misled about the real format. This is also the serve-05 vLLM canary the PR added device:-row e2e coverage for, and the PR's own test-plan notes it could not actually exercise that step live due to an unrelated environment gap — so this mismatch was never observed end-to-end.
There was a problem hiding this comment.
You are right about the mismatch and I fixed the code rather than the doc, in f54a8d2f. One correction to the attribution, which does not change the finding: the section at that line is ## 3. Lemonade GPU Verification (rocm serve qwen --engine lemonade ...), so the doc's own command is Lemonade, not vLLM. But your underlying claim is exactly right and the mismatch was real in two places:
engines/vllm/src/state.rshardcoded the literal"rocm_gpu", whichfetch_service_deviceprints verbatim. README.md'srocm gpu 0and this doc'srocm gpu <n>described a format vLLM never produced.engines/aim/src/state.rs(new onmainsince this PR opened) had the same literal.- Even for Lemonade,
reported_device(None, "rocm")returns a bare"rocm gpu", sorocm gpu <n>was not guaranteed there either.
What changed:
engines/vllm/src/state.rs:87gains areported_devicemirroring Lemonade's:"rocm gpu {index}", falling back to"rocm gpu".write_running_statenow persistsgpu_indices(introspection only, never read back to relaunch) so the ordinal is available. Test:reported_device_matches_the_shape_lemonade_reports(state.rs:759).engines/aim/src/state.rs:43reports"rocm gpu". No ordinal: AIM's state records nogpu_indices(the container gets the device list), so claiming one would be inventing it — stated in the comment rather than left implicit.docs/manual-testing.md:286-293now describes the real contract:rocm gpu <n>, or a barerocm gpuwhen no ordinal was pinned, and says the first word is the half that decides the check. README.md's example is now accurate for every engine.
Side effect worth flagging: main's every_advised_command_parses guard reads a backticked `rocm gpu 0` as advice to run a command. That gate was failing on this branch after the merge — six findings across README.md, docs/manual-testing.md and the two engines. They are now NOT_INVOCATIONS entries with reasons (apps/rocm/src/advised_commands.rs), which is the escape hatch that guard documents.
I could not verify the printed row on real hardware: the dev box has no amd-smi/rocm-smi and no usable GPU telemetry, so this is unit-level plus the strengthened e2e step, not an observed device: line.
| // actually reports right now. Only meaningful once the engine is ready | ||
| // to answer, same gate `rocm serve`'s summary uses. | ||
| if record.status == "ready" | ||
| && let Some(device) = fetch_service_device(paths, &record.engine, &record.service_id) |
There was a problem hiding this comment.
Minor efficiency/behavior-change note: before this PR, rocm services list (render_services_text) only read the locally persisted ManagedServiceRecords — pure local I/O. Now, for every record with status == "ready", it calls fetch_service_device, which issues a live engine Healthcheck RPC. For the Lemonade engine this RPC itself performs real HTTP requests to the running server's /v1/health and (on a miss) /v1/models endpoints (engines/lemonade/src/process.rs::query_loaded_model_endpoint), each with its own multi-second timeout, on every invocation (the latch only short-circuits the separate "has inference been verified" check, not this listed query).
So rocm services list --all with several ready services now makes that many live network round-trips to each server before printing anything, and a single slow/stalled/unresponsive server can noticeably delay what used to be an instant, purely-local listing command. This is probably an acceptable tradeoff for the new device: info, but worth confirming it's intentional — e.g. whether services list should cap total wait time or run these checks concurrently rather than sequentially per record.
There was a problem hiding this comment.
Confirmed and bounded in f54a8d2f. Your reading of the cost is right: render_services_text was pure local I/O, and the row made it one live Healthcheck RPC per ready record, sequentially, each with its own multi-second engine-side timeout. @juhovainio raised the same surface from the timeout angle, and the answer to his question turned out to be that engine_request enforces no deadline at all — engine_request_with_env_root ends in a bare child.wait().
What changed: fetch_service_devices (apps/rocm/src/engine_ipc.rs:365) runs the probes under one shared SERVICE_DEVICE_PROBE_BUDGET (:345, 10s). Once spent, the remaining services get no device: row rather than each adding its own wait. So rocm services list --all is now flat in the number of services instead of linear in it, and the row is informational so dropping it is strictly better than a long hang. services_text.rs:441 is the call site.
On your "run these concurrently" suggestion — I looked and it is not available: engine_request routes the three built-in engines through with_scoped_builtin_engine_env, which takes a process-wide mutex and mutates process-global ROCM_CLI_* env vars for the duration of the call. Concurrent probes would serialise on that lock anyway, and racing them would be unsound rather than merely useless. A cap was the option actually on the table, which is why I took it. The doc comment on fetch_service_devices says this so the next person does not re-derive it.
One honest limit, also in the doc comment: the budget is checked between probes, not inside one, so a single external plugin that never exits still blocks. Bounding that needs a detached thread, and abandoning a worker that holds the env lock mid-probe would leak mutated process env into the next operation. That is a worse trade than the one it fixes, and it is pre-existing behaviour of every engine call site in the CLI.
|
🤖 Automated review (Claude Code) · bce633d This supplements my review posted above (PR review #5655784868) with the required format. Findings (COMMENT only, non-blocking severity noted per item):
What I read: the full diff for this PR at bce633d, plus the surrounding non-diff context for What I could not verify: no Rust toolchain was available in this sandbox, so none of this was confirmed by No other correctness issues found beyond what's listed above and what's already been raised in the existing review thread. 🤖 Generated with Claude Code |
nowycondro
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · bce633d
Review
Full independent review of the whole PR diff (base d4e7c75 → head bce633d6, 14 files, +856/-53), using a two-worker fan-out plus direct source verification of every surviving finding. Outcome: needs work.
Intent: implements most of ROCMAI-550 ("clear messages for AMD GPUs that ROCm doesn't support but Vulkan does") — adds a GpuSupportLevel classification, GPU-aware Lemonade engine wording in rocm examine/rocm engines/the codex-bridge inventory, an install-sdk remediation sentence, and live backend/device reporting in rocm serve's summary and rocm services list. The ticket is still "In Progress" and the PR does not claim to close it; its own scope note says part of the ticket (gfx90c detection/rocm install sdk wording) was already covered by a prior PR (#648), which is consistent with what the ticket itself asks for.
Blocking
-
GpuSupportLevel::Unknownstill falls through to the stale "ROCm llama.cpp backend" claim —apps/rocm/src/main.rs:2831-2838(lemonade_engine_note): the match has an explicitVulkanOnlyarm and a catch-all_arm returning"default embedded Lemonade server with ROCm llama.cpp backend"._catches bothRocmHip(correct) andUnknown(wrong) — and perhost_gpu.rs,Unknowncovers both an unrecognized gfx target and no AMD GPU detected at all. So on a host with no AMD GPU,rocm examine,rocm engines, and the codex-bridge/MCP "engines" tool all still unconditionally print the exact claim this PR (ROCMAI-550) exists to remove. The one test covering this case,engines_inventory_does_not_claim_vulkan_for_an_unrecognized_gpu, only asserts the negative (!output.contains("no ROCm/HIP support")) and never asserts the positive ROCm claim is absent, so this passes CI undetected.
Confidence 90 · logic · Fix: add an explicitGpuSupportLevel::Unknownarm with wording that doesn't assert either backend (e.g. "GPU support not yet determined"), and extend the existing test to assert the ROCm string is also absent. -
unsupported_host_gpu_message's new remediation sentence can be false under a configured engine override —apps/rocm/src/therock.rs:3463: unconditionally appends"Try \rocm serve ` - it runs on Lemonade's Vulkan backend instead."Butselect_serve_engine(apps/rocm/src/serve_cmd.rs:60-109) checks a configureddefault_engine*before* any host-GPU preference (lines 73-78 return early on a non-empty configured value; the host-GPU branch at line 80 is only reached if that's unset). So a user on an unsupported-ROCm-GPU host with e.g.default_engine = "vllm"configured gets a message promising a Vulkan/Lemonade fallback thatrocm servewill not actually select. The sibling message (unsupported_requested_target_message) had this exact class of bug fixed on this same branch (commit16ccfb3) with the stated reasoning "only reached once host detection itself confirmed no ROCm build" — that reasoning holds for host detection, but doesn't account forselect_serve_engine's config-override precedence, so the asymmetry fix was incomplete. Confidence 85 · logic · Fix: condition the sentence on whatselect_serve_engine` will actually pick (or soften it to not name a specific backend unconditionally, the way its sibling message was changed to do). -
Two CI checks are currently failing:
E2E tests (Strix Halo, Windows, current)andE2E consolidated report (self-hosted)(re-verified live against headbce633d6immediately before this review). This review could not fetch the job logs (network egress to the Actions API was blocked in this review environment), so the root cause is not independently confirmed from here — the job names suggest a Windows self-hosted hardware lane and its rollup, which has been attributed elsewhere to download-infrastructure flakiness unrelated to this diff, but that should be reconfirmed by someone who can read the logs rather than assumed clean.
Confidence 100 (checks are red, verified live) · process · Fix: confirm cause, re-run if infra-flake, or fix if real.
Non-blocking
-
rocm services list --json(and the MCPservicestool) never gain the newdevicefield —ManagedServiceRecord(crates/rocm-core/src/service_record.rs:29) is serialized verbatim for JSON (render_services_json,apps/rocm/src/main.rs:6834) and carries nodevicefield; only the text renderer calls the newfetch_service_device. A scripting/MCP consumer gets strictly less information than a human reading text output now does.
Confidence 85 · architectural · Fix: either adddeviceto the JSON record (live-fetched, not persisted) or document the asymmetry. -
No test exercises the positive render path of the new
device:line, and the one e2e assertion that does touch it is weak —tests/e2e-cucumber/tests/e2e/serving_steps.rs'sassert_services_list_reports_deviceonly checks that a line starting withdevice:exists; it never checks the value. This is currently safe only becausefetch_service_device(apps/rocm/src/main.rs:10323) already filters out empty/"unknown"values before the CLI ever writes the label — an invariant the test itself doesn't pin, so if that filter weakened, this step wouldn't catch it. On the unit side,serve_summary.rs'sdevice_row_shown_only_when_knowntest only covers "shown when known" / "absent by default" — nothing seeds a real ready-state Healthcheck response end-to-end and asserts the exact rendered value.
Confidence 75 · test-coverage · Fix: add a test that seeds a concrete device string and asserts it appears verbatim. -
vLLM's actual
devicestring doesn't match the documented format —engines/vllm/src/state.rs:64hardcodes the literal"rocm_gpu"(no GPU index, underscore not space), while this PR's own doc comment (serve_summary.rs), README.md, and docs/manual-testing.md all document the shape as"rocm gpu 0"/"rocm gpu <n>"(matching Lemonade's{backend} gpu {index}format inengines/lemonade/src/state.rs:60-64). Not a functional bug — the string is still truthful — but this data was never user-facing before this PR, and now it is, inconsistently.
Confidence 80 · mechanical · Fix: either have vLLM's healthcheck report an index-bearing device string to match, or adjust the documented examples to not imply one exists for vLLM. -
gpu_support_level(...)is called with raw positional args, duplicated at two sites —build_codex_bridge_snapshot(apps/rocm/src/main.rs:2715) andexamine_human_report(apps/rocm/src/main.rs:~5097) both callrocm_core::gpu_support_level(examine.detected_gfx_target.as_deref(), examine.compatible_therock_family.as_deref())directly instead of through a single-call helper analogous to the PR's owngpu_support_level_for_host_gpu. Both arguments areOption<&str>, so a future transposition at either site would compile and silently misclassify.
Confidence 75 · mechanical · Fix: add anExamineSummary::gpu_support_level(&self)(or similar) and route both call sites through it. -
apps/rocmd/src/common.rs:186(rocmd_engine_inventory) still hardcodes the unconditional claim — "embedded Lemonade server with ROCm llama.cpp backend", with noGpuSupportLevelparameter and no comment in this file explaining the exemption. The only stated rationale for excludingrocmdlives in a doc comment on a different function in a different file ("the still-host-blind consumers ... keep the static row"), but README.md describesrocmdas "the background telemetry daemon used by the dashboard" that's auto-started locally — which is in tension with treating it as answering for an arbitrary, not-necessarily-local host. May be intentional, but it's unverified from the code as it stands.
Confidence 70 · logic · Fix: add a comment at the rocmd call site itself stating the rationale, or make it GPU-aware too if it in fact usually runs on the host it describes. -
rocm services list's text render now makes a live, synchronous, unparallelized RPC per listed service (fetch_service_device→engine_request,apps/rocm/src/main.rs:6730-6804) where it previously only read local disk state. Each call is individually timeout-bounded, so it won't hang indefinitely, but total latency now scales linearly with the number of concurrently-ready services being listed.
Confidence 70 · architectural · Fix: consider parallelizing the per-service fetch, or note the latency tradeoff is accepted.
Decisions for the author
GpuSupportLevelships three tiers (RocmHip/VulkanOnly/Unknown) rather than the ticket's literal "full ROCm/HIP support, Vulkan only, or unsupported" — tradeoff. Collapsing "unrecognized gfx target" and "no GPU detected" intoUnknowninstead of a confirmedUnsupportedavoids a false-negative claim on a detection gap, and is well-documented inhost_gpu.rs's doc comments with a dedicated test guarding the distinction. Worth the author explicitly confirming this reading of the ticket's "unsupported" tier is the intended scope, since it is a visible wording deviation from the ticket text.
Positive signals
gpu_support_level/gpu_support_level_for_host_gpu(crates/rocm-core/src/host_gpu.rs) are genuinely a single source of truth with a dedicated test (gpu_support_level_for_host_gpu_reads_a_vulkan_only_summary_as_vulkan_only) specifically guarding against the two positionalOption<&str>arguments being silently transposed.lemonade_engine_note(apps/rocm/src/main.rs:2831) is a single shared helper reused by all three GPU-aware call sites (rocm examine,rocm engines, codex-bridge) rather than duplicated inline per site, so the three renderers can't independently drift apart on the parts it does cover.serve_summary.rs'sredacted()test helper (masksapi key: ...before interpolating a rendered summary into a panic message) is a thoughtful, specific guard against leaking secrets into retained CI logs.- The PR description's test-plan section is unusually candid about what wasn't verified end-to-end (both GPU canaries failing locally for disclosed, pre-existing environment reasons) rather than silently claiming full coverage.
Deployment notes
- The PR currently shows
mergeable: CONFLICTING/mergeStateStatus: DIRTYagainst currentmain(re-verified live immediately before posting this review) — it needs a rebase/conflict resolution before it can be merged, independent of the findings above. - This surfaces a live per-service network RPC inside
rocm services listandrocm serve's summary that did not exist before (see the latency non-blocking item above); operators scripting against these commands at scale should be aware the command's cost profile changed.
What this covered
Read the complete diff (base d4e7c75194d1643eacc3b1bda33afe10766def73 → head bce633d6e143d72ee8a147bb189f7dbbd72552f0, all 14 changed files) directly for crates/rocm-core/src/host_gpu.rs, crates/rocm-core/src/lib.rs, apps/rocm/src/serve_cmd.rs, apps/rocm/src/serve_summary.rs, apps/rocm/src/therock.rs, README.md, and docs/manual-testing.md, including the functions each diff touches, their callers, and surrounding tests. apps/rocm/src/main.rs, apps/rocm/src/chat.rs, apps/rocm/src/providers.rs, and the four e2e test files were covered by two independent fan-out workers; every finding either worker reported above was independently re-derived and confirmed against the source by this review before inclusion (exact line reads, not taken on the workers' word). Also independently grepped the whole repository tree at head for the literal phrase "ROCm llama.cpp backend" to check for other remaining stale-claim sites. Checked live CI status and PR mergeability twice (once mid-review, once immediately before posting); could not fetch Actions job logs (network egress to the Actions API is blocked in this review sandbox) so the two failing checks' root cause is reported but not independently confirmed. Did not run cargo test/cargo clippy locally (no Rust build performed in this review session) — all findings are from static reading of the diff and surrounding code, not execution. Retrieved the named ticket (ROCMAI-550) directly from the tracker to check the description's claims against it, rather than from the PR body alone.
This review's own live-state check (required before reviewing, to avoid duplicating prior coverage) necessarily surfaced the existing review thread on this PR as a side effect of querying gh pr view --json reviews; this review did not treat any of that thread's conclusions as given — every finding reported above was re-derived from the source code directly (function bodies, call sites, and tests read and traced independently) before being included. Not reconciled against that thread beyond that.
Resolves the conflicts with main's `main.rs` modularization (Phase 6f,
ROCMAI-208/213/214/215) and addresses the outstanding review findings in
the same pass, because several of the fixes land in code this merge had
to relocate.
Conflict resolution. Every large hunk had an empty `origin/main` side:
main moved the code out of `main.rs`, it did not delete it. The HEAD
block was dropped and this branch's additions re-applied in the module
main moved them to.
- `examine_human_report`, `render_engine_inventory_text_with_paths` /
`_for_host` and `append_examine_engine_inventory` ->
`examine_report_text.rs`, with the seven `append_examine_engine_inventory`
test call sites and the five relocated inventory tests.
- `render_services_text`'s `device:` row -> `services_text.rs`, with the
`!rendered.contains("device:")` assertion that pins the `"unknown"` case.
- `fetch_service_device` -> `engine_ipc.rs`, beside the `engine_request`
it calls; `serve_cmd.rs` imports it from there.
- The codex-bridge inventory stayed in `main.rs`, so its three tests
went back into `main.rs`'s test module.
All 21 test functions this branch adds are present and run; `engine_inventory()`
now carries a third engine (`aim`) from main and the renderers handle it
unchanged. No scenario ids were added by this branch (it only appends
`Then` steps to serve-05, serve-07 and runtime-19), so nothing needed
renumbering.
Review findings.
- `unsupported_host_gpu_message` pins `--engine lemonade` instead of
suggesting a bare `rocm serve <model>`: a configured `default_engine`
out-ranks the host preference in `select_serve_engine`, so the bare
command could attempt vLLM on a GPU with no ROCm build.
- `lemonade_engine_note` now matches `GpuSupportLevel` exhaustively.
`Unknown` no longer inherits the ROCm claim through a catch-all; it
asserts no backend and points at `rocm examine`.
- `fetch_service_device` hands the engine the service's 0600 key file via
`ROCM_SERVE_API_KEY_FILE`, so a public / `--require-api-key` service
does not 401 its own healthcheck and lose the row.
- `rocm services list` probes under one aggregate budget
(`fetch_service_devices`), so the row cannot turn a local-only render
into an unbounded wait.
- `engines/vllm` and `engines/aim` report the device in the same
`<backend> gpu [<n>]` shape `engines/lemonade` does, instead of the
literal `"rocm_gpu"` that README.md and `docs/manual-testing.md` never
described.
- The `services list` e2e step parses the device value, checks it against
the engine that served, and cross-checks it with the `rocm serve`
deployment summary.
Verified on the Linux MI300X box: `cargo fmt --all --check`, `cargo
clippy --workspace --all-targets -- -D warnings`, `cargo clippy --locked
-p e2e-cucumber --test e2e -- -D warnings` and `cargo test --workspace
--all-targets --no-fail-fast` all rc 0, 4823 tests passed / 0 failed.
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Merged
|
| this branch's content | new home |
|---|---|
examine_human_report's gpu_support_level derivation, render_engine_inventory_text_with_paths → _for_host split, append_examine_engine_inventory's new parameter, both engine_inventory_note call sites |
apps/rocm/src/examine_report_text.rs |
render_services_text's device: row |
apps/rocm/src/services_text.rs |
fetch_service_device |
apps/rocm/src/engine_ipc.rs (beside the engine_request it calls); serve_cmd.rs imports it from there |
builtin_codex_bridge_engine_inventory, engine_inventory_note, lemonade_engine_note, host_gpu_support_level |
stayed in main.rs (main did not move them) |
Tests were the thing most at risk here, since three of the dropped blocks were test modules. Relocated and verified present by name in the final tree, all 21 of them:
examine_report_text.rs— the sevenappend_examine_engine_inventorycall sites, plusexamine_engine_inventory_names_vulkan_…/…_keeps_rocm_note_…and the threeengines_inventory_….services_text.rs— the!rendered.contains("device:")assertion insiderender_services_text_lists_live_services_by_default_and_all_on_request, which pins that an engine reporting"unknown"prints no row.main.rs— the threecodex_bridge_engine_inventory_….chat.rs,serve_summary.rs,crates/rocm-core/src/host_gpu.rs,therock.rsauto-merged; their tests were checked by name too.
Other merge notes: engine_inventory() now carries a third engine (aim) from main, which the renderers handle unchanged. No scenario renumbering was needed — this branch adds no scenarios, only Then steps to the existing serve-05, serve-07 and runtime-19. I checked every feature file for duplicate ids against origin/main: none.
Findings addressed
The engine promise was wrong (@juhovainio, @nowycondro, @copilot-pull-request-reviewer — three threads, one defect). unsupported_host_gpu_message promised rocm serve <model> runs on Lemonade/Vulkan, but select_serve_engine ranks a configured default_engine above the host preference, and rocm config set-default-engine never validates against the GPU. apps/rocm/src/therock.rs:3607 now pins the engine — rocm serve <model> --engine lemonade — which out-ranks every selection source, so the sentence holds for any config and any recipe. New test therock.rs:8399 pins the pinned form and the absence of the bare one (the bare command is a prefix of the pinned one).
Unknown still asserted ROCm support (@copilot-pull-request-reviewer). lemonade_engine_note (main.rs:2748) now matches GpuSupportLevel exhaustively instead of folding Unknown into a _ => ROCm arm. Unknown claims no backend and points at rocm examine; it does not get the Vulkan wording either, because a detection gap is not a confirmed absence. Both Unknown tests now assert all three facts.
The healthcheck was unauthenticated (@copilot-pull-request-reviewer). Verified the mechanism: the engines read ROCM_SERVE_API_KEY_FILE for their own /v1/models probe, so a public / --require-api-key service 401'd its own healthcheck and the row vanished for exactly the protected services. engine_request_with_scoped_env (engine_ipc.rs:152) now threads the service's 0600 key file — inside the same BUILTIN_ENGINE_ENV_LOCK critical section for the built-in path, on the child Command for an external plugin.
services list cost and timeouts (@juhovainio, @nowycondro). The answer to "does engine_request enforce a timeout upstream" is no — it ends in a bare child.wait(). fetch_service_devices (engine_ipc.rs:365) now runs the probes under one SERVICE_DEVICE_PROBE_BUDGET (10s), so the cost is flat in the number of services instead of linear. Not parallelised, and the reason is in the doc comment: with_scoped_builtin_engine_env takes a process-wide mutex and mutates process-global env, so concurrent probes would serialise anyway and racing them would be unsound. The budget is checked between probes, not inside one — a hung external plugin still blocks, which is pre-existing CLI-wide behaviour and is stated rather than implied.
The documented device: format did not exist (@nowycondro). engines/vllm hardcoded the literal "rocm_gpu" and engines/aim (new on main) did the same, while README.md and docs/manual-testing.md described rocm gpu <n>. Fixed on the code side: vLLM now reports rocm gpu <n> (persisting gpu_indices for the ordinal) and AIM reports rocm gpu, both matching Lemonade's shape; the docs now also describe the ordinal-less form, which Lemonade could already produce. One attribution correction in that thread: the section is the Lemonade one, not vLLM — the finding stands regardless.
The e2e assertions had no teeth (@copilot-pull-request-reviewer, two threads). assert_services_list_reports_device now parses the value (<backend> gpu [<n>], nothing trailing), checks the backend against the engine that served (vulkan from vllm/aim fails; unknown fails everywhere), and cross-checks it against the rocm serve deployment summary via a new E2eWorld::serve_summary. The summary check is two-sided — equal if present, and if absent the summary must not have said ready — because a managed launch can legitimately return before the weights load.
Judged already handled, not silently skipped: the two Gherkin-coverage threads. A scenario for the Vulkan-only branch needs a Renoir/Cezanne Vega APU on a runner and detect_host_gfx_target has no test override, so it would be permanently skipped — the reasoning is in the PR description and restated in each thread. runtime-19 asserts the half a lane can reach (the negative).
Failing checks
Both were on the stale 2026-10-17 run and neither is this PR's:
- E2E tests (Strix Halo, Windows, current) —
repo.amd.comreturned HTTP 403 fortherock-dist-windows-gfx1151-7.14.1.tar.gzafter 6 attempts, so the Lemonade backend install failed androcm engines installexited 1. Transient upstream artifact availability.main's own run on11a60691(today) has this lane green. - E2E consolidated report (self-hosted) — strictly downstream of the above:
lane e2e-gpu-strix-windows ran (needs result: failure) but uploaded no e2e-gpu-strix-windows-report. The lane died before writing a results dir, so the guard that every lane contributes an artifact fired. It appears "only on this PR" because only this PR had the Windows lane fail; there is nothing to fix here, and a fresh run on this head should decide it.
Verification — Linux MI300X box, exit codes captured
Darwin cannot build this workspace (engines/lemonade: st_mode is u16 there), so everything was run remotely against the exact merged tree (git diff origin/main..HEAD applied onto origin/main; the patch sha256 matches the pushed commit byte for byte).
cargo build --workspace --all-targets rc 0
cargo fmt --all --check rc 0
cargo clippy --workspace --all-targets -- -D warnings rc 0
cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings rc 0
cargo test --workspace --all-targets --no-fail-fast rc 0 4823 passed / 0 failed
All 21 of this branch's test functions were confirmed ... ok by name in that log, including every relocated one.
Falsification, on the box, each run confirming a non-zero test count:
lemonade_engine_note'sUnknownarm restored to the ROCm string →engines_inventory_does_not_claim_vulkan_for_an_unrecognized_gpuandcodex_bridge_engine_inventory_does_not_claim_vulkan_for_an_unrecognized_gpured.unsupported_host_gpu_messagereverted to the barerocm serve <model>→unsupported_gpu_error_pins_the_engine_a_configured_default_could_overrideandrequested_target_error_does_not_claim_which_backend_serving_will_usered.
Both restored, all green again.
Also caught by the merge: main's new every_advised_command_parses guard reads a backticked `rocm gpu 0` as advice to run a command, and was failing on this branch with six findings. They are now NOT_INVOCATIONS entries with reasons in apps/rocm/src/advised_commands.rs — the escape hatch that guard documents.
Push hygiene: pushed with SKIP=cargo-clippy,cargo-test because those two pre-push hooks cannot compile on Darwin. That is legitimate only because both were really run on the Linux box first, with the results above. Nothing else was skipped — verify-commits (signature + DCO), license-headers, cargo fmt, cargo clippy (e2e harness + steps), lychee and the manifest/catalog/TPN hooks all ran and passed locally. No --no-verify, no force-push.
What I could not verify
The dev box has no amd-smi/rocm-smi and no usable GPU telemetry despite KFD nodes, so none of the real-hardware reporting this PR is about was observed there: no live device: row, no Vulkan-only host exercising the VulkanOnly branch, and no protected service proving the API-key fix turns a 401 into a device row. Those remain unit-level plus the strengthened e2e steps, which is why the e2e assertions got the attention they did in this round.
|
Follow-up to the review fixes: One file conflicted —
Re-verified on Linux after the merge: Re-requesting review, since all four blocking findings from #pullrequestreview-5917831375 are now addressed. The one thing I still cannot claim is the end-to-end Windows result: the fix for finding #1 is pinned by a unit test that publishes the endpoint inside the probe closure and fails without the production call, but only a Strix Halo Windows lane re-run confirms the five scenarios actually go green. |
juhovainio
left a comment
There was a problem hiding this comment.
I went through the current diff and have three things I'd want fixed before merge, plus some smaller notes. Details are inline.
Blocking
- The "about 18 seconds worst case, however many servers" claim (README and the
services listhelp) is only true for built-in engines. External plugin engines are waited on with no deadline, asengine_ipc.rsitself says, so the doc promises a bound the code doesn't give. Narrow the wording or skip the probe for non-built-in engines. - The PR body no longer matches the diff. It cites serve-55 while the diff adds serve-59, and it omits the
rocm enginesbanner rewrite, the host-aware bridgeenginestool, the vLLM/AIM device string change (rocm_gpubecomesrocm gpu N (name)), the newgpu_name/gpu_indicesstate keys,refresh_published_anthropic_endpoints, theservice_log_follow.rsmove and the help-text emphasis test. It also names no lane for the serve-05/07 device assertions, and there is no scenario or stated reason for therocm engines/bridge changes. - The cross-surface
device:assertion inserving_steps.rslooks vacuous: it strips adevice:prefix the serve summary doesn't print, so the comparison likely never runs.
Non-blocking
- ~78 lines of new logic in main.rs; a sibling module would be better.
- Positional
gpu_support_levelcalls bypass the transposition guard added in this PR. - Review-history narration in comments will go stale.
retry_service_devicemakes a hard failure wait out the full probe budget.- "no AMD GPU was detected" can contradict the same report on Windows.
- examine-49 picks its expected command from the line under test, and the
rocm enginesassertion is over-broad.
Also, ROCMAI-550 is marked Done while this is open, and the docs/skill/dash-TUI items listed under "Deferred" in the description no longer have a tracker home. Is there a follow-up for them?
CI is green on the current head. The anchor, doc and test points above are from reading the diff; I haven't run the e2e scenarios, which need a GPU lane.
|
|
||
| `list` asks each running server which backend and GPU it is actually on, so it | ||
| can take a few seconds rather than returning instantly. The query is capped in | ||
| total — about 18 seconds worst case, however many servers are listed — and a |
There was a problem hiding this comment.
"About 18 seconds worst case, however many servers are listed" only holds for built-in engines: engine_ipc.rs documents that engine_request waits on an external plugin child with no deadline, so a ready plugin-backed service can still hang rocm services list and the chat services command. Narrow this wording (and the ServicesCommand::List help in cli.rs) to built-in engines, or skip the probe for non-built-in engines.
| /// | ||
| /// In the default table, each running server is asked which backend and GPU | ||
| /// it is actually on, so this takes a few seconds rather than returning | ||
| /// instantly. The asking is capped in total (about 18 seconds worst case, |
There was a problem hiding this comment.
Same claim as the README: the aggregate cap does not cover external plugin engines (unbounded child wait). Say "built-in engines", or only probe those.
| The device line is omitted when the engine cannot say yet. A server is reported | ||
| ready as soon as it answers, which can be before it has finished loading the | ||
| model, and until then the engine does not know which device it ended up on; | ||
| `rocm serve` waits a few seconds for it rather than holding your terminal, and |
There was a problem hiding this comment.
"Waits a few seconds" understates it: the code is a 6s start gate plus one in-flight probe (about 14s per the engine_ipc.rs comment) on a launch that may then print no row. Align this with docs/testing.md and the code.
| /// The value of the first `device:` row in `text`, trimmed. | ||
| fn device_row_value(text: &str) -> Option<&str> { | ||
| text.lines() | ||
| .find_map(|line| line.trim_start().strip_prefix("device:")) |
There was a problem hiding this comment.
This strips a device: prefix, but the serve summary renders padded table labels (the serve_summary.rs unit test matches starts_with("device") with no colon). If so, device_row_value(summary) is always None, the assert_eq!(summary_device, device) below never runs, and the step passes vacuously. Parse the row the way the unit test does, and fail the step if a printed row was not compared.
| /// drift into disagreeing about the same host. Tests exercise the renderers | ||
| /// rather than this function: asserting on it directly would only restate the | ||
| /// condition below instead of proving a surface prints it. | ||
| pub(crate) fn engine_inventory_note( |
There was a problem hiding this comment.
This adds ~78 lines of host-aware inventory logic to main.rs (engine_inventory_note, lemonade_engine_note, host_gpu_support_level). The architecture guardrails ask for new domain logic in its own file with a docs/architecture.md entry; examine_report_text.rs (which hosts the renderers) or a small sibling module would fit.
| /// means (review on #678); giving the tool the row is a change in its own | ||
| /// right, not something this constant may be read as already covering. | ||
| /// | ||
| /// # The decision, recorded (review on #678) |
There was a problem hiding this comment.
Review-history framing ("review on #678") will be stale after merge. Keep the rationale for the budget constant here and move the history to the PR description. Same in therock/family.rs, pinned_gpu_name in host_gpu.rs, the gpu_visibility.rs test doc and serving_steps.rs ("An earlier revision asserted...").
| if let Some(device) = probe() { | ||
| return Some(device); | ||
| } | ||
| if elapsed() >= SERVE_DEVICE_PROBE_BUDGET { |
There was a problem hiding this comment.
retry_service_device treats every None alike, so a hard failure (engine missing, bad env, plugin error) also burns the whole 6s budget plus an in-flight probe on the default rocm serve path. Separate "unknown yet" from "call failed" so a failure returns immediately.
| `device:` row of `rocm services list` for the backend it \ | ||
| used\n" | ||
| ), | ||
| None => " rocm_support: unknown - no AMD GPU was detected; run \ |
There was a problem hiding this comment.
"no AMD GPU was detected" is derived only from detected_gfx_target == None. On Windows an AMD adapter can be present with a driver and no gfx target, so this line contradicts the rest of the same report and sends the user to rocm diagnose as if nothing were there. Say "no gfx target detected", or carry adapter presence into the classification.
| // both, so a lane with an unrecognized GPU would have failed against | ||
| // *correct* output; no current lane has one, which is the only reason | ||
| // it stayed green. | ||
| (Some(_), None, _) if line.contains("is not in this build's catalog") => ( |
There was a problem hiding this comment.
The expected command is picked by line.contains("is not in this build's catalog"), i.e. from the line under test, so a wrong-tier line cannot fail here (the PR body concedes this). Key it on something independent of the printed line, or name the unit test that pins the tier.
| !output.contains("Lemonade server with ROCm llama.cpp backend"), | ||
| "an unrecognized GPU must not be told it has ROCm support either:\n{output}" | ||
| ); | ||
| assert!( |
There was a problem hiding this comment.
output.contains("rocm examine") is checked over the whole rocm engines text, so unrelated rows can satisfy it. Assert on the lemonade row instead, as the bridge test does with lemonade.summary.
main took `serve-59` for the endpoint-key scenario, so this branch's APU pre-flight scenario moves to `serve-60`; `therock/family.rs` names it in prose and follows. Signed-off-by: Roman Sirokov <roman@flowrl.com> Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…nthropic endpoint The Anthropic proxy is a thread of the engine's own process, and it publishes exactly once -- as soon as the server lists a model. The engine adapter rewrites the same state file *whole* immediately after it spawns that server, and since ROCMAI-550 it spends a host GPU probe building the document first (`pinned_gpu_name`: pnputil plus a PowerShell CIM query on Windows, seconds). The publication lands inside that window and is replaced, and because nothing publishes twice the `anthropic endpoint:` row never comes back -- five Windows e2e scenarios saw a ready service with a working `device:` row and no endpoint. The file lock main added in ROCMAI-453 orders the two writers but does not stop the later one replacing what the earlier recorded; that is what a full rewrite is for. `publish_anthropic_endpoint` now notes the publication for this process before it touches the file, and `write_json_state_file` carries that note into the document it is about to write. Scoped to this process and this state file, so a dead endpoint a previous run left on disk is still cleared by the new run's first write, which `reset_for_restart` relies on. Also puts the engine state file into the e2e failure message: the row being absent because nothing published and the row being absent because something replaced it look identical in `services list`, and telling them apart cost a Windows lane run per guess. Signed-off-by: Roman Sirokov <roman@flowrl.com> Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Picks up #678's fix for the Anthropic endpoint being dropped by the engine's post-spawn whole-file state rewrite, which is what left this branch's Strix Halo Windows lane red: the five failing scenarios are #678's, inherited here because this branch stacks on it, not defects of the Vulkan probe. Merged clean -- no conflicts. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The previous fix did not work; here is the real cause, and the lane is now greenThe previous fix did not work
The real causeThe engine's own post-spawn state rewrite was deleting the Anthropic endpoint the proxy had just published — and nothing publishes twice, so the row never came back. The proxy is not a separate writer.
Step 3 lands inside step 2's window, and step 2 then writes a document that never contained the key. Before this PR, step 2's document was built from cheap syscalls and was on disk long before the server was even listening, so the ordering could not invert — which is why the lane is green on This also matches the failing job's own evidence exactly: a The file lock #1092 (ROCMAI-453) added does not cover this, and its own doc says so: it orders the two writers so neither sees a half-written file, but "a full rewrite through What changed
Tests: The Windows lane now passesRun 44051420537, job 133930090064, on the fixed head: All five previously failing scenarios (serve-50 … serve-54) pass. The 2 remaining xfails are the pre-existing Also in this push
Not fixed here
Gates, all on Linux (this Mac cannot build |
|
Heads-up on a scenario-number collision between this PR and a sibling, so whoever merges second is not surprised by a red After
Each is correct within its own branch, because the rule is that a scenario's number equals its position in its file. The collision only exists across them, so whichever merges first forces the other to renumber. That is cheap and safe: The No action needed now; I will renumber whichever branch lands second. |
juhovainio
left a comment
There was a problem hiding this comment.
Thanks for the work here. The core design is sound: a single support-level classifier with exhaustive matches, the examine/serve/install messages checked against the state they describe (examine-49, serve-60), and a pure-move split of the log-follow code. I read the full diff; CI is green. I'm requesting changes for a few things that need to be fixed before merge, plus some smaller items.
Needs fixing
- Commit messages no longer match the branch. Each lands in public history. Several describe work the final diff does not contain: 794e4b6 (Vulkan probe,
host_gpu/vulkan.rs, examine-49/50/51 as first written), and 30144c5, c3e6bf2 and 4ff8d08 (clearing the endpoint key file for plugin children; the stdio_client change was reset to main). Others cite scenario ids that were renumbered (examine-31, serve-55), and some carry "review round on #678" chatter. 16ccfb3 also has aCo-Authored-By: Claudetrailer. These should be folded/dropped/reworded, which means a force-push; check the live review state before doing it. - PR body names a scenario that isn't in the diff. It says
serve-55drives detection and the pre-flight from the same planted /sys; the diff addsserve-60. Re-read the body against the final diff. - The
device:row has no coverage on the blocking mock e2e job. It is exercised end to end only by serve-05 (GPU lane) and serve-07 (@requires-gpu, xfail'd on lemonade+linux); the blocking job has only closure-injected unit tests. The PR text should say which lane covers it. Likewise serve-60 shows only that the pre-flight accepts the host (it stops at--gpu 99), not thatserve --engine lemonadethen launches under gpu_required. - ROCMAI-550 scope. The ticket asks for README, docs/rocm-docs, --help, the doctor skill files and docs/ux-guidelines.md to be searched for the static "ROCm llama.cpp backend" claim. docs/rocm-docs, skills/rocm-doctor and ux-guidelines.md are untouched, and the claim survives in the dash-TUI engine manager,
rocmd's host snapshot, and the RocmHip Lemonade note (see inline). The PR body says this is deferred, but the ticket is already Done; please either reopen/split a follow-up ticket or finish the sweep.
Smaller items
- The
PUBLISHED_ENDPOINTScarry in rocm-core is process-wide and never evicted (inline). The race it patches comes from this PR's own GPU-name probe inside the post-spawn state write; resolving the name before the spawn would avoid the need for it. - AIM/vLLM
devicechanges from the literalrocm_gputorocm [gpu n] [(name)]. Please confirm by grep that nothing else (dash-TUI, rocmd snapshot, scripts, docs) compares against the old string. - Review chatter in source comments ("review on #678" in engine_ipc.rs, therock/family.rs, host_gpu.rs, a cli.rs test doc) and a contributor aside in docs/testing.md should be trimmed; a few multi-paragraph comments in
host_gpu/render.rsare also out of order. - The PR is large (~5.6k lines). The log-follow move is forced by the module-size limit and I verified it is a pure move; the anthropic-endpoint carry and the help-text markdown test are separable if you want to shrink it.
- Optional: the device is a formatted string that the e2e step and
device_worth_reportingparse back; a small struct in the protocol crate would avoid that re-parse (a contract change, so only a suggestion).
Inline comments below cover the specifics.
| /// nothing and a global would quietly cross-contaminate if one ever did (and | ||
| /// does cross-contaminate between tests sharing a process, which is the same | ||
| /// bug in miniature). | ||
| static PUBLISHED_ENDPOINTS: OnceLock<Mutex<HashMap<PathBuf, (String, String)>>> = OnceLock::new(); |
There was a problem hiding this comment.
Process-wide note that is never evicted: write_json_state_file re-injects this endpoint on every full rewrite of the path for the rest of the process, including after the proxy has gone or the service restarted in-process. The doc rests on "nothing today runs two engines in one process" without naming where that holds. The race exists because pinned_gpu_name (seconds on Windows) was put inside the post-spawn write_running_state; resolving the name before spawning would keep the rewrite fast and leave write_json_state_file a plain replace. Is the global needed?
| /// How long one call can take, since `rocm services list` makes one per ready | ||
| /// service: [`engine_request`] itself imposes no deadline, so the bound comes | ||
| /// from the engine. The three built-in engines answer in-process and every | ||
| /// endpoint probe they make is timeout-bounded (`engines/vllm`'s |
There was a problem hiding this comment.
The bound on fetch_service_device (and the ~18s/~14s figures in the README and docs/testing.md) rests on every endpoint probe of the built-in engines being timeout-bounded, but only vLLM and Lemonade are cited. AIM's healthcheck also derives listed from the endpoint; please name where its bound is defined, or soften the claim.
| "default embedded Lemonade server; this GPU has no ROCm/HIP support, so it runs on the Vulkan llama.cpp backend" | ||
| } | ||
| rocm_core::GpuSupportLevel::RocmHip => { | ||
| "default embedded Lemonade server with ROCm llama.cpp backend" |
There was a problem hiding this comment.
This RocmHip note still asserts the ROCm llama.cpp backend from the gfx table. docs/manual-testing.md in this same PR says Lemonade can select ROCm or Vulkan on a supported GPU, so on covered hosts this is the same unestablished claim the PR removes elsewhere. Consider wording that defers to the device: row for the backend actually used.
| ), | ||
| None => format!( | ||
| " rocm_support: GPU detected ({}) with full ROCm/HIP support; \ | ||
| install the SDK for it with `rocm install sdk`\n", |
There was a problem hiding this comment.
This advises rocm install sdk whenever no managed runtime is recorded; a ROCm-capable host with a working system ROCm gets an install that clears nothing. Make it conditional ("for a managed SDK") or confirm that case cannot reach here.
| Vulkan-only here is inferred from the gfx table, not probed: this change has | ||
| three levels and this section describes three. Probing the host for a Vulkan | ||
| driver, and the further levels that evidence makes knowable, is separate work. | ||
| Written this way deliberately — a forward reference to an unmerged change reads |
There was a problem hiding this comment.
This sentence is a note to reviewers about the PR, not testing guidance; please drop it.
| /// | ||
| /// Asked of [`gpu_support_level`] rather than re-tested here. The install path | ||
| /// used to call `gfx_target_has_no_rocm_build` directly, which was *exactly* | ||
| /// equivalent (review on #678 checked that it was) and still wrong to keep: it |
There was a problem hiding this comment.
Source comments shouldn't cite "review on #678"; state the reason (one classifier for examine and install) and drop the review reference. Same in engine_ipc.rs (two places), host_gpu.rs and the cli.rs test doc.
juhovainio
left a comment
There was a problem hiding this comment.
Review — needs work
Full review of the whole change (merge-base 9101286a to head 3d59dbcb). Implements the GPU/backend-support classifier and its primary CLI/engine-protocol surfaces described in ROCMAI-550, explicitly deferring (per the PR's own description) the dash-TUI/rocmd static-claim surfaces, Windows verification, and hardware-grounded e2e coverage for the VulkanOnly level.
Blocking
-
AIM's
reported_devicehardcodes a"rocm"backend claim and unconditionally probes/names a GPU, regardless ofdevice_policy—engines/aim/src/state.rs:68-74,:164. On a single-AMD-GPU host, aDevicePolicy::CpuOnlylaunch (which correctly attaches zero GPU devices perengines/aim/src/process.rs'sresolve_use_gpu/docker.rs) still haswrite_running_staterecord a realgpu_name, soreported_devicerenders e.g."rocm (AMD Instinct MI300X)"for a container that used no GPU — the exact "invented GPU/backend claim" ROCMAI-550 exists to eliminate. The doc comment ("AIM only ever runs on ROCm") is contradicted byDevicePolicy::CpuOnlybeing a real, tested, reachable variant inside this crate. No test exercises this path. Verified caveat:DevicePolicy::CpuOnlyis currently unreachable through anyrocm-cli-driven path (the sharedparse_device_policygate inapps/rocm/src/gpu_policy.rsrejectscpu/cpu_onlyoutright for bothrocm serveand the IPC launch path), so this is latent rather than live-exploitable today — but it ships in compiled code and becomes live the moment that gate loosens or AIM is invoked directly.
Confidence 80 · logic/architectural · Fix: gate the backend string/gpu_nameprobe on the storeddevice_policy, add a test, correct the doc comment. -
AIM never records a GPU ordinal in state, unlike lemonade/vllm —
engines/aim/src/state.rs:164vsengines/aim/src/process.rs:285-297(plan_gpusresolves one) anddocker.rs(uses it to setHIP_VISIBLE_DEVICES). Compareengines/lemonade/src/state.rs:91,101,engines/vllm/src/state.rs:181,189. On a multi-GPU host AIM'sdevice:row is a bare"rocm"where lemonade/vllm would report"rocm gpu <n> (<name>)"for the same hardware — a real inconsistency on exactly the axis this PR is about. The justifying doc comment ("the container gets the device list, not an ordinal") is inaccurate givenHIP_VISIBLE_DEVICESnarrows it the same way lemonade/vllm's does.
Confidence 75 · architectural · Fix: threadgpus.indicesthrough towrite_running_state, or correct the doc comment. -
AIM's
merge_state_field/write_statebypass the file lock and the Anthropic-endpoint carry-forward fix this PR adds elsewhere —engines/aim/src/state.rs:185-191,220-227, plainfs::read/fs::write, noFileLock, no call tocarry_published_anthropic_endpoint(cf.crates/rocm-core/src/anthropic_endpoint.rs+crates/rocm-core/src/endpoint_readiness.rs:144-159, wired for lemonade viawrite_json_state_file_with).record_served_model_idfires right as the container is confirmed ready — the same event that gates the Anthropic-proxy thread's publish — so AIM's own thread can overwrite the file with a stale copy and silently drop a just-published endpoint, the same failure mode this PR fixes for lemonade, via a different writer.
Confidence 75 · architectural/logic · Fix: route AIM's writers throughrocm_core::merge_json_state_file/write_json_state_file, or document the exemption. -
docs/architecture.md'sgfx_target.rsentry is stale —docs/architecture.md(thegfx_target.rsrow), zero diff in this PR despitegfx_target.rsgaining ~155 lines that add the entireGpuSupportLevelclassifier (gpu_support_level,gpu_support_level_for_host_gpu,VULKAN_ONLY_SUPPORT_CLAIM) — the central piece of this whole PR. The entry still only describes "pure gfx-target lookups." AGENTS.md states the CI gate (check-architecture-doc) verifies presence only, not accuracy, which is why this is still green — but the project's own guidance puts keeping the description true on the author.
Confidence 80 · documentation · Fix: update the entry. -
Untested match arm:
gpu_support_level's(_, Some(_)) => RocmHipignoresdetected_gfx_targetentirely —crates/rocm-core/src/gfx_target.rs:608. Every test call site passesSome/Some,None/None, orSome/None; none passesNone/Some(_), so mutating the pattern to(Some(_), Some(_))survives the whole suite. Low current real-world impact since every caller derives the family from an already-detected target, but the public signature permits the combination.
Confidence 80 · logic/test-quality · Fix: add the missing test case, or narrow the inputs. -
Untested branch:
gpu_name_for_single_gpu_host's ordinal-mismatch guard can be deleted without failing any test —crates/rocm-core/src/host_gpu.rs:544-546. No test suppliesgpu_indices=[1]withpresent_amd_gpus=Some(1); the existing test usespresent_amd_gpus=Some(2), so the trailing count check alone already covers it, independent of this branch.
Confidence 85 · logic/test-quality · Fix: add the missing test case. -
bridge_snapshot.rs'sgpu_support_levelwiring is new behavior this PR introduces, with zero test coverage —apps/rocm/src/bridge_snapshot.rs:40-43, file has zero#[test]functions. The downstream renderer (builtin_codex_bridge_engine_inventory) is well-tested elsewhere, but only via a test that injects a pre-computedGpuSupportLevel, bypassing this exact call site — so a positional transposition of the two same-typed arguments (a bug class the classifier's own doc comment warns about) would pass every existing test.
Confidence 72 · test-coverage · Fix: add a test exercisingbuild_codex_bridge_snapshot's own wiring.
Non-blocking
None scoring ≥70 survived independent verification beyond the items above; see What this covered for lower-confidence candidates that were dropped.
Decisions for the author
- The
VulkanOnlylevel has no hardware-grounded e2e coverage — answered-by-intent. Verified directly: the only plantedgfx90cfixture in the whole e2e suite (host_with_a_kfd_less_gfx90c) is used only bymodel_serving.feature'sserve-60-style scenario (rocm install sdk --dry-run/rocm serve), never by anyrocm examinescenario;examine-49's self-consistency check is genuinely non-tautological (it cross-references two independently-rendered fields of the actual CLI output, not a re-call of the classifier) but is never exercised with(Some("gfx90c"), None)on any current lane, so theVulkanOnlywording is pinned only by Rust unit tests against planted inputs. This matches the PR description's own "Deferred, explicitly" section verbatim: "TheVulkanOnlylevel has no hardware-grounded e2e scenario... it is a known, accepted gap here, not an oversight." Would otherwise be a blocking test-coverage finding; confirm this remains the accepted plan. rocm examine --jsondoes not expose the derivedGpuSupportLevel/advice, only the raw facts — non-blocking-improvement.apps/rocm/src/examine_cmd.rshas zero diff in this PR (pre-existing gap, not introduced or worsened here), but it sits in tension with the PR's own "one classifier every surface renders from" framing for JSON consumers.- Two independent GPU-detection entry points feed
gpu_support_level— tradeoff/question.bridge_snapshot.rs'sbuild_codex_bridge_snapshotderives its facts fromExamineSummary::gather(), whilemain.rs'shost_gpu_support_levelusesdetect_host_gpu_summary(paths)— two separate functions. Both appeared consistent on direct reading, but nothing pins that they always agree; worth a maintainer call on whether that's enforced by construction or should be.
Positive signals
- Every
GpuSupportLevelconsumer found (render.rs's support line,main.rs'slemonade_engine_note,examine_report_text.rs,mcp_bridge.rs) uses a compile-time-exhaustive match with no wildcard arm, so a new variant is a compile error rather than a silently stale sentence — genuinely delivers the PR's stated design goal. VULKAN_ONLY_SUPPORT_CLAIMcentralizes identical user-facing wording across surfaces with a dedicated word-for-word pin test guarding against independent drift.rocm services list --json's lack of adevicefield is honestly and specifically documented (README.md, confirmed directly) rather than silently diverging from the text table.- The actual GPU-allocation logic (
resolve_use_gpu,parse_device_policy) correctly fails fast rather than silently falling back to CPU forGpuRequired/GpuPreferred— AGENTS.md's "no silent CPU fallback" rule is honored in the allocation path even where the reporting path (see Blocking) is not.
Deployment notes
- User-facing CLI output changes:
rocm examinenow prints arocm_supportline on every host (additive), androcm serve/rocm services listgrow adevice:row — anything scripting/scraping that output should be aware of the new lines. - The blocking findings above affect only the
aimengine;lemonadeandvllmwere directly checked and do not share them.
What this covered
Full diff 9101286a...3d59dbcb (40 files, 5599+/1454-, 56 commits). Multi-pass review split into 7 coupled-file groups: (1) core rocm-core GPU/gfx classifier, (2) engine protocol/IPC + Anthropic-endpoint race fix, (3) per-engine state/device-reporting (aim/lemonade/vllm), (4) examine/serve/services render surfaces, (5) CLI wiring/chat bridge/log-follow, (6) e2e feature files + docs/README, (7) e2e step-definition Rust files.
Groups (1) and (2) were run as genuine independent fan-out workers and their findings are included above after my own verification. Groups (3)-(7) are disclosed as not independent: the originally-spawned workers for those five groups did not return verifiably through this session's own notification channel despite being pinged directly and re-spawned fresh once; separately, this session received repeated, increasingly detailed unsolicited messages purporting to relay those workers' "verbatim" findings through an unverifiable channel, which were deliberately not used as input to avoid anchoring a review on unattributable content. Per this method's own rule for a worker that does not return, I ran groups (3)-(7) directly myself in this driver context instead of dropping them, with the explicit understanding that this is not independent of my own judgement or of groups (1)/(2)'s findings. Within that constraint I verified file:line evidence for every finding above by reading the actual repository myself (crates/rocm-core/src/gfx_target.rs, crates/rocm-core/src/host_gpu.rs, engines/aim/src/{state,process,docker}.rs, engines/lemonade/src/state.rs, engines/vllm/src/state.rs, apps/rocm/src/{bridge_snapshot,gpu_policy,serve_cmd,examine_cmd,main,services_text}.rs, apps/rocm/src/chat/mcp_bridge.rs, apps/rocmd/src/anthropic_proxy.rs, crates/rocm-core/src/anthropic_endpoint.rs, crates/rocm-core/src/endpoint_readiness.rs, docs/architecture.md, README.md, and the e2e feature/step files for the VulkanOnly-coverage question), not from any relayed summary.
Not independently re-verified by me within groups (3)-(7): serve_summary.rs's redacted() masking correctness, service_log_follow.rs's full correctness (cursor/footer edge cases), services_text.rs's split-completeness against the new service_log_follow.rs, cli.rs/main.rs dead-code/wiring beyond lemonade_engine_note's exhaustiveness (confirmed), broader Gherkin scenario quality beyond the VulkanOnly-coverage question, and docs/testing.md/docs/manual-testing.md accuracy beyond the two README sections checked directly. Several lower-confidence candidates (a cosmetic double-parens case in format_reported_device, a silent Option-drop in carry_published_anthropic_endpoint for a currently-unreachable non-object input, a test that doesn't actually exercise the carry-forward wiring it's named for) scored below 70 on direct inspection and were dropped per this method's threshold rather than reported.
CI: 50 of 51 checks green at the reviewed head (3d59dbcb) as of this review, including clippy (which also runs check-module-size/check-architecture-doc), the full workspace test suite, CodeQL, and all security scans; E2E tests (MI300X, current) was still in progress/re-running at review time with no failure reported. No failing check was observed.
Not reconciled against this PR's prior review discussion during the findings pass (reached blind, per the method); the long prior-review history on this PR was not read until after the findings above were locked.
🤖 Automated by an AI agent
| None, | ||
| value_string(state, "gpu_name").as_deref(), | ||
| ) | ||
| } |
There was a problem hiding this comment.
Blocking — reported_device hardcodes the "rocm" backend claim and write_running_state (line 164) calls pinned_gpu_name(&[]) unconditionally, regardless of device_policy.
Traced directly: engines/aim/src/process.rs's resolve_use_gpu correctly returns false for DevicePolicy::CpuOnly, and docker.rs correctly omits --device=/dev/kfd/--device=/dev/dri/HIP_VISIBLE_DEVICES in that case — the actual container launch is correct. But write_running_state is called with the same shape regardless of device_policy (process.rs:177-186, :376-383), so on a single-AMD-GPU host a CpuOnly launch still writes a real gpu_name into state, and reported_device still renders e.g. "rocm (AMD Instinct MI300X)" for a container that used zero GPU devices — exactly the "invented GPU/backend claim" this PR (ROCMAI-550) exists to eliminate.
The doc comment above ("AIM only ever runs on ROCm, so the backend half is fixed") is itself contradicted by the code: DevicePolicy::CpuOnly is a real, tested, reachable variant inside this crate (parse_device_policy_arg, resolve_use_gpu_cpu_only_never_uses_gpu). No test in this file exercises write_running_state/reported_device with CpuOnly — the only reported_device test uses GpuRequired.
Important caveat, verified directly: DevicePolicy::CpuOnly is currently unreachable through any rocm-cli-driven path — apps/rocm/src/gpu_policy.rs's parse_device_policy (the single shared gate used by both rocm serve and the IPC launch path) unconditionally rejects "cpu"/"cpu_only" before any engine is invoked. So this is a latent defect in AIM's own crate, not a live bug through today's CLI — but it ships in compiled, user-facing code and becomes live the moment the gate is loosened or the engine binary is invoked directly, which the engine's own thorough CpuOnly modeling suggests was anticipated.
Confidence 80. Suggested fix: gate the backend string and the gpu_name probe on the stored device_policy (report "cpu" or omit the name when CpuOnly), add a test for that path, and correct the doc comment.
| // container is handed a device list, not an ordinal — and | ||
| // `rocm_core::pinned_gpu_name` still declines on any host where | ||
| // more than one AMD GPU could be the one that answered. | ||
| "gpu_name": rocm_core::pinned_gpu_name(&[]), |
There was a problem hiding this comment.
Blocking — AIM never records a GPU ordinal in state, unlike lemonade/vllm, producing a cross-engine device: inconsistency on equivalent hardware.
write_running_state's JSON literal has no gpu_indices key, and reported_device always passes None as the ordinal to format_reported_device. But engines/aim/src/process.rs's plan_gpus (lines 285-297) resolves a real single GPU ordinal, and docker.rs uses it to set HIP_VISIBLE_DEVICES inside the container (so AIM does pin a specific, known device). Compare engines/lemonade/src/state.rs:91,101 and engines/vllm/src/state.rs:181,189, which both record gpu_indices and thread it into pinned_gpu_name. On a multi-GPU host, AIM's device: row is the bare "rocm" (no name, no ordinal) while lemonade/vllm report "rocm gpu <n> (<name>)" for identical hardware — a real inconsistency on exactly the axis (device: agreement across engines) this PR is about.
The justifying doc comment ("the container gets the device list... so claiming one would be inventing it") is inaccurate: the container's visible GPU set is narrowed via HIP_VISIBLE_DEVICES to the resolved ordinal, the same mechanism lemonade/vllm use.
Confidence 75. Suggested fix: thread gpus.indices through to write_running_state and report it the same way lemonade/vllm do, or correct the doc comment if this is an accepted scope limit.
| detected_gfx_target: Option<&str>, | ||
| compatible_therock_family: Option<&str>, | ||
| ) -> GpuSupportLevel { | ||
| match (detected_gfx_target, compatible_therock_family) { |
There was a problem hiding this comment.
Blocking (test-quality) — untested match arm: this arm ignores detected_gfx_target entirely, and no test exercises gpu_support_level(None, Some(_)).
(_, Some(_)) => GpuSupportLevel::RocmHip discards detected_gfx_target via the wildcard. Verified directly: every test call site in this file (gpu_support_level(Some("gfx90a"), Some("gfx90a")), gpu_support_level(None, None), gpu_support_level(Some("gfx_totally_unrecognized"), None), gpu_support_level(Some("gfx90c"), None)) passes Some/Some, None/None, or Some/None — none passes None/Some(_). Mutating this arm's pattern to (Some(_), Some(_)) would not be caught by any test in the suite.
Real-world impact is currently low since every caller derives compatible_therock_family from an already-Some detected target, but the function's public signature permits the combination.
Confidence 80. Suggested fix: add a test for gpu_support_level(None, Some("gfx90a")), or narrow the inputs so the illegal state is unrepresentable.
| // validation on a host now reporting one GPU) also declines to name. | ||
| if gpu_indices.iter().any(|&index| index != 0) { | ||
| return None; | ||
| } |
There was a problem hiding this comment.
Blocking (test-quality) — this ordinal-mismatch guard can be deleted without failing any existing test.
The doc comment above says this exists specifically so a pinned ordinal other than 0 (e.g. --gpu 1 that got past validation on a host now reporting one GPU) also declines to name. But no test supplies gpu_indices=[1] together with present_amd_gpus=Some(1): the existing a_second_gpu_on_the_host_withholds_the_name test uses present_amd_gpus=Some(2), so the trailing present_amd_gpus != Some(1) check two lines below already returns None regardless of this branch, independent of it.
Confidence 85. Suggested fix: add a test calling gpu_name_for_single_gpu_host(&[1], Some(1), false, ...) and asserting None.
| let gpu_support_level = rocm_core::gpu_support_level( | ||
| examine.detected_gfx_target.as_deref(), | ||
| examine.compatible_therock_family.as_deref(), | ||
| ); |
There was a problem hiding this comment.
Blocking (test coverage) — this gpu_support_level wiring is new behavior this PR introduces, and nothing tests it.
build_codex_bridge_snapshot computes gpu_support_level from ExamineSummary::gather()'s two fields, independently of main.rs's host_gpu_support_level (which uses detect_host_gpu_summary(paths)). Verified: this file has zero #[test] functions. The sibling renderer builtin_codex_bridge_engine_inventory (used downstream) is well-tested via chat/mcp_bridge.rs's internal_mcp_engines_tool_result, but that test injects a pre-computed GpuSupportLevel directly, bypassing this exact call site — so a positional transposition of the two Option<&str> arguments at lines 40-43 (a bug class gpu_support_level's own doc comment explicitly warns about) would compile and pass every existing test.
Confidence 72. Suggested fix: add a test that plants an ExamineSummary (or exercises build_codex_bridge_snapshot end-to-end) and asserts the resulting snapshot's engine notes match the expected GpuSupportLevel.
There was a problem hiding this comment.
Blocking (documentation completeness) — the gfx_target.rs entry (line 313) is stale: it omits the entire new GpuSupportLevel classifier this PR adds to that file.
The entry reads "Pure gfx-target lookups: extracting a gfx token, mapping a target to its TheRock family, the AMD PCI-id and marketing-name tables, and kernel gfx-version decoding" — no mention of GpuSupportLevel, gpu_support_level(), gpu_support_level_for_host_gpu(), or VULKAN_ONLY_SUPPORT_CLAIM, all of which this PR adds to this exact file (confirmed via git diff: gfx_target.rs gains ~155 lines introducing exactly this classifier, the central piece of this whole PR). This line of docs/architecture.md itself has zero diff in this PR.
AGENTS.md is explicit that cargo xtask check-architecture-doc only verifies an entry's presence, not that its description stays true ("The check reads no prose, so keeping an entry's description true is still on you") — which is why this didn't fail CI, but it is still a violation of the project's own stated authoring requirement, and it's introduced by this PR's own changes to the file in question.
Confidence 80. Suggested fix: update the entry to mention the GpuSupportLevel classifier.
There was a problem hiding this comment.
Blocking — merge_state_field (lines 185-191, pre-existing/unchanged by this PR) and write_state (lines 220-227) bypass the file lock and the Anthropic-endpoint carry-forward fix this PR adds elsewhere, leaving AIM exposed to the race the fix targets. (Posted at file level because lines 185-227 are unchanged context outside this PR's diff hunks, so an inline comment there is rejected.)
merge_state_field/write_state do a plain fs::read_to_string/fs::write with no FileLock and no call to carry_published_anthropic_endpoint. Compare crates/rocm-core/src/anthropic_endpoint.rs's fix, wired through crates/rocm-core/src/endpoint_readiness.rs's write_json_state_file_with — a path AIM's own writers never go through. record_served_model_id (line 173, which calls merge_state_field) is invoked right as the container is confirmed ready, the same event that gates the Anthropic-proxy thread's publish in apps/rocmd/src/anthropic_proxy.rs. Two threads of the same rocm __engine-serve-http aim ... process can race: AIM's own thread reads state before the proxy's publish lands, then overwrites the file with its stale copy, silently dropping the just-published endpoint — the same failure mode this PR's fix targets for lemonade, via a different writer.
Confidence 75 (traced directly via the absence of locking/carry-forward in this file and the call-site timing; the actual interleaving was not reproduced by running concurrent threads).
Suggested fix: route AIM's merge_state_field/write_state through rocm_core::merge_json_state_file/write_json_state_file, or document why AIM is intentionally exempt.
Main landed ROCMAI-703 (#1050), which rewrites the same deployment-summary prose and adds its own serve scenario, so all three conflicts are with it. tests/e2e-cucumber/features/model_serving.feature: main took `serve-60` for its smoke-test-failure scenario. Main's keeps 60; this branch's APU scenario becomes `serve-61` and moves after it, so each number still equals its position. `apps/rocm/src/therock/family.rs`'s prose reference follows. The two `-n "serve-60"` repro filters in docs are main's own and are left alone. docs/testing.md: two independent sections appended at the same place, kept whole. They are unnumbered `##` headings, so nothing renumbers. README.md: both sides rewrite the one paragraph describing the deployment summary, so this is a real merge rather than a choice between them. The summary sentence keeps this branch's `device:` row and main's endpoint/model/ smoke-test list; main's smoke-test-failure paragraph and this branch's "device line is omitted when the engine cannot say yet" paragraph both survive; and the shared closing sentence is kept once. Taking either side whole would have silently dropped the other's user-facing contract. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
|
Correction to my earlier note: the colliding number has moved.
As before, each is correct within its own branch (a scenario's number equals its position in its own file); the clash only exists across the two, so whichever merges first forces the other to renumber. Worth noting this is the third time these numbers have moved — #854's pair has gone 57/58 → 59/60 → 60/61 → 61/62 purely from I will renumber whichever branch lands second. |
Main landed the WSL librocdxg work (#1091), which appends to the same two examine files this branch does. tests/e2e-cucumber/features/examine.feature: main took `examine-49` through `examine-53`. This branch's support-level scenario becomes `examine-54` and moves after them, so each number still equals its position. Three references follow it: `host_gpu/render.rs`'s doc comment, and the prose and repro filter in `docs/testing.md`. The `examine-49`..`examine-53` references further down that file are main's own librocdxg paragraphs and are left alone. tests/e2e-cucumber/tests/e2e/examine_steps.rs: two independent step blocks appended at the same place, both kept. The split left each side's final function unterminated while sharing a single closing brace, so the brace is reproduced for each rather than closing only the block that came last. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
#678 merged #1091, which took `examine-49` through `examine-53` and moved #678's own support-level scenario to `examine-54`. This branch's four Vulkan scenarios collided with that block, so they become `examine-55` through `examine-58`. They already sat last in the file, so only the numbers moved. Nine references follow them: three comments in `examine_steps.rs`, the `host_gpu/vulkan.rs` row in `docs/architecture.md`, and the four support-level table rows plus the repro filter in `docs/testing.md`. Two neighbouring `examine-51` references are main's librocdxg ones -- the `render.rs` test comment and the librocdxg paragraphs in `docs/testing.md` -- and are left alone. crates/rocm-core/src/host_gpu/render.rs: both sides added a field to the same `format!` template. Resolved as the union -- this branch's `vulkan: {}` and main's extra placeholder before `managed_runtimes` -- because the argument list git had already merged carries both `self.vulkan_line()` and `self.runtime_librocdxg_line()`. Taking either template alone would have left the placeholders and the arguments out of step. docs/testing.md: this branch raises the support levels from three to five, which is its own change, so its wording wins over the base's. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Main landed runtime provenance (#1090), which appends its own scenario to examine.feature and took `examine-54`. This branch's support-level scenario becomes `examine-55` and moves after it, so each number still equals its position. Two references follow it: `host_gpu/render.rs`'s doc comment and the prose and repro filter in `docs/testing.md`. The other `examine-54` references in that file belong to main's provenance scenario (`@id:examine-reports-where-the-active-runtime-came-from`) and are left alone. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Main's #1090 took `examine-54`, so #678's support-level scenario moved to `examine-55` and this branch's four Vulkan scenarios shift again, to `examine-56` through `examine-59`. Nine references follow them: three comments in `examine_steps.rs`, the `host_gpu/vulkan.rs` row in `docs/architecture.md`, and the four support-level table rows plus their repro filter in `docs/testing.md`. The `examine-55` repro filter immediately above that table is #678's and stays, as do the `examine-54` references to main's provenance scenario. docs/testing.md: this branch raises the support levels from three to five, which is its own change, so its wording wins over the base's. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Review — needs work
Full review of the whole change (base 9e50eac5a03df41a2f23e73dac3a7ec470ea78a7 → head 30f91a09c6f27419e0511914fd8c6208a587b1bd, 40 files, +5605/-1458). Ticket ROCMAI-550 is named throughout the description/title; not retrieved (internal tracker identifier, kept out of a public review to avoid pulling internal-system content into it). The description is otherwise unusually detailed and largely accurate against the diff — its own "Deferred, explicitly" section discloses several gaps this review independently confirmed (e.g. the VulkanOnly tier having no hardware-grounded e2e scenario) — but it omits an entire subsystem of the diff; see Blocking below.
Blocking
-
engine_ipc.rs's new doc comment describes"unknown"-until-ready device gating as uniform across engines; Lemonade's gate differs from vLLM/AIM's —apps/rocm/src/engine_ipc.rs:366-370(device_worth_reportingdoc), againstengines/lemonade/src/lib.rs'shealthcheck_service_at(gates on full inference-verifiedready) vs.engines/vllm/src/state.rs/engines/aim/src/state.rs(gate merely onstate.is_some()). The comment cites both vLLM and Lemonade as sharing "no state on disk" semantics; they don't.
Confidence 85 · logic/doc-accuracy · Fix: reword to describe the actual per-engine difference, or make Lemonade's gate consistent with the other two. -
The e2e "both surfaces agree" device check can never fire under any current CI lane's harness —
tests/e2e-cucumber/tests/e2e/serving_steps.rs:3807(assert_services_list_reports_device'sdevice_row_value(summary)check).world.serve_summaryis captured viarun_rocm_bounded's file-redirected stdout, soserve_cmd.rs'ssummary_modegate is always false and the CLI takes the device-row-freeprint_managed_launch_plainpath; even were that not so,serve_summary.rs'srender_summarywrites the row with no colon (" {label:<label_width$} {value}") whiledevice_row_valuelooks for"device:". Either way the comparison is structurally dead for serve-05 and serve-07, the only two scenarios that use it. AGENTS.md's own standard: "a check whose name or comment claims a branch it cannot fail on is a defect, not missing coverage... rank it worse than no test at all."
Confidence 85 · logic/test-coverage · Fix: driverocm servethrough a real PTY here (as serve-28/41/60 already do) and align the two renderers' row formats, or state plainly that the cross-check is currently unreachable. -
vLLM and AIM remain exposed to the exact Anthropic-endpoint publish race this PR hardens only for Lemonade, and the new module doc overclaims general coverage —
engines/vllm/src/state.rs:189,235andengines/aim/src/state.rs:164,220(both add the newrocm_core::pinned_gpu_nameprobe, but bothwrite_stateremain a raw, unlockedfs::write), vs.engines/lemonade/src/state.rs:160-161(the onlywrite_staterouted throughrocm_core::write_json_state_file, the sole caller ofcarry_published_anthropic_endpoint,crates/rocm-core/src/endpoint_readiness.rs:156);crates/rocm-core/src/anthropic_endpoint.rs:16-20's doc states generally "the engine adapters write it whole... keep the whole-file write from dropping what the sibling thread published" — true only for Lemonade. The PR author's own review-thread comment traces this exact mechanism for Lemonade and notes "Not fixed here:engines/vllmandengines/aim... that gap predates this PR... it is a separate change" — accurate that the unlocked write predates this PR, but by the same comment's own causal chain, the slow probe that makes the race practically reachable is newly added to vLLM/AIM by this very diff, which the "predates this PR" framing undersells. The current PR description doesn't mention this scope limitation at all.
Confidence 85 · architectural · Fix: route vLLM's/AIM'swrite_statethroughrocm_core::write_json_state_file, or state the scope limitation explicitly and track the remainder. -
docs/manual-testing.md: this diff drops--foregroundfrom the examplerocm servecommand, but the prose ~34 lines later still calls it "that foreground server" with no stop instruction — command at line 408 (diff:...--device gpu_required --foreground --port 11435→...--device gpu_required --port 11435), contradicted by "With that foreground server stopped, check that..." at line 442 (HEAD), which — unlike the parallel flow later in the same file — gives norocm services stop <id> --yes-style instruction.
Confidence 90 · mechanical · Fix: restore--foregroundif intended, or reword the follow-on prose for a backgrounded service and add the stop step. -
No test in any of the three engines drives the device field through the real
healthcheck_service/healthcheck_service_atRPC entry point against a populated/ready state — only the privatereported_deviceformatter is unit-tested directly:engines/vllm/src/state.rs:753,engines/aim/src/state.rs:416,engines/lemonade/src/device_policy.rs:259,270. Grepping each engine'slib.rsfor.device/response.devicereturns zero hits. These tests would all still pass if the RPC handler's wiring to the formatter were reverted.
Confidence 70 · test-coverage · Fix: add, for at least one engine, a test driving the real RPC handler end-to-end and asserting.device. -
The PR description does not mention the Anthropic-endpoint publish-race subsystem of the diff at all —
apps/rocmd/src/anthropic_proxy.rs,crates/rocm-core/src/anthropic_endpoint.rs,crates/rocm-core/src/endpoint_readiness.rs, plus thepinned_gpu_namecalls newly added to every engine's state-writing path, are a substantial, independently-discoverable piece of this diff with no mention in the PR body (whose "Also in this diff"/"Known costs"/"Deferred, explicitly" sections are otherwise thorough). This is very plausibly connected to the previous finding going unnoticed through this PR's many review rounds — a reviewer working from the description would not know to check this subsystem's completeness.
Confidence 80 · description-scope (Step 9) · Fix: cover this subsystem in the PR body per AGENTS.md §5 ("write the PR body from the final diff").
Non-blocking
publish_anthropic_endpointrecords its note unconditionally before the merge it precedes is known to succeed, with no rollback on non-retryable failure —crates/rocm-core/src/anthropic_endpoint.rs:110-131. A narrow edge case (needs a non-retryable merge failure on first publish) that could carry a dead endpoint into a later rewrite.
Confidence 70 · logic.- The new carry-forward mechanism's tests are all single-threaded/sequential —
crates/rocm-core/src/anthropic_endpoint.rs's three new tests never drive the publisher and the rewriter on different threads with the lock actually contended, despite the repo having exactly that infrastructure elsewhere (endpoint_readiness.rs'sconcurrent_merges_across_threads_and_processes_keep_every_key).
Confidence 75 · test-coverage. lemonade_engine_note'sRocmHiparm hardcodes a string literal byte-identical to, but untied from,engine_inventory()'s own lemonade catalog entry (apps/rocm/src/main.rs:1083-1098vs.~1189-1191), which it always overrides. A future edit to one would silently stop matching the other.
Confidence 70 · architectural.
Decisions for the author
- tradeoff — bundling the Anthropic-endpoint race hardening into this PR's scope (defensible: the new host-GPU probe is what creates/widens the exposure) vs. AGENTS.md §11's one-logical-change guidance. Given the hardening is incomplete (see Blocking), a split might have gotten the gap more scrutiny.
- non-blocking-improvement —
gpu_support_level's two positionalOption<&str>arguments could be tightened into a type that prevents transposition, closing the gap the function's own doc comment already flags (no current call site has them backwards).
Positive signals
GpuSupportLevel's 3 variants are exercised as a genuine state matrix inhost_gpu/render.rs's tests (RocmHip with matching/mismatched/no install, VulkanOnly, Unknown-detected-unrecognized, Unknown-not-detected), each with its own assertion.pinned_gpu_name's decision logic is a pure, fully unit-tested helper, with a test asserting the expensive probe closure is never invoked when the decision short-circuits — exactly the kind of test that would catch a broken guard.service_log_follow.rs's "pure move, not a rewrite" claim was mechanically verified byte-identical against the pre-changeservices_text.rscontent, not just trusted.- The device-row feature has an explicit state matrix in both
serve_summary.rsandservices_text.rs, including a probe-not-even-called assertion for the reuse path, not just a value-absent one. VULKAN_ONLY_SUPPORT_CLAIMconsolidates three independently-worded copies of the same user claim into one pinned, tested constant.docs/testing.md/docs/manual-testing.md's other new technical claims (probe-budget numbers,format_reported_device's exact shape, scenario IDs) were all independently cross-checked against code and found accurate — an unusually well-verified documentation PR apart from the one contradiction above.advised_commands.rs's mechanical scan automatically covers every new remediation command string with no special-casing required.- The PR's own "Deferred, explicitly" section's disclosure of the
VulkanOnlye2e-coverage gap was independently confirmed accurate by this review.
Deployment notes
rocm services list's documented ~18s worst-case block (a process-wide mutex serializing concurrent CLI invocations) is extended, not introduced, by this PR, and is already disclosed by the author as an accepted, reversible cost.- Windows hosts running the vLLM or AIM engine are where the unprotected race (see Blocking) is reachable in practice — Linux's GPU probe is cheap enough that the window is negligible, per the author's own traced root-cause analysis for the Lemonade case.
- No schema migration: new engine-protocol/state fields are additive and optional.
What this covered
Full diff read via 9 coupled file-group workers fanned out in parallel (grouped by shared contract/coupling per this method's own sizing guidance, given the diff's size): core GPU detection/classification (crates/rocm-core/src/{gfx_target,host_gpu,host_gpu/*}.rs); family catalog + examine report + advised-commands + README/architecture docs; engine IPC/protocol + all three engines' state + MCP/bridge rendering; serve/services rendering + the new service_log_follow.rs split; CLI/main/providers wiring; the Anthropic-endpoint race-fix subsystem; docs/manual-testing.md/docs/testing.md; and the e2e feature/step files (examine+runtime, then model-serving), each split further where one worker's group was still too large. Every file in every group was read in full at the PR head via git show/git diff against the fetched refs (no checkout), plus surrounding caller/callee code as needed; two workers explicitly noted sampling rather than exhaustively line-reading very large, mostly-untouched pre-existing regions of docs/testing.md (~5600 lines) and examine_steps.rs/runtime_steps.rs (~3600/~1900 lines) beyond the new content and its directly-adjacent helpers — disclosed per this method's rule rather than silently partial. CI was read directly from the GitHub API: all required checks (clippy, module-size, architecture-doc, full e2e suite, security scans) pass at the reviewed head.
Did not run: no Rust test binary or the e2e-cucumber suite was executed by this review (all "would this still pass reverted" verdicts are from static tracing, disclosed per finding); a dedicated "history"/git blame pass and a separate "prior PR review comments" pass were not run as independent passes — each file-group worker instead incidentally used git log/git diff against the merge-base where it bore on a specific claim (e.g. the service-log-follow "pure move" check), which is narrower than the method's normal separate-pass treatment, disclosed here as a cost-driven deviation.
Findings were reached blind, from the code only; the PR description and ticket were read afterward (Step 9) and this PR's extensive prior review history (dozens of rounds, none carrying this method's marker) was read only after that, per the task's own instructions, to check whether these specific findings had already surfaced — none of the six Blocking findings above appear in the prior discussion under any wording this review could find, though the vLLM/AIM race gap (third Blocking item) was separately, partially disclosed by the author in a review reply (not in the PR description itself) while fixing the Lemonade-specific case; that reply is quoted and reasoned about directly in that finding rather than treated as settling it, consistent with this review not being reconciled against prior discussion.
🤖 Automated by an AI agent
| device_worth_reporting(response.device) | ||
| } | ||
|
|
||
| /// The engine's reported device, or `None` when it said nothing usable. |
There was a problem hiding this comment.
Blocking — this doc comment describes "unknown"-until-ready gating as uniform across engines; it isn't.
The comment says "unknown" is "the engines' own device value when no state is on disk for this service id yet (see engines/vllm/src/state.rs and engines/lemonade/src/lib.rs)". That's accurate for vLLM/AIM (Some(state) => reported_device(state), None => "unknown" — gated only on state.is_some()), but not for Lemonade: healthcheck_service_at in engines/lemonade/src/lib.rs gates on ready (let device = if ready { ... } else { "unknown".to_owned() }), which stays "unknown" even once a state file clearly exists, until the server's own inference probe has verified it can serve. So the three engines disagree on when device leaves "unknown", and this new comment asserts a uniformity that doesn't hold — the kind of claim AGENTS.md §3 says must be checked against the code it describes.
Confidence 85 (traced directly against engines/lemonade/src/lib.rs's healthcheck_service_at, engines/vllm/src/state.rs, and engines/aim/src/state.rs). Suggested fix: reword to name the actual per-engine gating difference, or make Lemonade's gate consistent with the other two if the divergence is unintentional.
| let Some(summary) = world.serve_summary.as_deref() else { | ||
| return; | ||
| }; | ||
| if let Some(summary_device) = device_row_value(summary) { |
There was a problem hiding this comment.
Blocking — this cross-surface "both surfaces agree" check can never actually fire under any current CI lane.
device_row_value(summary) looks for a line matching strip_prefix("device:") (colon-terminated). But world.serve_summary is the stdout of a rocm serve invocation captured via run_rocm_bounded (tests/e2e-cucumber/tests/e2e.rs), which redirects stdout to a plain file — never a terminal. apps/rocm/src/serve_cmd.rs's summary_mode = background && IsTerminal::is_terminal(&stdout()) is therefore always false here, so the CLI takes the print_managed_launch_plain branch (apps/rocm/src/service_launch/start.rs), whose output has no device line at all. And even if summary_mode were somehow true, serve_summary.rs's render_summary writes the device row as " {label:<label_width$} {value}" — a padded label with no colon (confirmed by its own test asserting starts_with("device"), not "device:") — so strip_prefix("device:") still wouldn't match. Either way, device_row_value(summary) is always None, so the assert_eq! on the two lines above this one never executes, for both scenarios that use this step (serve-05, serve-07).
This is the class AGENTS.md's own test-quality standard calls out explicitly: "a check whose name or comment claims a branch it cannot fail on is a defect, not missing coverage... rank it worse than no test at all." A real regression in cross-surface device agreement would pass silently.
Confidence 85 (fully traced: the summary_mode gate, print_managed_launch_plain's body, render_summary's no-colon format and its own unit test proving that format, and run_rocm_bounded's file-redirected stdout — not executed, per this review's constraints). Suggested fix: either drive rocm serve through a real PTY here (as serve-28/41/60 already do elsewhere) so summary_mode activates, and align device_row_value's parsing with whatever format render_summary actually emits — or state plainly in this step's own comment that the cross-check is currently unreachable.
| // path. `rocm_core::pinned_gpu_name` answers `null` when the host | ||
| // probe cannot be attributed to the pinned ordinal, so the row | ||
| // falls back to `rocm gpu <n>` rather than naming another card. | ||
| "gpu_name": rocm_core::pinned_gpu_name(&request.gpu_indices), |
There was a problem hiding this comment.
Blocking — vLLM (and AIM, same pattern) remain exposed to the exact Anthropic-endpoint publish race this PR hardens only for Lemonade.
This PR adds the rocm_core::pinned_gpu_name(...) probe (new here — confirmed via diff against merge-base) to vLLM's and AIM's write_running_state too, but their write_state (engines/vllm/src/state.rs ~line 235, engines/aim/src/state.rs ~line 220) still does a raw, unlocked fs::write, never going through rocm_core::write_json_state_file — the only function carry_published_anthropic_endpoint is wired into (crates/rocm-core/src/endpoint_readiness.rs:156). crates/rocm-core/src/anthropic_endpoint.rs's new module doc says generally "Both write the one engine state file, and the engine adapters write it whole... keep the whole-file write from dropping what the sibling thread published" — true only for Lemonade.
The author's own review-thread comment on this PR traces the exact mechanism for Lemonade (a multi-second Windows GPU probe inside a post-spawn whole-file rewrite racing the Anthropic proxy's publish) and explicitly notes "Not fixed here: engines/vllm and engines/aim write their state files with their own unlocked write_state... that gap predates this PR... it is a separate change." That's only half right: the unlocked write predates this PR, but the slow probe that makes the race practically reachable is new here, added to vLLM/AIM by this same diff — by the author's own causal chain for Lemonade, this PR is what newly widens the window for these two engines as well. The current PR description doesn't mention this scope limitation at all.
Confidence 85 (the pinned_gpu_name/raw-fs::write pattern is confirmed by direct read of both files; the causal argument is the PR's own, applied consistently). Suggested fix: either route vLLM's/AIM's write_state through rocm_core::write_json_state_file, or state the scope limitation in the PR description and track the remainder as a tightly-scoped, explicit follow-up.
| // container is handed a device list, not an ordinal — and | ||
| // `rocm_core::pinned_gpu_name` still declines on any host where | ||
| // more than one AMD GPU could be the one that answered. | ||
| "gpu_name": rocm_core::pinned_gpu_name(&[]), |
There was a problem hiding this comment.
Same finding as the one on engines/vllm/src/state.rs:189 (this comment is just the second affected site, so it doesn't scroll out of view): this PR adds the pinned_gpu_name probe here too, but AIM's write_state (~line 220) is also a raw, unlocked fs::write that never goes through rocm_core::write_json_state_file, so it gets neither the file lock nor the new carry-forward protection. See the other comment for the full trace and the author's own partial acknowledgment of this gap.
|
|
||
| ```powershell | ||
| rocm serve qwen --engine lemonade --device gpu_required --foreground --port 11435 | ||
| rocm serve qwen --engine lemonade --device gpu_required --port 11435 |
There was a problem hiding this comment.
Blocking — this diff drops --foreground from the example command, but the prose ~34 lines below (line 442 at HEAD) still calls it "that foreground server" and gives no stop instruction.
Diff against merge-base: rocm serve qwen --engine lemonade --device gpu_required --foreground --port 11435 → rocm serve qwen --engine lemonade --device gpu_required --port 11435. Per this same file's own stated convention ("rocm serve returns to the shell once the server is ready. Background is the default... Only --foreground or --verbose attach to the log stream"), the server launched by the edited command now runs in the background — yet "With that foreground server stopped, check that a catalogued recipe reaches Lemonade..." (line 442) still describes it as foreground and, unlike the parallel flow later in the same section (which explicitly says rocm services stop <service-id> --yes), gives no instruction for how to actually stop it before continuing.
Confidence 90 (direct diff + content read). Suggested fix: either restore --foreground if that's genuinely intended, or reword the follow-on prose to describe a backgrounded service and add the stop command.
| /// is the format README.md and `docs/manual-testing.md` never described and | ||
| /// which made one row disagree with itself across engines. | ||
| #[test] | ||
| fn reported_device_matches_the_shape_lemonade_reports() { |
There was a problem hiding this comment.
Blocking (test-coverage) — no test in any of the three engines drives the device field through the real healthcheck_service/healthcheck_service_at RPC entry point against a populated/ready state; only the private reported_device formatter is tested directly.
This test (reported_device_matches_the_shape_lemonade_reports) calls reported_device(...) directly, as does AIM's reported_device_for_existing_state_matches_the_shared_engine_shape (engines/aim/src/state.rs:416) and Lemonade's reported_device_reflects_backend_pinned_ordinal_and_gpu_name/reported_device_names_the_gpu_when_no_ordinal_was_pinned (engines/lemonade/src/device_policy.rs:259,270). Grepping each engine's lib.rs (where the real healthcheck_service/healthcheck_service_at RPC handlers live) for .device/response.device returns zero hits in all three. So: would these tests still pass if the RPC handler's Some(state) => reported_device(state) wiring were reverted to its old hardcoded form? Yes — nothing here would notice, since none of them call the RPC handler itself.
Confidence 70 (confirmed by reading all three engines' full test suites; the current wiring is in fact correct, so this is a coverage gap rather than a live defect). Suggested fix: add, for at least one engine, a test that calls the real RPC handler with a populated/ready state and asserts the returned .device.
| /// Called more than once for one endpoint when the merge has to be retried | ||
| /// (`rocmd`'s proxy retries a state file caught mid-write); the note is the | ||
| /// same each time. | ||
| pub fn publish_anthropic_endpoint( |
There was a problem hiding this comment.
Non-blocking — the in-memory "published" note is recorded unconditionally before the merge it precedes is known to succeed, and is never rolled back on a non-retryable failure.
published.insert(...) runs before crate::merge_json_state_file(...) is called, and the function just returns that call's Result with no cleanup on Err. If the merge fails non-retryably (e.g. JsonStateRefusal::NotAnObject, or a FileLock::acquire error — apps/rocmd/src/anthropic_proxy.rs's publish_with_pause doesn't retry these), the stale note persists in PUBLISHED_ENDPOINTS for the rest of the process, and a later full-state rewrite for the same path would still carry a dead endpoint into the document via carry_published_anthropic_endpoint.
Confidence 70 (straight-line control-flow trace, single-threaded). This is a narrow edge case (requires a non-retryable merge failure on first publish) — flagging as non-blocking rather than blocking. Suggested fix: only insert the note after the merge succeeds, or clear it on non-retryable failure.
| /// now answers for every level, and this says what that answer is: the support | ||
| /// level, not the backend. The backend is settled by launching one, which that | ||
| /// line in turn points at. | ||
| const fn lemonade_engine_note(level: rocm_core::GpuSupportLevel) -> &'static str { |
There was a problem hiding this comment.
Non-blocking — RocmHip's arm hardcodes a string literal byte-identical to, but untied from, engine_inventory()'s own lemonade catalog entry ("default embedded Lemonade server with ROCm llama.cpp backend", apps/rocm/src/main.rs ~line 1189), which engine_inventory_note always overrides for every GpuSupportLevel including this one.
Since catalog_note is already in scope at the one call site (engine_inventory_note), this arm could take it as a parameter and return it verbatim instead of re-stating it — the catalog entry is now dead data on every real call path (bridge_snapshot.rs, examine_report_text.rs, chat/mcp_bridge.rs), and nothing ties the two literals together, so a future edit to one (e.g. a typo fix in the catalog) would silently stop matching the other. The only test covering this (examine_report_text.rs's *_keeps_rocm_note_for_a_rocm_capable_gpu) asserts against its own separately hand-written literal, not against engine_inventory() itself.
Confidence 70 (confirmed both literals are currently byte-identical and unconnected). Suggested fix: have this arm take catalog_note and return it for RocmHip instead of re-stating the string.
…tate under the lock AIM's `reported_device` hardcoded `rocm` and probed the host for a GPU name whatever the launch's `device_policy` was, so a `cpu_only` container — which `resolve_use_gpu`/`docker::RunSpec` correctly give no /dev/kfd, no /dev/dri and no HIP_VISIBLE_DEVICES — still rendered e.g. `rocm (AMD Instinct MI300X)`. That is the invented device claim this branch exists to remove, shipped in compiled code even though `parse_device_policy` currently refuses `cpu` at the CLI. The policy the launch recorded now decides, and the probe is not made at all when no GPU was attached. AIM also recorded no `gpu_indices`, so on a multi-GPU host it answered a bare `rocm` where lemonade and vLLM answer `rocm gpu <n> (<name>)` for the same hardware — the engine does pin an ordinal, `plan_gpus` resolves it and `HIP_VISIBLE_DEVICES` narrows the container to it. `run_container` now reports the ordinals it attached so the state file and the container cannot disagree. Both engines' `write_state` were a lock-free `fs::write` that bypassed `carry_published_anthropic_endpoint`. The unlocked write predates this branch, but the slow host GPU probe that makes the race practically reachable is added to these two state writes by this branch itself, so the exposure is this branch's to close: routing them through `rocm_core::write_json_state_file` orders them against every merge and carries a sibling thread's publication through the rewrite, as lemonade's already does. The status writers patch through `merge_json_state_file` instead of rewriting the document whole. `healthcheck_service_at` is split out of both handlers, the way lemonade's already was, so a test can drive the real RPC entry point: every device test in the three engines called the private formatter, so reverting a handler's wiring to a hardcoded literal kept the suite green. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…ld not `gpu_support_level`'s `(_, Some(_)) => RocmHip` arm ignores the detected target on purpose — `resolve_family` asks it about a `--family` the user named, where there may be no detected target — but every call in the suite passed Some/Some, None/None or Some/None, so narrowing the arm to `(Some(_), Some(_))` was invisible. `gpu_name_for_single_gpu_host`'s ordinal guard was in the same position: the only test with a non-zero ordinal also supplied a host count of 2, which the check two lines below already declines on, so the guard could be deleted outright with the suite still green. Both mutations now turn it red. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…et it read both renderers `assert_services_list_reports_device`'s comparison against `rocm serve`'s summary was dead twice over. `world.serve_summary` comes from `run_rocm_bounded`, which redirects stdout to a file, so `serve_cmd.rs`'s `summary_mode` is false and the CLI prints `print_managed_launch_plain`, which has no device row in any state; and `device_row_value` looked for `"device:"` while `render_summary` writes a padded label with no colon, so it could not have matched a summary even if one carried the row. The parser now reads both renderers' shapes (and still refuses the `device_policy:` row the serve plan prints), and the step says in full that on today's lanes the comparison does not run and the backend assertion above it is what discriminates — rather than letting the step's name imply coverage it has not got. Making it live means driving the vLLM and Lemonade per-PR serve canaries through a `TuiSession`, which is a change to those canaries' own harness. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…onade serve flow `device_worth_reporting` claimed the `"unknown"`-until-ready gate was uniform and cited vLLM and Lemonade as sharing it. They do not: vLLM and AIM answer as soon as a state file exists, while Lemonade keeps saying `"unknown"` until its own inference probe returns. That difference is the whole reason `SERVE_DEVICE_PROBE_BUDGET` exists, so the comment now names it. The probe bound alongside it also cites AIM's own `HEALTHCHECK_TIMEOUT_MS`, which it relied on without naming. `docs/architecture.md`'s `gfx_target.rs` entry still described pure lookups after that file gained the whole `GpuSupportLevel` classifier; the gate checks presence, not accuracy, so nothing caught it. `docs/manual-testing.md` dropped `--foreground` from the Lemonade serve example but still called the result "that foreground server" 34 lines later, with no way to stop it. The prose now matches a backgrounded service and gives the stop command, as the parallel flow below it already does. Also drops review-thread references from source comments and a note to reviewers from docs/testing.md; each says the reason instead. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
commented
Nov 17, 2026
|
Three CHANGES_REQUESTED reviews (5936137783, 5936357643, 5939041029) went unread until now — apologies. Head is now Fixed1. AIM's 2. AIM recorded no GPU ordinal (5936357643). Fixed. 3. vLLM's and AIM's unlocked On the "predates this PR" framing — you are right and I was wrong. The unlocked 4. 5. 6. 7. 8. 9. No test drove the device through the real RPC entry point (5939041029). Fixed. Partly fixed, and said plainlyThe e2e cross-surface device check is structurally dead (5939041029). Both halves of your trace are correct, and I confirmed them. I took your second option and said so in full, in the step's own comment and in the PR body: on today's lanes the comparison does not fire, the backend/shape assertion above it is what discriminates for serve-05 and serve-07, and making it live means driving the vLLM and Lemonade per-PR serve canaries through a Not fixed, with reasonsCommit-message rewrites (5936137783, item 1). Agreed on the substance — several messages describe work the final diff no longer contains, cite renumbered scenario ids, carry review chatter, and ROCMAI-550 scope (5936137783, item 4). Still deferred and now recorded as such in the body:
Confirmed by grep, as askedNothing compares against AIM/vLLM's old literal Also trimmedThe review-thread references in VerificationLinux devbox: |
One conflict, in `apps/rocm/src/chat/mcp_bridge.rs`, and it is the code-was-removed shape rather than two additions colliding. Main's #980 deletes `internal_mcp_arguments` outright and replaces it with `internal_mcp_tool_call` plus `mcp_call_cli_arguments`, so that a pasted MCP envelope is unwrapped in one named step instead of silently inside the dispatcher -- the point of that change being that the approval card and the executed call cannot disagree. This branch still carried the old function. Resolved by taking main's design whole and keeping only this branch's own addition from the other side, the `engines` MCP tool-result helper. Keeping this branch's `internal_mcp_arguments` would have reinstated the unwrapping main had just removed, which is the opposite of what #980 is for; grep confirms no caller of it survives. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Main landed the engine stdio protocol versioning (#1073), which appends to the same two places in `apps/rocm/src/engine_ipc.rs` this branch does: the module's free functions and its test module. Both conflicts are the same shape -- each side's last item was left unterminated by the split while a single closing brace sat below for both -- so the brace is reproduced for each block rather than closing only whichever came last. Nothing is interleaved: this branch's device-probe helpers and tests and main's `answer_builtin_engine_stdio` and its protocol-contract test are kept whole and side by side. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
20c48b4 to
f04357f
Compare
commented
Nov 17, 2026
|
Force-pushed the commit-message rewrite you asked for. The tree is byte-identical. That was the safety property I wanted before touching a 65-commit branch, and it is verifiable rather than asserted — the root tree object is What the rewrite did, across all 65 commits:
All 65 commits are re-signed and still signed-off ( #1036 is repaired and did not need a force-push. Rewriting this branch gave every commit a new SHA, so #1036's merge-base fell back to Both PRs are |


ROCMAI-550: report the GPU/backend support this host actually has
rocm examineand the engine notes asserted "default embedded Lemonade server with ROCm llama.cpp backend" on every host, andinstall sdkoffered package families for GPUs ROCm has never shipped a build for. On a Ryzen 4000/5000 laptop whose Vega iGPU isgfx90c— the machine this ticket came from — that is a claim about hardware nobody looked at.This PR makes the support claim come from one place and say only what the host supports. Body regenerated from the final diff at @juhovainio's request; the previous version omitted two changes and listed no deferrals.
The one classifier
rocm_core::gpu_support_levelderives the level from the two factsExamineSummaryalready computes — the detected gfx target and whichever TheRock family covers it:unknowndeliberately covers both "nothing detected" and "unrecognized target". Conflating the latter with "confirmed no ROCm build" would turn a gap in this build's catalog into a claim about the card, so an emptycompatible_therock_familynever reads as a failed probe.A
rocm_supportline is now printed on every host, not only the unsupported one — silence could not answer the questionlemonade_engine_notesends users torocm examineto ask.Every surface that makes a support claim renders from this one function, with exhaustive matches, so adding a level is a compile error rather than a silently stale sentence.
What each level advises
The advice has to be a command that can actually clear the condition it is printed for:
rocm install sdk, or, when a managed runtime is already installed, says so instead of recommending an install that is already done.rocm serve <model> --engine lemonade.rocm serve <model>, then read thedevice:row ofrocm services list. Deliberately not--engine lemonade: an unrecognized target may be a newer ROCm-capable GPU, and naming the Vulkan backend would be the same unestablished claim in the other direction. The engine chooses, anddevice:reports what it used.rocm diagnose.rocm serveis refused by the no-usable-GPU pre-flight on such a host, so recommending it would be useless.The reported backend device
rocm serveandrocm services listboth grew adevice:row, read from the engine over one Healthcheck RPC so the two surfaces cannot disagree. The row is optional by design: an engine still loading answers"unknown", and asserting that areadyservice must have a row is a false failure (serve-05/serve-07 assert the row's shape and backend, not its presence — see Tests for what their cross-surface comparison can and cannot reach).rocm_core::pinned_gpu_namenames the GPU only on a host with exactly one AMD GPU and no visibility mask, because every name probe answers for the primary adapter rather than a HIP ordinal. Multi-GPU hosts get the backend and ordinal with no name — correct and unspecific rather than specific and wrong.Also in this diff
Listed because the previous body omitted them:
runtime-19: the--family/ROCM_CLI_THEROCK_FAMILYrefusal is reworded. Asking for a target ROCm does not build is refused outright, and no other family is substituted.redacted()inserve_summary.rstests: a helper that masksapi key:values so the summary snapshots can be asserted without pinning a secret.The Anthropic-endpoint publish race
Omitted from every earlier version of this body, which is very plausibly why it took several review rounds to be looked at.
apps/rocmd/src/anthropic_proxy.rs,crates/rocm-core/src/anthropic_endpoint.rsandcrates/rocm-core/src/endpoint_readiness.rsare a real subsystem of this diff.The cause is this PR's own change.
rocm_core::pinned_gpu_nameis now called inside every engine's post-spawn whole-file state write, and on Windows that probe takes seconds. The Anthropic proxy is a thread of the same process (run_builtin_engine_serve_httpstarts it, then runs the adapter), and it publishes its endpoint into the same state file — inside the window the probe opened. The whole-file rewrite then replaced it.publish_anthropic_endpointnotes the publication in a process-local map before touching the file, andwrite_json_state_filecarries that note back into any full rewrite of the same path, under the state-directory lock.All three built-in adapters now go through it. The first revision wired only Lemonade and argued the vLLM/AIM gap "predates this PR"; that was half right at best — the unlocked
fs::writepredates it, but the slow probe that makes the race practically reachable is added to those two engines by this diff, so closing it is this diff's job. Theirwrite_statenow routes throughrocm_core::write_json_state_fileand their status writers throughmerge_json_state_file, and both engines have a test that fails if the rawfs::writecomes back.The carry note is process-wide and never evicted. Resolving the GPU name before the spawn would remove the need for it; that is a larger restructuring of the launch path and is not attempted here.
Tests
examine-55holds the printed level against thedetected_gfx_targetandcompatible_therock_familyin the same report, so every lane checks the level its own hardware implies rather than hard-coding one.serve-61drives detection and the serve pre-flight from the same planted/sys, so the advised command is checked against the host that was shown it — it shows the pre-flight accepts that host, stopping at an out-of-range--gpu, not that a Lemonade launch then succeeds there.no_gpu_advice_names_a_command_that_host_can_runpins the advice against behaviour rather than wording.Which lane covers the
device:row. End to end, only the GPU lanes:serve-05(vLLM, the per-PR canary) andserve-07(Lemonade, xfail'd on lemonade+linux). The blocking mock job has no livedevice:row at all; there it is covered by unit tests — the per-enginereported_deviceformatters, theserve_summary.rs/services_text.rsrow matrices,pinned_gpu_name's decision logic, and (new in the latest round) tests that drive each engine's realhealthcheck_service_atRPC handler and assert the.deviceit answers with, which is the one thing a reverted handler wiring would have slipped past.assert_services_list_reports_device's cross-surface comparison againstrocm serve's own summary does not run on any current lane, and the step now says so: the summary those scenarios capture is pipe-redirected, so the CLI prints the row-less plain form rather than the interactive deployment summary. What discriminates there is the backend/shape assertion above it. Making the comparison live means driving those two canaries through a PTY, which is a change to the canaries' harness and is not bundled here.Known costs, recorded
rocm services listcan block ~18s (10s budget + one in-flight 8s probe; the budget is a start gate, tested before each probe, never inside one), androcm chat's natural-languageservicescommand pays the same, holding the process-wide engine env lock. (Not the structured MCP tool that shares the name: it renders throughrender_services_tool_result_text, which has nodevice:row and makes no probe.) Moving the probe off the render path would stop the row reflecting the host at the moment it prints, which is the point of it; probing once and printing nothing after makes output depend on service ordering. Reversible: it is a constant plus a start gate, and a hard 10s cap needs a cancellable probe, not a smaller number.pinned_gpu_namereads the display inventory per call — deduplicated within a call, not cached across calls, andwrite_running_stateruns more than once per launch. Launch path only; a cache would need hotplug invalidation.Deferred, explicitly
None of these are done, and none should be read as done:
docs/rocm-docs/, the doctor skill files, anddocs/ux-guidelines.mdare untouched. ROCMAI-550 asks for them, and the ticket is already marked Done — so this needs a follow-up ticket rather than a note here.crates/rocm-dash-tui/src/ui/engine_manager.rsandapps/rocmd/src/host_snapshot.rs. Both mirrorapps/rocm's own staticengine_inventory()catalog, where the dynamic note is printed alongside it; in the TUI pane and the rocmd snapshot there is no note, so the static claim stands alone. Making them dynamic means giving the TUI layer arocm-coredependency it deliberately avoids, and threading a support level into rocmd — a design change rather than a reword, which is why it is not bundled here.VulkanOnlylevel has no hardware-grounded e2e scenario. No CI lane carries a gfx90c-class GPU, soexamine-55can only check that the printed level is self-consistent with thedetected_gfx_targetandcompatible_therock_familyof whatever hardware that lane has — on every current lane that is theRocmHiporUnknownbranch, and one of its step assertions derives its expected value from the line it is checking. TheVulkanOnlywording itself is asserted literally only by Rust unit tests (host_gpu/render.rs,examine_report_text.rs,mcp_bridge.rs,therock/family.rs) against planted inputs. Closing this needs either such a lane or a planted gfx90c host driven through anexaminescenario (the repo does this elsewhere, e.g.host_with_a_kfd_less_gfx90c); it is a known, accepted gap here, not an oversight.Verification
Linux devbox (this Mac cannot build
rocm-engine-lemonade):cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings,cargo clippy -p e2e-cucumber --test e2e -- -D warnings,cargo test -p rocm-core(1627 passed),cargo test -p rocm --bin rocm(1947 passed),cargo test -p rocm-engine-aim(95) /-p rocm-engine-vllm(172) /-p rocm-engine-protocol,cargo test -p e2e-cucumber --test feature_naming,cargo xtask check-crate-edges/check-module-size(431 files) /check-architecture-doc.Every test added in the latest round was mutation-checked: breaking the production change it covers turns it red, restoring it turns it green. That includes the AIM
cpu_onlygate, AIM's recorded ordinal, both engines' RPC-handler device wiring, both engines' locked state write,gpu_support_level's(_, Some(_))arm andgpu_name_for_single_gpu_host's ordinal guard.serve-05/serve-07cannot run on that devbox — their Given step (rocm serve Qwen3-0.6B-GGUF --engine lemonade --managed) fails there with rc=1. Confirmed pre-existing and environmental by running them on this PR's own previous head with no local changes: identical failure. They pass in CI.