Repository navigation
Conversation
c3612ca to
54625d3
Compare
54625d3 to
9ac33b1
Compare
tomastola
left a comment
There was a problem hiding this comment.
Review — needs work
Blocking
-
docs_redirectcan panic on a crafted service id —apps/rocmd/src/api_docs.rs:287-289
Path<String>is percent-decoded, so/services/%0A/docsyields a newline, andRedirect::permanent(&format!("/services/{service_id}/docs/"))then builds an invalidLocationheader value, which panics (read from the code, not run). The loopbackHostguard does not stop it: a web page the user visits can make their browser requesthttp://127.0.0.1:11430/services/%0A/docs, and thatHostpasses. Only that connection is lost, but the handler also never checks that the service exists and answers a cacheable 308. No test covers the redirect.
Confidence 70 · logic · Fix: percent-encode the id, or checkfind_running_servicefirst and return 404. -
The
docs:output has no scenario on the summary and attached paths, anddocs/testing.mdoverstates the coverage. —apps/rocm/src/serve_cmd.rs:937,apps/rocm/src/service_launch.rs:634-637,922-933,957-968,983-985,docs/testing.md
The e2e harness pipes stdout, sosummary_modeis false and serve-05/serve-07 only reach the plaindocs:line. Nothing runs the TTY summary wiring (docs_url: docs_link), the--verboseattached output, or the already-runningdocs:line; theserve_summaryunit tests buildDeploymentSummaryby hand, so they stay green if theserve_cmdwiring breaks. AGENTS.md §3 requires a scenario for user-observable output, anddocs/testing.mdsays the linkrocm serveprints is "checked by serve-05/07", which holds only for non-TTY plain output.
Confidence 85 · architectural · Fix: add a scenario or PTY check for the summary and attached paths, or narrow thedocs/testing.mdclaim and name the manual check (docs/manual-testing.md4b covers only the TTY summary). -
rocm services list --helpdoes not mention the newdocs:line. —apps/rocm/src/main.rs:1046-1050
serveanddaemonhelp were updated; theServicesCommand::Listdoc comment was not. AGENTS.md requires the--helpdoc comment to change with observable behaviour.
Confidence 85 · mechanical · Fix: add a clause about the docs link. -
Tests that stay green when one branch of the change is broken.
service_docs_links's/healthshort-circuit (apps/rocm/src/api_docs_link.rs:~88):no_link_when_nothing_listens_on_the_docs_portasserts only<2s, so deleting the probe leaves it green, although the doc comment claims it costs one refused connection.- The
.filter(managed_service_is_live)atapps/rocm/src/main.rs:6666: service-docs-03 stays green without it, because the docs server answers 404 for a stopped record. no_link_when_the_docs_port_is_zerostays green withserved_port()?removed, because connecting to port 0 fails anyway.- service-docs-04 passes with the whole feature reverted (a reverted build prints no
docs:line); it does go red if the probe is removed.
Confidence 70 · mechanical · Fix: have the stand-in server count connections and assert exactly 1 for/health; assert the stopped-record case against a server that answers 200 for that id; make the zero-port test use a port that would answer.
Non-blocking
- Record read errors are shown as "not running" and logged nowhere —
api_docs.rs:~158,running_services.load_managed_services(..).unwrap_or_default()turns an unreadable records directory into "No model server is running" and a 404. Fix: log on Err, and ideally return 5xx on the per-service routes. (Confidence 75) - Daemon shutdown can hang on an active try-it stream —
api_docs.rs:62-65,~96,apps/rocmd/src/service.rs:~558. Graceful shutdown waits for in-flight responses, the forwarder has no overall timeout, andstop()then joins the thread synchronously. Fix: bound the serve future with a timeout after the shutdown signal. (Confidence 70) - The liveness predicate is duplicated —
apps/rocmd/src/api_docs.rs:217(service_is_running) andapps/rocm/src/main.rs:8189(managed_service_is_live). Same four statuses today; drift would give no link or a link to a 404. A shared helper inrocm-corewould fix it. (Confidence 70) - Attached mode waits silently for up to 5 s —
apps/rocm/src/service_launch.rs:957-962.service_docs_link(..., &mut || {})passes no progress callback, so afterendpoint:there is a silent stall when a just-spawned helper's docs never answer. (Confidence 70) - Misplaced comment —
api_docs.rs:~286. The comment about the endpoint key lasting "until the page is closed" sits above.try_it_out_enabled(true), which does not control that; it relies on the Swagger UI default and nothing asserts it. Fix: move the comment or setpersist_authorization(false)explicitly. (Confidence 70)
Decisions for the author
- Docs are served only by the persistent helper (
automations_enabled) — tradeoff. An on-demandrocmd runreturns beforestart_api_docs. The CLI probes before printing a link, so no dead link appears; the cost is that no docs exist in that mode. Confirm this is intended. - The Origin check admits any loopback origin on any port — tradeoff. Another local web app (e.g. a dev server on
localhost:3000) can POST through the forwarder to any running model server. That is deliberate for a same-origin docs page and still blocks cross-site pages and DNS rebinding. Confirm this is the intended boundary. - Forwarded responses keep only
Content-Type— non-blocking improvement. Atext/htmlreply from a model server (e.g. llama.cpp's UI) would run same-origin with the docs. ConsiderX-Content-Type-Options: nosniffandContent-Security-Policy: sandboxon forwarded responses. - The change is large for one PR — about 2.7k lines across a new HTTP subsystem, CLI wiring, e2e scenarios and licence plumbing; the licence work could ship separately.
nowycondro
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 9ac33b1
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (29 files, +2709/-42), split into five independent file-coupling groups (forwarder/docs server, protocol/config contract surface + engine consumers, CLI link-printing, e2e tests, docs/license/lockfile), plus a direct read of CI results and git history for the touched files. Implements ROCMAI-671 ("Serve browsable API docs for every running model server") in full — the ticket's description is the PR description verbatim, backfilled from GitHub.
Blocking
- The scenario the PR's own description names as the real-hardware verification for this feature fails on CI, on the exact commit under review —
tests/e2e-cucumber/features/model_serving.feature:73,tests/e2e-cucumber/tests/e2e/service_docs_steps.rs:288-289
This PR adds aThenstep, "the API docs link printed when it started opens the server's API docs," to both GPU serve canaries (model_serving.feature:46for vLLM/serve-05,:73for Lemonade/serve-07). The PR description states explicitly: "serve-07 is an expected failure on Linux Lemonade lanes for an unrelated reason, so there the check is effective only on Windows Lemonade lanes." TheE2E tests (Strix Halo, Windows, current)check on this PR's head commit (9ac33b1) failed:Reconciliation: ... 1 unexpected failure(s). FAIL: 'serve-lemonade-inference' was expected to pass on this host but FAILED — a regression.The captured output shows the model actually served and answered chat completions correctly (the two preceding assertions on the real chat response passed); only the new step panicked:`rocm serve` printed no API docs link, i.e.docs_link_in_serve_output(stdout)found nothing in the capturedrocm servestdout within whatever window the implementation waits.expectations.toml:96confirms Windows is not in the xfail scope for this scenario (when = { effective_engine = "lemonade", os = "linux" }), so this is a genuine, newly-introduced regression on exactly the lane the PR's own text claims validates the feature — not a known flake and not something the PR's xfail bookkeeping accounts for.
Confidence 100 · architectural · Fix: investigate why the Windows/Lemonade lane'srocm servestdout doesn't carry the docs link within the up-to-5s wait this PR implements (slow backend install before serve, as seen in the log, may be eating into or racing that window) and get this lane green, or narrow the description's "effective on Windows Lemonade lanes" claim if it cannot be fixed before merge.
Non-blocking
- The "shared route catalogue" this PR adds doesn't actually feed the docs server's rendered routes; two independent implementations are kept in sync only by a unit test —
crates/rocm-engine-protocol/src/lib.rs(OPENAI_CHAT_ROUTES/chat_supported_routes, new),apps/rocmd/src/api_docs.rs:430-435(routes_for's health-path dispatch),engines/vllm/src/state.rs,engines/lemonade/src/lib.rs.
A workspace-wide grep for.supported_routesturns up zero production read sites — the new catalogue only feedsEndpointResponse.supported_routes, a field nothing outside engine unit tests reads. The docs renderer that this PR's own description calls the consumer of "the route list now lives once inrocm-engine-protocol" in fact derives its per-engine health path independently:let health_path = if record.engine == "vllm" { "/health" } else { "/v1/health" };— a raw string match with an implicit "everything else is Lemonade" fallback, never reading the catalogue. The two are kept consistent today only bydocumented_routes_are_the_routes_the_engines_report(api_docs.rs:671-686); a third engine with a different health convention would get silently mis-documented unless someone remembers to touch this unrelated match and the test happens to catch it.
Confidence 70 · architectural · Fix: haveroutes_forread the health path from the engine's own reported route data (or arocm_engine_protocolhelper keyed by engine id) instead of re-deriving it fromrecord.engine, so the "single source of truth" the description claims is actually one.
Decisions for the author
- Cross-site
GETrequests with noOriginheader bypass the same-origin forwarder guard — tradeoff.apps/rocmd/src/api_docs.rs:171-173treats an absentOriginheader as allowed (headers.get(header::ORIGIN).is_none_or(...)), which a non-fetch/XHR cross-origin load (e.g. an<img src="http://127.0.0.1:11430/services/<id>/api/v1/models">on any other open tab) satisfies. Impact is low today — every forwarded route is read-only GET, and the gap is explicitly named in a comment atapi_docs.rs:159-164as a deliberate scope choice ("requests that are not from a browser carry no Origin and are not affected"). Worth confirming this remains acceptable if a state-changing route is ever added behind this forwarder. - Security-relevant behavior (Host/Origin/DNS-rebinding guards, the Authorize bearer-key flow) is covered only by crate-level HTTP integration tests, not a Gherkin scenario — non-blocking-improvement. AGENTS.md §3 requires a Gherkin scenario for user-observable behavior changes ("a unit test asserting the internal helper does NOT discharge this"). The existing tests in
apps/rocmd/src/api_docs.rsare real HTTP requests against a really-bound router (not internal-helper unit tests), and the PR description is accurate about this being unit-level coverage rather than overclaiming e2e coverage — so this isn't a description mismatch. But it is a judgment call whether security behavior answered by a browser request counts as the kind of "user-observable behavior" §3 means; worth the author's explicit confirmation either way. - The bundled Swagger UI third-party-notices section is guarded by a version-string match, not a content match — non-blocking-improvement.
the_bundled_swagger_ui_is_the_version_named_in_the_notices(api_docs.rs:717-739) only asserts the version string appears inTHIRD_PARTY_NOTICES.txt; it does not check the hand-copied sub-license list (classnames, buffer, DOMPurify, js-yaml, etc.) still matches that version's actual bundle. Nothing is stale today (independently verified: 5.32.6 is indeed whatutoipa-swagger-ui-vendored0.2.0 vendors), but a future version bump that changes the bundle's own third-party set would pass this test with stale notices. Consider noting the gap inMANIFEST.md's maintenance section, or parsing the bundle's own license comments for comparison. - No end-to-end scenario for
api_docs.enabled: false— non-blocking-improvement.docs/manual-testing.mdlists it as a manual check andapi_docs_link.rsunit-tests it, but none of the four newservice_docs.featurescenarios exercise the disabled-config path through a real daemon. Low-risk (thin wrapper around an already unit-tested config read). - IPv6 loopback (
[::1]) passes the Host/Origin validators but the server only ever binds IPv4127.0.0.1— non-blocking-improvement,apps/rocmd/src/api_docs.rs. Harmless today (nothing is listening on::1to reach), just a minor dual-stack completeness gap that could affect browsers that try::1first when resolvinglocalhost.
Positive signals
- The Host/Origin/DNS-rebinding defenses are adversarially tested against the named bypass patterns, not just the happy path: a test exercises the classic
127.0.0.1.evil.examplerebinding string and a cross-site POST rejection, both against a real forwarded request (api_docs.rs, tests around lines 699-715, 939-964). - The forwarder is proven immune to the proxy-bypass it claims: a dedicated test sets
HTTP_PROXY/ALL_PROXYto a dead endpoint and confirms the real request still succeeds via.no_proxy()(api_docs.rs:992-1047), rather than just asserting the client builder option is set. rocm serve's printed docs link is tied to the server actually having answered for that service, not printed unconditionally — the invariant is stated in the module doc, backed by the control flow, and covered by both positive and negative unit tests plus an e2e scenario that fetches the realswagger-initializer.jsrather than a stand-in (apps/rocm/src/api_docs_link.rs,tests/e2e-cucumber/features/service_docs.feature).- Every failure boundary (docs disabled, port 0, taken port, no helper running, another program on the port, config load failure) degrades to "no link" rather than a crash or a broken URL, each with its own test.
- Crate-layering is intact in both directions:
apps/rocm/Cargo.tomlis untouched by this PR, andapps/rocmd/Cargo.toml's new dependencies (reqwest,utoipa,utoipa-swagger-ui) are crates.io-only — no new first-party edge forcargo xtask check-crate-edgesto catch. - The ASCII-output-contract convention (
xtask/src/ascii_output_contract.rs, from the unrelated but highly detailed#683history on these same files) is respected: every non-ASCII character introduced by this PR is inside a comment or doc-comment, none inside a user-facing string literal.
Deployment notes
- A new default-on local listener is introduced:
api_docs.enabled: trueby default, binding127.0.0.1:11430. Loopback-only, but operators scanning for new listening sockets after upgrade should expect it. - New third-party bundled content ships in the binary (Swagger UI 5.32.6, Apache-2.0, vendored via
utoipa-swagger-ui'svendoredfeature) —about.hbs/THIRD_PARTY_NOTICES.txt/MANIFEST.mdupdated accordingly; see the notices-maintenance note above for what isn't automatically guarded on a future bump. - Three new crates.io dependencies land in
apps/rocmd(reqwest,utoipa,utoipa-swagger-ui);Cargo.lock's diff was checked and is narrowly scoped to them (no incidental version churn elsewhere).
What this covered
Read completely, against 1c33248c318c72a9e837627ff71476c8583f1930...9ac33b1a39589178aa8eacdd34dab3fe496eff80: every changed file in full at the PR head plus the pre-change version of each, via five independent workers grouped by coupling (apps/rocmd/src/api_docs.rs + its service/lib/Cargo.toml wiring; rocm-engine-protocol + rocm-core config + the two engine crates; apps/rocm's link-printing/serve-summary/service-launch cluster; the full e2e feature/step/naming-check set; README/MANIFEST/architecture docs/about.hbs/THIRD_PARTY_NOTICES/Cargo.lock), each also reading the relevant surrounding/caller code (e.g. webhook.rs for HTTP-server convention comparison, crate_edges.rs's ALLOWLIST, xtask/src/ascii_output_contract.rs's history). I additionally read AGENTS.md in full, checked git log/git blame on the touched pre-existing files for relevant prior conventions (this surfaced the ASCII-output-contract and ticket-ID history used above), read all CI check results for the head commit via live gh/GitHub-API queries including the actual failing job's log, and read the PR description and its named Jira ticket (ROCMAI-671) only after all findings above were reached independently from the code and CI.
Did not run: no Rust toolchain was available to any review worker in this sandbox, so no cargo build/check/clippy/test/xtask check-crate-edges was executed locally — all code-correctness findings are from static reading (quoted and line-cited) plus the one piece of dynamic evidence available, which is the project's own CI run on this exact commit. Analyze (rust), clippy, Test (affected crates), windows-build-and-test, and build-and-test all show passing on this commit's checks, which is taken at face value rather than re-derived. E2E tests (MI300X, current) and E2E tests (W7900, current) were still pending at review time and are not reflected above. Not reconciled against any PR discussion/comments — per the method, none were read.
|
🤖 Automated review (Claude Code) · 9ac33b1 I read the full diff (29 files, ~2.7k additions) with focus on the new security-sensitive surface: Non-blocking: Could not verify: I did not have a Rust toolchain/network access set up to run 🤖 Generated with Claude Code |
9ac33b1 to
a170585
Compare
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review (Claude Code) · a170585
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (32 files, +3125/-50) at HEAD a1705854a248dc6aa3568502610b53597564f36d. Implements ROCMAI-671 ("Serve browsable API docs (Swagger UI) for every running model server") in full; the ticket is still "In Progress," consistent with this PR being open.
Blocking
-
e2e-cucumber's test target does not compile under the project's own CI gate, on a function this PR adds —tests/e2e-cucumber/tests/e2e/service_docs_steps.rs:112,140
async fn missing_docs_link_evidence(world: &E2eWorld, stdout: &str) -> Stringholds&E2eWorldacrossclient.get(&url).send().awaitat line 140.E2eWorldcarries a field that isSendbut notSync(aportable_pty::MasterPtytrait object per the diagnostic), so the future is!Send, andclippy::future_not_sendfires as a hard error under-D warnings. Verified directly from the live CI job logs for this exact commit: theclippyjob and theWindows build and test (debug)job both fail identically —error: could not compilee2e-cucumber(test "e2e") due to 3 previous errors— quoting this same function/line. This also cascades to thewindows-build-and-testgate job (fails because itswindows-debuginput isfailure).
Confidence 100 · mechanical · Fix: stop holding&E2eWorldacross the.await— extract the few fieldsmissing_docs_link_evidenceactually needs (service id already comes fromstdout;paths/config/event-log reads can happen before the.awaitor via owned copies) so the future no longer captures a non-Sendreference. -
The new GPU-canary docs-link assertion fails on live CI at this exact commit, on the lane the PR's own text calls the only one that still exercises it — root-caused to the e2e harness's docs-port randomization silently no-op'ing —
tests/e2e-cucumber/tests/e2e/service_docs_steps.rs:75-94(give_api_docs_a_port_of_their_own),tests/e2e-cucumber/tests/e2e/serving_steps.rs:179,580(its call sites),crates/rocm-core/src/config.rs:93-97(RocmCliConfig.api_docs: ApiDocsConfigunder plain#[serde(default)]),docs/testing.md("effective on the Windows Lemonade lanes" claim), PR description (same claim, "serve-07 ... effective only on Windows Lemonade lanes").
give_api_docs_a_port_of_their_ownonly randomizes a port whenconfig["api_docs"]["port"]is absent fromconfig.json; its own doc comment frames the early-return as "a serve retried within one scenario reuses the helper the first attempt started." But#[serde(default)]onRocmCliConfig::api_docsonly fills a missing field on deserialize — it does not suppress the field on serialize — so the very first time anything in the scenario's isolated tree loads-then-savesconfig.json(which an ordinaryrocminvocation does), the file gains an explicit, literalapi_docs.portequal to the stock default (DEFAULT_API_DOCS_PORT = 11_430), indistinguishable from a port this function chose itself. From that point on the "already chosen" short-circuit returns11430instead of ever callingfree_port(). This matches live evidence precisely: theE2E tests (Strix Halo, Windows, current)job on this commit showsserve-lemonade-inference(serve-07) failing, with the helper's own logged event"action":"api_docs_unavailable","message":"...failed to bind API docs server to 127.0.0.1:11430: ...Only one usage of each socket address...(os error 10048)"and the scenario's ownconfig.jsondump reading"api_docs":{"enabled":true,"port":11430}— the un-randomized stock default, colliding with whatever else on the shared runner already held it.docs/testing.mdand the PR description both assert this exact lane ("Windows Lemonade") is the one place this check is "effective" — at this commit it is red for exactly the mechanism above, so that claim does not hold at HEAD.
Confidence 85 · architectural · Fix: makegive_api_docs_a_port_of_their_own(or its call sites) distinguish "already randomized by me this scenario" from "config.json merely contains the serialized default" — e.g. track the chosen port inE2eWorldinstead of re-reading it back out ofconfig.json, or always rewrite a non-zero sentinel/random port rather than trusting whatever key already exists. -
The docs renderer never reads the "single source of truth" route catalogue it documents itself as using, and the mismatch is reachable today through the already-shipping "aim" engine, untested —
apps/rocmd/src/api_docs.rs:500-542(routes_for),apps/rocmd/src/api_docs.rs:561-575(engine_docs_note),crates/rocm-engine-protocol/src/lib.rs:631-639(OPENAI_CHAT_ROUTES/chat_supported_routes),engines/aim/src/state.rs:66-72(EndpointResponse.supported_routesreports health at"/health"),apps/rocm/src/main.rs:166(SUPPORTED_ENGINES = ["lemonade", "vllm", "aim"]).
routes_for's doc comment says the routes come from "its health route plusrocm_engine_protocol::OPENAI_CHAT_ROUTES... The single place the docs learn what a server serves," but the function body never calls into that crate — it hardcodes the three OpenAI-style paths as string literals and picks the health path withif record.engine == "vllm" { "/health" } else { "/v1/health" }.engine_docs_noterepeats the same binary split, printing Lemonade/llama.cpp-specific prose for anything that isn't literally"vllm". Meanwhilelemonadeandvllmwere correctly updated to report their routes through the sharedchat_supported_routeshelper (confirmed in their diffs) — but a third engine,aim, is already a real, currently-shipping, selectable engine (rocm serve --engine aim) whose ownEndpointResponse.supported_routesnames its health route"/health", not"/v1/health". Becauseroutes_for's match only special-cases"vllm", an AIM-served model's generated docs page documents (and lets "Try it out" hit) the wrong health path, and describes the server with Lemonade/llama.cpp prose that is false for AIM's Docker-based backend. No test inapi_docs.rs's suite exercises"aim":chat_server_docs_list_its_routes_with_the_served_model,documented_routes_are_the_routes_the_engines_report, andengine_note_links_vllm_docs_and_llama_cpp_routesall iterate only["vllm", "lemonade"]. This also means the PR description's claim that "the route list now lives once inrocm-engine-protocol... used by both engines and tested against the docs" is accurate only for the engines' side; the docs side duplicates it independently and is kept consistent with the engines only by a test that itself never covers the third engine.
Confidence 85 · architectural · Fix: haveroutes_for/engine_docs_noteread the health path (and backend description) from the engine's own reportedsupported_routes/arocm_engine_protocolhelper keyed by engine id, instead of re-deriving it from a two-wayrecord.enginematch; add an"aim"case to the three tests above.
Non-blocking
None
Decisions for the author
- The forwarder accepts any HTTP method and treats an absent
Originas allowed — tradeoff.apps/rocmd/src/api_docs.rs:173-176(require_loopback_host) and the router's.route("/services/{id}/api/{*path}", any(forward)...): this is not limited to read-onlyGET— it forwardsPOST /v1/chat/completions(model-invoking, cost-bearing) and any other method. A same-origin "Try it out" page orcurlcarries noOriginand is intentionally admitted; the in-code comment frames this as deliberate for non-browser tooling. Worth the author's explicit confirmation that the scope is still right now that the forwarder's surface is confirmed to be more than GET. - A taken docs port leaves docs permanently off for that daemon's lifetime rather than retrying another port — tradeoff, directly relevant to the finding above: predictable, bookmarkable addressing (
127.0.0.1:11430) vs. resilience to the exact kind of port collision seen in CI. Confirm the fixed-address choice is still preferred. - The bundled-Swagger-UI notices are guarded by a version-string match only — non-blocking-improvement.
the_bundled_swagger_ui_is_the_version_named_in_the_notices(apps/rocmd/src/api_docs.rs:792-813) asserts the served version string appears inTHIRD_PARTY_NOTICES.txt, not that the hand-copied sub-license list (classnames, buffer, DOMPurify, js-yaml, React, …) still matches that version's actual bundle. Nothing is stale today; a future version bump that changes the bundle's own third-party set would pass this test while leaving stale sub-license entries. Consider noting the gap inMANIFEST.md's maintenance section. - IPv6 loopback (
[::1]) passes theHost/Originvalidators but the server only binds IPv4127.0.0.1— non-blocking-improvement,apps/rocmd/src/api_docs.rs. Harmless today (nothing listens on::1to reach); a minor dual-stack completeness gap.
Positive signals
- Every blocking/non-blocking item a stale round of review raised against an earlier commit of this same branch has been independently re-verified as fixed here, each now behind its own regression test: the
docs_redirectpanic on a crafted id (docs_redirect_only_names_a_running_server, testing literal%0A/%0D%0Ainjection), unreadable service records surfacing as 503 instead of silently "no servers" (unreadable_records_are_reported_not_shown_as_no_servers), daemon shutdown no longer blocking on an open try-it stream (stopping_does_not_wait_for_an_open_try_it_stream, bounded bySHUTDOWN_GRACE), the live/liveness predicate unified into oneManagedServiceRecord::is_live()used by bothapps/rocmandapps/rocmd(crates/rocm-core/src/service_record.rs), and--helptext updated forrocm serve/rocm daemon/rocm services list. - The
Host/Origin/DNS-rebinding guard is adversarially tested against concrete bypass strings (127.0.0.1.evil.example, a craftedHostvia percent-encoded newline, a cross-sitePOST, an absent-vs-same-originOrigin) against a really-bound router, not just unit-level string matching. - The proxy-bypass claim is backed by a test that actually fails when
.no_proxy()is removed (502), with careful env-var save/restore to avoid cross-test interference, rather than only asserting the builder option was set. apps/rocm/Cargo.tomlis untouched; the new HTTP/OpenAPI dependencies land only inapps/rocmd, preserving therocmd-must-never-depend-on-rocmlayering invariant without needing acargo xtask check-crate-edgesallowlist change.
Deployment notes
- A new default-on loopback listener is introduced (
127.0.0.1:11430by default,api_docsinconfig.json, can be disabled). - New bundled third-party web assets ship in the
rocmdbinary (Swagger UI 5.32.6, Apache-2.0, vendored viautoipa-swagger-ui'svendoredfeature);about.hbs/THIRD_PARTY_NOTICES.txt/MANIFEST.mdupdated accordingly. - Three new crates.io dependencies land in
apps/rocmdonly (reqwest,utoipa,utoipa-swagger-ui); theCargo.lockdiff is scoped to that dependency tree (two coexisting versions ofbase64/wasm-streamsare normalcargo-about/lockfile output for differing transitive requirements, not unrelated churn).
What this covered
DEGRADED REVIEW (partial fan-out failure) — every Agent spawn attempt (two batches, nine workers total, across roughly 15 minutes) failed immediately with "Concurrent subagent limit reached," confirmed via repeated ListAgents calls showing this sandbox's 20-concurrent ceiling permanently saturated by unrelated sessions sharing the same pool, never freeing a slot across the whole review. No fan-out worker of mine ever started, so this is entirely a single-context (driver-only) review rather than the coupled-file-group fan-out the method calls for. I compensated by reading, myself, every one of the 32 changed files in full (the five new files — apps/rocmd/src/api_docs.rs all 1342 lines, apps/rocm/src/api_docs_link.rs, tests/e2e-cucumber/tests/e2e/service_docs_steps.rs, tests/e2e-cucumber/features/service_docs.feature, and the about.hbs/THIRD_PARTY_NOTICES.txt/MANIFEST.md additions) or the complete diff plus surrounding/caller context (every changed file), against 9875940ebc7d1dbf9db0706e01d29842e8ec1d27...a1705854a248dc6aa3568502610b53597564f36d; cross-file consumer searches (git grep at the PR head) for chat_supported_routes/OPENAI_CHAT_ROUTES, is_live, and the "aim" engine's selectability; the root AGENTS.md in full; and live CI job logs for this exact commit read directly (not just the status-check rollup) for clippy, Windows build and test (debug), windows-build-and-test, E2E tests (Strix Halo, Windows, current), and E2E tests (Strix Halo, Ubuntu, nightly, current) (the last of these failed on "No space left on device," an infrastructure flake unrelated to this diff, not raised as a finding). Recommend re-running the fan-out passes once subagent capacity is available, particularly a dedicated pass over apps/rocm/src/main.rs/serve_cmd.rs (read here only via diff-plus-context, not in full, since both files are very large) and a git log -p/git blame history pass per file, neither of which I completed to the method's normal standard in degraded mode.
One unavoidable step exposed prior-round text: listing existing reviews with perPage=100 (required to find the current HEAD's state and avoid double-posting) returned the full bodies of two earlier reviews on this PR, both at an older commit (9ac33b1a) — one flagging a real failing e2e lane among other things. All findings above were independently re-derived from the code and from live CI at the current head; in several cases my own reading reached the same conclusion that review had already reached (re-verified fresh, not assumed), in several cases I confirmed something it flagged is now fixed (see Positive signals), and the three Blocking findings above are either new (the aim-engine mismatch, never raised before) or reach the same CI lane by a different, independently-traced root cause (a concrete port-randomization bug rather than a timing/wait-window guess). Not reconciled against any PR discussion or review comments beyond that one mechanical, disclosed exception.
🤖 Generated with Claude Code
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review (Claude Code) · 981ca91
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (32 files, +3137/-50) against base 95e63388, head 981ca91e. This is round 3 on this PR (prior rounds at 9ac33b1a and a1705854a); several earlier findings are now fixed, but three blocking issues raised in earlier rounds are still present at this exact head, plus one new one. Implements ROCMAI-671 ("serve browsable API docs for every running model server") in full for the vLLM/Lemonade case; see the first Blocking item below for where the "single source of truth" claim doesn't hold for the third shipping engine.
Blocking
-
routes_for/engine_docs_notemis-document the already-shipping "aim" engine — standing from round 2, still unfixed —apps/rocmd/src/api_docs.rs:505-509,562-575
Both functions special-case onlyrecord.engine == "vllm"and treat every other engine as Lemonade.aimis a real, selectable engine (apps/rocm/src/main.rs:166,SUPPORTED_ENGINES = ["lemonade", "vllm", "aim"]) whose ownEndpointResponse(engines/aim/src/state.rs:66-72) reports its health route at/health, not/v1/health— so an AIM-served model's generated docs page documents (and lets "Try it out" hit) the wrong health path, andengine_docs_noteprints false prose claiming the server "runs with the llama.cpp server," which is meaningless for AIM's Docker-based backend. The PR's own description framesrocm_engine_protocol::chat_supported_routesas "the single place the docs learn what a server serves... used by both engines and tested against the docs" — true for vllm/lemonade (both were migrated onto the shared helper), false for aim, whose ownEndpointResponsekeeps a hand-written duplicate list never routed throughroutes_for. None of the three relevant tests (chat_server_docs_list_its_routes_with_the_served_model,documented_routes_are_the_routes_the_engines_report,engine_note_links_vllm_docs_and_llama_cpp_routes) include an"aim"case — adding one would fail today, confirming this is live, not hypothetical.
Confidence 88 · architectural · Fix: haveroutes_for/engine_docs_noteread the health path from the engine's own reportedsupported_routes(or arocm_engine_protocolhelper keyed by engine id) instead of a two-wayrecord.enginematch; add an"aim"case to the three tests above. -
The e2e harness's docs-port randomization silently no-ops the first time any scenario saves
config.json— standing from round 2, still unfixed —tests/e2e-cucumber/tests/e2e/service_docs_steps.rs:75-94(give_api_docs_a_port_of_their_own),crates/rocm-core/src/config.rs(ApiDocsConfigunder plain#[serde(default)])
The helper only randomizesapi_docs.portwhen the key is absent fromconfig.json; it treats presence as proof "this scenario already chose a port." ButRocmCliConfig::save()always serializes the whole struct, so the very firstrocminvocation that loads-then-saves a fresh config (e.g."a managed runtime is active"'srocm runtimes activate, which every GPU canary scenario runs first) writes the stock default (DEFAULT_API_DOCS_PORT = 11430) to disk — indistinguishable from a chosen port. From then on,serve_gpu_model/serve_and_wait's call to this helper returns11430instead of callingfree_port(), defeating the isolation this function exists to provide, for exactly the two scenarios (serve-05,serve-07) this PR adds docs assertions to, and for every other GPU scenario that happens to run after an activate/install step. This was the traced root cause of a Windows-lane CI failure in the previous review round; no CI has yet run on the current head to show whether it still reproduces (see next finding), but the code path itself is unchanged.
Confidence 80 · architectural · Fix: track "did this harness already choose a port for this scenario" inE2eWorldinstead of re-deriving it from whatever happens to already be on disk. -
A probe-removal mutation leaves a test green — standing from round 1, still unfixed —
apps/rocm/src/api_docs_link.rs:240-253(no_link_when_nothing_listens_on_the_docs_port)
The test only asserts the return value isNoneand that it returned in under 2s; it cannot distinguish "probed and got refused" from "never probed at all." Ifservice_docs_link/service_docs_links's actual TCP connect were deleted and the functions made to unconditionally returnNone/empty, this test would still pass unchanged. Contrast with the file's owna_docs_server_that_is_down_is_asked_once_for_the_whole_list, which does instrument a connection counter and genuinely verifies the probe ran.
Confidence 85 · test-quality · Fix: have the stand-in server count connections and assert exactly 1, as the sibling test already does. -
No CI check has run for this exact head commit —
981ca91ea84e8074c63fa8d70ca0383384fbfb6f
get_check_runsreturns zero check runs for this SHA. None ofbuild-and-test,clippy, the Windows lane, or any E2E lane (including the two GPU canaries this PR's own description names as its real-hardware verification) have reported one way or the other for the commit under review — including whether the port-randomization bug above still reproduces as it did at the prior head.
Confidence 100 · Fix: push a no-op or otherwise re-trigger CI for this head and confirm green (in particular the two GPU canaries and the e2e consolidated report) before merge.
Non-blocking
- The new docs-link wait (up to 5s) is not gated on terminal interactivity, unlike every other progress indicator in the same call paths —
apps/rocm/src/serve_cmd.rs:1225-1230,apps/rocm/src/service_launch.rs:1097-1111(run_attached_service)
Both callservice_docs_link(...)unconditionally, including in the plain/piped branch that the surrounding code's own comments describe as "CI, the chat assistant."cli_progress::Spinneris hard-gated onstderr().is_terminal(), so in a piped session the wait still runs (and blocks for up to 5s on a cold helper start) with zero visible progress output. No test exercises "helper cold-starting + non-interactive/attached serve" (everyservice_docs.featurescenario starts withGiven the background helper is running).
Confidence 75 · logic · Fix: gate the wait behind the same interactivity check the rest of the function already uses, or shorten/skip it for non-TTY output.
Decisions for the author
- The forwarder accepts any HTTP method and treats an absent
Originas allowed — tradeoff,apps/rocmd/src/api_docs.rs:171-176. ForwardsPOST /v1/chat/completions(model-invoking) and any other method, not just read-onlyGET; a same-origin page orcurlcarries noOriginand is intentionally admitted, per the code's own comment. Worth re-confirming this scope now that the surface is more than GET. - A taken docs port leaves docs off for the daemon's whole lifetime rather than retrying another port — tradeoff. Predictable, bookmarkable addressing vs. resilience to exactly the collision class the port-randomization bug above can produce.
rocmdnow bundles two separate HTTP-client stacks (reqwestfor the new try-it forwarder, pre-existingureqelsewhere) — non-blocking-improvement. TheCargo.tomlcomment justifies this (thestreamfeatureureqlacks), reasonable but worth noting as accepted dependency growth.- IPv6 loopback (
[::1]) passes the Host/Origin validators but the server only ever binds IPv4127.0.0.1— non-blocking-improvement, harmless today since nothing listens on::1to reach, but a dual-stack completeness gap for browsers that try::1first when resolvinglocalhost.
Positive signals
- Two defects from earlier review rounds are now genuinely fixed and independently re-verified this round: the
docs_redirectpanic on a crafted service id (now covered bydocs_redirect_only_names_a_running_server, testing literal%0A/%0D%0Ainjection), and theclippy::future_not_sendfailure inmissing_docs_link_evidence(confirmed by checking every async fn and every&E2eWorld/&mut E2eWorld-taking function in the file — the fix is complete and nothing else in the file reintroduces the pattern). ManagedServiceRecord::is_live()is a clean, correctly-reused consolidation of a liveness predicate that was previously duplicated betweenapps/rocmandapps/rocmd.- The Host/Origin/DNS-rebinding guard is adversarially tested against concrete bypass strings (
127.0.0.1.evil.example, a cross-site POST, an absent-vs-same-originOrigin) against a really-bound router, and the proxy-bypass claim is backed by a test that actually fails when.no_proxy()is removed (502), not just an assertion the builder option was set. - Crate layering is intact both ways:
apps/rocm/Cargo.tomlis untouched, and the three new crates.io dependencies (reqwest,utoipa,utoipa-swagger-ui) land only inapps/rocmd. Cargo.lock's diff is narrowly scoped to the new dependency tree,MANIFEST.md's new dependency-table rows match it exactly, and the new README/docs sections (API docs defaults, forwarder guarantees,persistAuthorization: false) were spot-checked against the actual test suite and found accurate with no overclaiming.- Every failure boundary (docs disabled, taken/zero port, no helper, config load failure) degrades to "no link" rather than a crash, each backed by its own test.
Deployment notes
- New default-on loopback listener:
api_docs.enabled: trueby default, binding127.0.0.1:11430(api_docsinconfig.json, can be disabled). - New bundled third-party web assets ship in the
rocmdbinary (Swagger UI 5.32.6, Apache-2.0, vendored viautoipa-swagger-ui'svendoredfeature);about.hbs/THIRD_PARTY_NOTICES.txt/MANIFEST.mdupdated accordingly. - No CI has run on this exact head commit — see Blocking.
What this covered
Every file in the diff read completely at head 981ca91e against base 95e63388, via five independent fan-out workers grouped by coupling: (A) apps/rocmd/src/api_docs.rs + its daemon/lib/Cargo.toml wiring; (B) rocm-engine-protocol + rocm-core config + the engine crates (lemonade/vllm/aim); (C) apps/rocm's link-printing/serve-summary/service-launch cluster; (D) the full e2e feature/step files; (E) README/MANIFEST/docs/about.hbs/THIRD_PARTY_NOTICES/Cargo.lock. Each worker also grepped the whole tree for consumers of changed symbols outside its own group; all candidates above were independently re-verified against the source before inclusion, and cross-checked against the two prior review rounds on this PR only to confirm which of their findings are now fixed (not to anchor new findings on their text).
Did not run: cargo build/test/clippy — no Rust toolchain available in this environment; all findings are from direct source reading (the aim-engine mismatch and the e2e port bug were both traced by hand through the actual call chains, not assumed). CI was read via the Checks API (zero runs for this head — see Blocking) rather than assumed. Not reconciled against this PR's own review discussion beyond the minimum needed to confirm which earlier findings are now fixed.
🤖 Generated with Claude Code
`rocm services docs [<id>]` serves browsable API docs (Swagger UI) for the running model servers on 127.0.0.1, in the foreground, until Ctrl+C. Each server's page lists the routes it answers, with request examples filled in with the served model, and its "Try it out" forwards requests through the page to the server, so it works for every engine whatever the engine's own cross-origin settings. Examples, titles and labels use the name the server answers to, which for AIM is not the image reference. The command refuses, before serving anything, when the named server is not running (naming `rocm services restart <id> --yes`) or when no server is. It uses port 11430, or a free port when 11430 is in use and says so on stderr; `--port` picks one and never falls back. Nothing runs in the background: after Ctrl+C nothing is left serving and the model servers are not touched. The docs judge which servers are running with the loader `rocm services list` uses, which checks each server's process and endpoint, so a server that stops or crashes while the docs are open drops off them about a second after the read that finds it gone. Requests that arrive together share one read. `rocm serve` (summary, plain and attached output) and `rocm services list` print a `docs:` line naming the command for every running server, and an `engine docs:` line with vLLM's own /docs page. The routes the docs describe come from one table in rocm-engine-protocol: the OpenAI chat routes plus each engine's health route. The engines report the same table as `supported_routes` in their `endpoint` answer, which a test in `rocm` asks each engine for and compares with the docs. The forwarder passes on only Content-Type, Accept and Authorization, keeps the request path percent-encoded, accepts bodies up to 64 MiB, does not follow redirects, gives up on a server that does not accept the connection within 5 s, and bypasses any configured proxy. It reaches a server bound to `0.0.0.0` on 127.0.0.1 and one bound to `::` on ::1, so an IPv6-only socket is reached too; that mapping is now one helper in rocm-core (`reachable_host`), which the Anthropic proxy uses as well. Browser requests are admitted only from the docs server's own pages: by `Origin` when the browser sends one, and by `Sec-Fetch-Site` when it does not. The `Host` guard keeps DNS rebinding out. Every answer carries `frame-ancestors 'none'`, `X-Frame-Options: DENY` and nosniff, and forwarded answers are also CSP-sandboxed. Server ids are percent-encoded in links and escaped in pages. Swagger UI is vendored (no CDN); its licence is in the notices. Scenarios service-docs-01 to -11 cover the listed command opening the docs, try-it, a server stopped while its docs are open, refusals made before any port is used, Ctrl+C stopping the docs while the model server keeps answering, the fallback from a taken port 11430, the `engine docs` link, the advice given for a stopped server, and the rows `rocm serve` prints when the model is already running, piped and attached. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
981ca91 to
588a3c6
Compare
CI's per-crate clippy run over the e2e harness (`cargo clippy -p e2e-cucumber --test e2e`) rejects `.map(..).unwrap_or_else(..)` on an Option; use `map_or_else` instead. No behaviour change. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Security scan found no critical or high findings on lines changed by this pull request3 medium, low, or unrated finding(s) are not listed; the scan run has them. View the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
rocm services docs (ROCMAI-671)
|
@tomastola thanks for the review. The branch is rebuilt as one commit, and the reason is your first decision below, so the shape has changed since "Docs are served only by the persistent helper ( "The Origin check admits any loopback origin on any port ... Confirm this is the intended boundary." Forwarded responses: they carry Size / licence split: the licence work stays here, since the notices only describe the Swagger UI and crates this change adds. The helper wiring, the link probe and the Your blocking findings from the first round, as they stand now:
|
|
Replies to the automated review at Blocking
Non-blocking
Decisions
|
|
Replies to the earlier automated reports on this PR (the comment at
Windows Lemonade canary printing no docs link (
Route catalogue not feeding the docs, and Decisions
|
juhovainio
left a comment
There was a problem hiding this comment.
Review — no blocking findings
Full review of the whole change (40 files, +4467/-47), base ee605065 (merge-base with main), head 9d4e2957. Implements ROCMAI-671 ("Serve browsable API docs (Swagger UI) for every running model server") for the scope the PR describes; the ticket is "In Progress," consistent with this PR being open.
Blocking
None
Non-blocking
- Two unreachable defensive branches —
apps/rocmd/src/api_docs.rs:498-505(docs_redirect'sHeaderValue::try_fromErrarm) andapps/rocmd/src/api_docs.rs:571,628-640(raw_forwarded_path'sNonearm). Both look unreachable given the current call sites (service_pathonly emits valid header bytes; axum's route pattern guarantees the 5-way split always matches). Harmless dead code, not functional bugs — see inline comments. chat_routes()hardcodes its three OpenAI route literals instead of readingrocm_engine_protocol::OPENAI_CHAT_ROUTES—apps/rocmd/src/api_docs.rs:677-700. The sibling health-route lookup just above it (routes_for, line 666) correctly reads from the new shared table; the chat routes don't. Kept consistent today only by tests, not by construction — see inline comment.- A second, pre-existing engine→health-path mapping doesn't read from the new shared table —
apps/rocm/src/engine_runtime_select.rs:395-399(service_http_readiness_paths), used for serve-readiness polling. This file has no diff against the merge-base (not touched by this PR) and the extra probe paths it tries for non-Lemonade engines are currently inert, so there's no live behavior bug — but it means the route list doesn't literally "live once in rocm-engine-protocol" repo-wide, only for the two things this PR touches (enginesupported_routesand the docs server's health route). - No per-engine unit test directly asserts
supported_routes()content inengines/aim,engines/lemonade, orengines/vllmthemselves — that invariant is verified only byrocm-engine-protocol's own table test and by a cross-crate integration test (apps/rocm/src/services_docs.rs) that drives each engine's real stdio protocol. The cross-crate test does catch a regression end-to-end, so this is a coverage-layering gap rather than an untested behavior. README.mdline-wrap convention broken at 3 spots introduced/touched by this PR (lines 1256, 1585, 1590 — 87/100/107 columns vs. the surrounding ~75-80). Cosmetic only.
Decisions for the author
utoipa-swagger-ui-vendoredappears inMANIFEST.mdbut not inTHIRD_PARTY_NOTICES.txt's generated section — tradeoff/unconfirmed. Plausibly correct if it's a build-dependency ofutoipa-swagger-ui(about.toml'signore-build-dependencies = truewould then explain the asymmetry), but this could not be confirmed without registry/network access to the crate's real manifest. Worth a quick sanity check before merge.- The "try it out" forwarder accepts any HTTP method and any path under
/api/..., not just the documented GET routes — tradeoff,apps/rocmd/src/api_docs.rs. Low incremental risk since the model server is already directly loopback-reachable by the same local user, but worth confirming the scope is still intended now that it's more than read-only GET. - Forwarded requests have only a 5s connect timeout, no overall request timeout — tradeoff, needed to support streaming chat completions; documented and tested, just worth the author reconfirming it's the intended tradeoff.
- An absent
Originand absentSec-Fetch-Sitetogether are treated as same-origin ("curl") and admitted — tradeoff, narrow gap for pre-Fetch-Metadata browsers on a cross-site GET navigation; low blast radius since only GET routes are reachable that way, and it's a deliberate, commented choice.
Positive signals
- Deep, adversarial security test coverage: DNS-rebinding (
127.0.0.1.evil.example-style Host headers), cross-siteOriginrejection, the fullSec-Fetch-Siteadmission matrix, and CSP/X-Frame-Options/nosniffasserted on every route including error pages — not just the happy path. - The proxy-bypass claim (
.no_proxy()) is backed by a test that actually fails when the option is removed, not just an assertion the builder flag was set. reachable_host(crates/rocm-core/src/net.rs) is a clean de-duplication: it replaces logic that was previously inline-duplicated inapps/rocmd/src/anthropic_proxy.rs, confirmed to be the only two call sites repo-wide.- The engine health-route centralization (
ENGINE_HEALTH_ROUTESinrocm-engine-protocol) is a correct, behavior-preserving extraction for all three engines, includingaim— it is not special-cased or defaulted, and is guarded by a bidirectional test (every_supported_engine_has_a_health_route_and_no_other_engine_does) plus an integration test that drives each engine's real stdio protocol and diffs the result against the docs server's computed routes. - The new GPU-canary e2e step (
serve-05/serve-07/serve-26) extracts every documented GET route from the live OpenAPI spec and round-trips each through the real engine via the docs proxy, rather than only checking that a page renders — it would catch a docs page that renders but lists stale or wrong routes. - The Swagger UI version-consistency test extracts the real vendored bundle's version string at test time and compares it against
THIRD_PARTY_NOTICES.txt, rather than relying on a coincidental literal match. - Crate layering is respected both ways: the new
reqwestdependency isdefault-features = false(no TLS pulled in), and no new dependency edge fromrocmdontorocmis introduced. docs/architecture.mdentries for both new production files (apps/rocmd/src/api_docs.rs,apps/rocm/src/services_docs.rs) are present and accurate, as are the other touched entries.
Deployment notes
- A new default-on loopback HTTP listener is introduced: binds
127.0.0.1:11430by default, falling back to a free port only when no--portwas given and 11430 is taken; an explicit--portnever falls back and fails instead. - Three new crates.io dependencies land in
apps/rocmdonly:reqwest(no TLS,streamfeature),utoipa,utoipa-swagger-ui(vendored, so no network fetch at build time). TheCargo.lockdiff is narrowly scoped to their transitive tree. - New bundled third-party web assets ship in the
rocmdbinary (Swagger UI, Apache-2.0);THIRD_PARTY_NOTICES.txt/about.hbs/MANIFEST.mdupdated accordingly (one item above for the author to double-check).
What this covered
Every file in the diff read completely at head 9d4e2957 against the merge-base ee605065, via 5 independent fan-out workers grouped by coupling: (A) apps/rocmd/src/api_docs.rs + its daemon/lib/Cargo.toml wiring + rocm-core's net.rs/service_record.rs/lib.rs; (B) rocm-engine-protocol + the three engine crates (aim/lemonade/vllm); (C) apps/rocm's CLI/link-printing/serve-summary/service-launch/services_docs cluster; (D) the full e2e-cucumber feature/step/naming files; (E) README.md/MANIFEST.md/docs/about.hbs/THIRD_PARTY_NOTICES.txt/Cargo.lock. Each worker read its files completely (one pre-existing, untouched file's unchanged test module — anthropic_proxy.rs — was sampled rather than read in full) and grepped the repo for cross-file consumers of changed symbols; every candidate finding was independently re-verified against the source before inclusion here.
CI was checked live via the GitHub API for this exact head commit: every reported check (build-and-test, clippy, Windows build/test, E2E tests, E2E consolidated report, security scans, commit-signatures, etc.) is green, none failing.
Did not run: no Rust toolchain was invoked by me or any worker in this environment, so no cargo build/test/clippy was executed locally — all code-correctness findings come from static reading of the source (quoted/line-cited) plus the project's own live CI results on this commit, taken at face value rather than re-derived.
One procedural note for transparency: listing this PR's existing reviews (required to confirm this account hadn't already reviewed it, to avoid double-posting) surfaced the text of earlier review rounds at prior commits on this branch. All findings above were independently reached by the 5 fan-out workers from fresh reading of the current code and live CI, not from that text. Where this review's own findings happen to overlap with something flagged in an earlier round (e.g. the engine health-route centralization, previously inconsistent for a third engine, now confirmed correct for all three), that is noted as independently re-verified, not assumed. Not otherwise reconciled against this PR's review-comment discussion.
🤖 Automated by an AI agent
| ) | ||
| .into_response(), | ||
| Err(_) => not_running_page(&service_id), | ||
| } |
There was a problem hiding this comment.
Non-blocking (mechanical, confidence 75): the Err branch of HeaderValue::try_from(...) here looks unreachable given the current service_path implementation — it only ever emits ASCII alnum/-._~//%XX bytes, all valid HeaderValue content, so nothing can currently drive this into not_running_page. No test exercises this arm. Not a functional bug, just dead defensive code — consider a short comment noting why it's kept (e.g. "defensive, should be unreachable") or dropping it.
| } | ||
|
|
||
| /// The part of `/services/<id>/api/<path>` after `api/`, still percent-encoded. | ||
| fn raw_forwarded_path(request_path: &str) -> Option<&str> { |
There was a problem hiding this comment.
Non-blocking (mechanical, confidence 70): the None arm this function can return (consumed at line 571 as a 404) looks unreachable in practice — axum's route pattern /services/{id}/api/{*path} guarantees the 5-way split here always matches (Some(""), Some("services"), Some(_), Some("api"), Some(path)). Harmless dead code, not a functional bug; a short comment noting the fallback is defensive would save the next reader the trace.
| health.into_iter().chain(chat_routes()).collect() | ||
| } | ||
|
|
||
| fn chat_routes() -> [RouteDoc; 3] { |
There was a problem hiding this comment.
Non-blocking (architectural, confidence 80): chat_routes() hardcodes the three OpenAI route literals (/v1/models, /v1/chat/completions, /v1/completions) instead of mapping over rocm_engine_protocol::OPENAI_CHAT_ROUTES, even though routes_for just above (line 666) correctly reads the health route from the shared engine_health_route helper. The protocol crate's own doc comment on OPENAI_CHAT_ROUTES says the API docs describe the same list — today that's true only because tests (documented_routes_are_the_routes_each_engine_reports and similar) keep the two in sync, not because this function reads from the shared constant. Worth deriving this [RouteDoc; 3] from OPENAI_CHAT_ROUTES directly so the "single source of truth" the PR describes is structural rather than test-enforced.
| quick smoke test (time to first token and approximate tokens per second). Control | ||
| with the status, the engine, the API-qualified model name, the full inference | ||
| endpoint, the command that serves the server's API docs (see API docs below), | ||
| and a quick smoke test (time to first token and approximate tokens per second). Control |
There was a problem hiding this comment.
Non-blocking (mechanical, confidence 85): this line and two others touched by this PR break the surrounding paragraphs' ~75-80 column wrap convention: this line runs to 87 columns, line 1585 ("...vLLM also documents its full API at") to 100, and line 1590 ("...work offline. They use port 11430, or a free port when 11430 is") to 107. Purely cosmetic — worth re-wrapping to match the rest of the file.
Why
rocm servehands the user an endpoint URL and nothing that says what it accepts. Of the servers ROCm CLI runs, only vLLM generates API docs; Lemonade's router and the llama.cpp server behind it serve none, so on most hosts there is no reference to read and no way to try a request without writing one by hand.What changes
rocm services docs [<id>] [--port <port>]serves Swagger UI for the running model servers on127.0.0.1, in the foreground, until Ctrl+C./lists the running servers;/services/<id>/docs/is one server's page;/services/<id>/openapi.jsonits description. Without an id it links every running server (the docs of every running server stay reachable on that port either way). It refuses, before serving anything, when the named server is not running, namingrocm services restart <id> --yes, or when none is. It uses port 11430, or a free port when 11430 is in use and says so on stderr;--portnever falls back, and no other startup failure falls back either. After Ctrl+C nothing is left serving and the model servers are not touched.rocm services listuses (process and endpoint checks), passed in to the docs server, so a server that stops or crashes drops off about a second later. Requests that arrive together share one read of the servers.rocm serve(summary, plain and attached output) androcm services listprintdocs: rocm services docs <id>for each running server, andengine docs: <url>/docsfor vLLM.rocm services list --helpdescribes both lines.--jsonis unchanged.utoipafrom one route table inrocm-engine-protocol(OPENAI_CHAT_ROUTESplus each engine's entry inENGINE_HEALTH_ROUTES). The three engines report the same table assupported_routesin theirendpointanswer. Examples, the page title and the index label use the name the server answers to (served_model_name), which for AIM is not the image reference recorded as the canonical id; a server started with an endpoint key gets an "Authorize" box, and the key is not persisted in the browser./services/<id>/api/...) that streams the answer back, so it works on every engine whatever its cross-origin settings. It passes on onlyContent-Type,AcceptandAuthorization, keeps the path percent-encoded, accepts bodies up to 64 MiB, does not follow redirects, gives up after 5 s on a server that does not accept the connection, and bypasses any configured HTTP proxy. A server bound to0.0.0.0is reached on127.0.0.1, one bound to::on::1; that mapping is one helper inrocm-core(reachable_host), which the Anthropic proxy now uses as well.Origin, or bySec-Fetch-Site(same-origin/none) when there is noOrigin. TheHostguard refuses non-loopback names (DNS rebinding). Every answer carriesframe-ancestors 'none',X-Frame-Options: DENYandnosniff; forwarded answers are also CSP-sandboxed. Ids are percent-encoded in links and escaped in pages.ManagedServiceRecordgainslocal_base_urlandengine_docs_url.rocm services docsis added to the command surface the e2e coverage report counts (crates/e2e-report).utoipa-swagger-uiwithvendored, so the build downloads nothing and the page works offline. It is not a crate, socargo-aboutcannot see it: its Apache-2.0 NOTICE and the notices its bundle retains are in a hand-maintained "Bundled web assets" section ofabout.hbs, documented inMANIFEST.md. A test fails if the served version stops matching the notices.rocmd(and sorocm):utoipa,utoipa-swagger-uiandreqwest(no TLS,stream).Verification
cargo test --workspace --all-targets: all pass except three tests that fail on the test host for reasons unrelated to this change (stop_of_a_container_engine_is_unconfirmed_when_the_engine_stop_fails, where the host's podmandockershim ignoresDOCKER_HOST, and twobuild_script::tests::git_watch_paths_*, wheregit commitfails in the test's temp repo); on that heavily loaded host the*_never_reuses_another_writers_temp_filetiming probes also report themselves inconclusive at times.cargo fmt --check,cargo clippy --workspace --all-targets -D warnings,check-architecture-doc,check-module-sizeandcheck-crate-edgespass, andcargo xtask e2e -- -n "service-docs-"passes 11/11.cargo xtask smokewas not run on that host.Content-TypeorAcceptfrom the allowlist;::on127.0.0.1instead of::1;Sec-Fetch-Site, and dropping either anti-framing header;the_docs_list_what_the_loader_reports), and giving the docs a loader that reads records as written instead ofrocm services list's (a_server_whose_process_died_has_no_docs);endpointanswer reporting another engine's routes (documented_routes_are_the_routes_each_engine_reports, which asks each engine for that answer the way the CLI does);requests_within_the_ttl_share_one_read_of_the_servers: five requests at once against a slow loader read once, and once more after the TTL).service_docs.feature,cargo xtask e2e -- -n "service-docs-"), each on a port of its own:service-docs-01: the commandrocm services listnames opens docs listing the chat routes with the server's model.service-docs-02: try-it reaches the model server.service-docs-03: a stopped server's docs say it is not running and it gets nodocs:line (Linux only, it stops a process).service-docs-04/05: an unknown server, or no server, is refused while another program holds the port given, so a command that served before refusing would fail on the port instead. Checked red with serving moved before the refusal.service-docs-06: Ctrl+C printsAPI docs stopped., closes the port, and the model server still answers and is still listed (Linux only, it sends SIGINT). Checked red with the message removed and with Ctrl+C not ending the command.service-docs-07: with port 11430 held, the command says so and its link opens on a free port (@serial), and the note is on stderr, not among the links on stdout. Checked red with the note removed and with it printed on stdout.service-docs-08: theengine docslinkrocm services listprints opens the model server's own/docs(the stand-in answers it as vLLM would). Checked red with the URL changed.service-docs-09: a stopped server's refusal namesrocm services restart e2e-mock --yesandrocm services list --all; the scenario runs the list it names and finds the server stopped with that same restart command. Like04/05it holds the port it passes, so a command that served instead of refusing fails rather than hanging. Checked red with the old advice. Running the restart through needs an engine to start, so no lane does.service-docs-10/11(Linux, vLLM record):rocm serveof a model that is already running, piped and with--verbose, prints thedocsandengine docsrows, and the printed docs command opens the docs. No GPU needed, since the server is reused. Checked red with the rows removed from each path.serve-05(vLLM),serve-07(Lemonade) and nightlyserve-26(AIM) run thedocs:commandrocm serveprinted and try every GET route the docs list against the real engine.serve-07is an expected failure on Linux Lemonade lanes for an unrelated reason, so there the check is effective only on Windows Lemonade lanes.docs/manual-testing.md4e: the summary'sdocsrow (only shown on a terminal; the harness pipes output; its placement is unit-tested inserve_summary), and the rows of an attached fresh launch (rocm serve --verboseof a model not yet running) and its "detached - server still running" block, which need an engine to start. All print the rows of the samelaunch_docs_rows, which is unit-tested. The Ctrl+C handler is installed before anything is started; a Ctrl+C landing in that first instant ends the command before anything is served, and no test can aim at that window.Scope
No
--jsonfield for the docs command:rocm services list --jsonprints the records verbatim, as before.If this PR fixes a bug, searched
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. (Not a bug fix.)If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per
docs/architecture.md. (apps/rocm/src/services_docs.rs,apps/rocmd/src/api_docs.rs.)Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state, not only the wording, per AGENTS.md §3.