Repository navigation
refactor(rocmd): move artifact prefetch out of sandbox.rs into its own module - #1006
Conversation
Move the artifact-prefetch cluster into its own artifact_prefetch.rs: prefetch_artifact_value_with_policy and everything only it calls (declared_source_policy_block_message, huggingface_policy_uri_block_message, artifact_declares_huggingface_authenticated_policy, artifact_is_huggingface, is_huggingface_url, http_url_host, prefetch_blocked_value, artifact_bytes_path_for_marker) together with ARTIFACT_PREFETCH_TIMEOUT, so the new module needs nothing from sandbox.rs and the two do not form a cycle. sandbox.rs keeps the sandbox-tool dispatch and calls into the new module. The atomic writer comes from rocm_core::atomic_write, the same import sandbox.rs used. The tests move with the code: twelve sandbox_prefetch_* tests, huggingface_url_detection_is_host_scoped, and the serve_one_http_response and sha256_hex test helpers. Shared fixtures are still reached through crate::test_support. The leaf test names are unchanged, so the leaf-name test list of the rocmd crate is identical before and after. Every edit that is not a verbatim move: - prefetch_artifact_value_with_policy is widened from private to pub(crate), because sandbox.rs now calls it across the module boundary. - lib.rs declares the new private module. - Import lines: sandbox.rs drops the imports only the moved code used and imports the entry point; artifact_prefetch.rs adds its own. Path, which sandbox.rs's Linux-only bubblewrap test used through the old top-level import, is now imported inside that tests module. - sha256_hex was a module-level #[cfg(test)] function; it now sits in the new tests module, so its #[cfg(test)] attribute is gone with the move. - Two of the fourteen sandbox_prefetch_* tests stay in sandbox.rs: sandbox_prefetch_requires_artifact_ref and sandbox_prefetch_unknown_artifact_ref_errors drive run_sandbox_tool, which stays, and moving them would make the new module's tests import sandbox.rs back. - A module doc comment on the new file. - docs/architecture.md gains the entry for the new file and the sandbox.rs entry is corrected to say it now hands the artifact over. No behaviour change. Production lines of sandbox.rs: 826 before, 456 after (artifact_prefetch.rs: 386). Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the artifact-prefetch refactor.
No open issues 🎉
What changed in this PR
This refactor separates artifact prefetching from rocmd’s sandbox execution code, reducing sandbox.rs’s responsibilities while preserving existing behavior and reusing the shared atomic writer.
Changes:
- Moves download policy checks, streaming downloads, helpers, and related tests into
artifact_prefetch.rs. - Keeps sandbox dispatch and artifact-reference validation in
sandbox.rs, with a one-way call into the new module. - Registers the private module and documents the revised module boundaries.
| File | Description |
|---|---|
docs/architecture.md |
Documents the prefetch module and updates sandbox responsibilities. |
apps/rocmd/src/sandbox.rs |
Delegates prefetching and retains dispatch-level validation tests. |
apps/rocmd/src/lib.rs |
Declares the private artifact_prefetch module. |
apps/rocmd/src/artifact_prefetch.rs |
Houses the moved prefetch implementation, helpers, timeout, and tests. |
Review effort: Balanced
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
…andbox Signed-off-by: Jussi Elo <jussi.elo@amd.com> # Conflicts: # docs/architecture.md
juhovainio
left a comment
There was a problem hiding this comment.
The move itself looks right. Compared against the removed lines in sandbox.rs, the code in artifact_prefetch.rs is identical apart from the changes the PR description lists (imports, the module doc comment, prefetch_artifact_value_with_policy becoming pub(crate), and sha256_hex moving into the tests module). CI is green and the docs/architecture.md entries match the code. I found nothing that breaks behavior.
Two things I'd like addressed before merge:
- Test count vs. the ticket. 12 of the 14
sandbox_prefetch_*tests moved; the two that callrun_sandbox_toolstay insandbox.rsto avoid an import cycle. Please either move them too (for example behind a small shared test helper) or state the exception explicitly in the ticket and PR text, so the "all 14 moved" expectation doesn't silently go unmet. - Test-list evidence. The PR description says the
cargo test -p rocmd -- --listleaf names are unchanged before and after, but the comparison isn't shown. Please paste the diff of the two lists so reviewers can confirm no test was dropped or renamed.
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head bf9e77c7
Review — no blocking findings
Full review of the whole change (4 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, docs/architecture.md).
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (this is a public review and ROCMAI-822 is an internal tracker reference); judged only against what the PR description itself states, which this review found to be accurate.
Blocking
None
Non-blocking
- No single test exercises the full dispatch seam for
prefetch_artifact—apps/rocmd/src/sandbox.rs:221-235,apps/rocmd/src/sandbox.rs:913-947,apps/rocmd/src/artifact_prefetch.rs:875
run_sandbox_tool'sPrefetchArtifactarm resolves the artifact viaresolve_model_recipe_artifactand then callsprefetch_artifact_value_with_policy. The two tests kept insandbox.rs(sandbox_prefetch_requires_artifact_ref,sandbox_prefetch_unknown_artifact_ref_errors) only exercise the validation before that call (missing/unknown artifact ref); every test that actually drivesprefetch_artifact_value_with_policy(e.g.sandbox_prefetch_downloads_direct_artifact_when_approved) calls it directly inartifact_prefetch.rs, bypassingrun_sandbox_tooland the recipe-registry resolution step. This gap predates the move (same split of behavior already existed in one file), so it isn't introduced by this PR, but splitting the two halves across files makes the gap harder to notice from either file alone.
Confidence 75 · logic · Fix: a small integration test (can live in either file) that drives a successful or policy-blocked prefetch throughrun_sandbox_toolend to end.
Decisions for the author
- Two of the fourteen
sandbox_prefetch_*tests stay insandbox.rs— tradeoff
Movingsandbox_prefetch_requires_artifact_ref/sandbox_prefetch_unknown_artifact_ref_errorsintoartifact_prefetch.rswould require them to callrun_sandbox_tool, which stays insandbox.rs, creating the import cycle the PR is designed to avoid. Keeping them put is the correct call given that constraint, and the PR text already states the exception explicitly. Confirm this is still the intended resolution to the open reviewer question about matching the ticket's count of 14, since the two options raised there (move them behind a shared test helper, or state the exception) — this PR already does the latter. - Policy-decision logic and network-IO/download-test-harness code share one file (
artifact_prefetch.rs, ~387 production lines) — non-blocking-improvement
A further split (pure policy gating vs. streaming download + its one-shot test HTTP server) could let the policy rules be unit-tested without any IO harness in scope, but the current file is well under the 1,500-line module-size cap and the current boundary is coherent (everything onlyprefetch_artifact_value_with_policycalls moved together); not necessary now.
Positive signals
- The
docs/architecture.mdentries for both files accurately describe the real import graph (artifact_prefetch.rshas nouse crate::sandbox— confirmed no back-edge — andsandbox.rs's entry correctly states it is the only caller), rather than the generic restatement these entries sometimes degrade into. prefetch_artifact_value_with_policy's widened visibility ispub(crate), which is the minimal visibility this move requires (both modules are declared as direct siblings inlib.rs), not broader than needed.- Every function and test in the claimed move list was verified byte-for-byte identical between the old
sandbox.rsand the newartifact_prefetch.rsapart from the explicitly enumerated exceptions (visibility, imports, module doc comment,sha256_hexlosing its test-only attribute since it now sits directly insidemod tests).
Deployment notes
None — pure internal module split, no behavior, CLI surface, or output change.
What this covered
Read both files completely in both their before and after states (apps/rocmd/src/sandbox.rs 2013→1034 lines, new apps/rocmd/src/artifact_prefetch.rs 1001 lines), plus the full lib.rs and docs/architecture.md diffs, against a14f82b41e75cc6d88d71b77193c59496bb81493...bf9e77c7427c8656339b705dc375d2d04ddb4c98. Two independent fan-out workers: one on the coupled sandbox.rs+artifact_prefetch.rs group verifying the move is behavior-preserving (4a), one on the combined agent-instruction/history/prior-PR/code-comment passes (4b) against this repo's AGENTS.md. Design review (Step 7) was investigated directly rather than via separate spawned workers, given how narrowly answerable the questions were for a change this size and shape. Also independently re-verified: module-line counts against the 1,500-line cap, the pub(crate) visibility is minimal (both modules are crate-root siblings), no import cycle, a leak scan of the diff (clean — only example.invalid/.evil.example test fixture URLs and the real huggingface.co domain used for policy tests), and that the PR's own merge commit (which recorded a real conflict in docs/architecture.md against a concurrently-landed PR) resolved to exactly the two intended entries with no stray conflict markers or stale text.
CI: all checks green (gh pr view --json statusCheckRollup), including clippy, the affected-crate test jobs, E2E tests, CodeQL, and the security-scan suite; Skill checks (skillscope) and Sphinx docs build are SKIPPED but unrelated (this change touches neither skills/ nor docs/rocm-docs/).
Did not run locally: cargo test -p rocmd, the e2e scenario, or the full workspace suite — relied on CI's green result for these, which the project treats as authoritative. Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference).
Not reconciled against this PR's own review thread. A prior human reviewer left two open questions (why 12 of 14 tests moved, and asking for the before/after cargo test -- --list diff to be pasted) that this blind pass happens to independently speak to: direct comparison of the old and new file contents confirms the moved test set is unchanged and that the two-test exception is correctly reasoned (avoids an import cycle) — but this review did not read that thread and this is not a response to it.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
…andbox Conflict in docs/architecture.md: main reworded the anthropic_proxy.rs entry in the apps/rocmd list while this branch added the artifact_prefetch.rs entry directly below it. Both are kept: main's anthropic_proxy.rs entry as it now reads, followed by the new artifact_prefetch.rs entry. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head ef92fc0d
Review — no blocking findings
Full review of the whole change (4 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, docs/architecture.md).
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (this is a public review and ROCMAI-822 is an internal tracker reference); judged only against what the PR description itself states, which this review found to be accurate.
Blocking
None
Non-blocking
- No single test exercises the full dispatch seam for
prefetch_artifact—apps/rocmd/src/sandbox.rs:221-235,apps/rocmd/src/sandbox.rs:913-947,apps/rocmd/src/artifact_prefetch.rs:875
run_sandbox_tool'sPrefetchArtifactarm resolves the artifact viaresolve_model_recipe_artifactand then callsprefetch_artifact_value_with_policy. The two tests kept insandbox.rs(sandbox_prefetch_requires_artifact_ref,sandbox_prefetch_unknown_artifact_ref_errors) only exercise the validation before that call (missing/unknown artifact ref); every test that actually drivesprefetch_artifact_value_with_policy(e.g.sandbox_prefetch_downloads_direct_artifact_when_approved) calls it directly inartifact_prefetch.rs, bypassingrun_sandbox_tooland the recipe-registry resolution step. This gap predates the move (the same split of behavior already existed within one file), so it isn't introduced by this PR, but splitting the two halves across files makes the gap harder to notice from either file alone.
Confidence 75 · logic · Fix: a small integration test (can live in either file) that drives a successful or policy-blocked prefetch throughrun_sandbox_toolend to end.
Decisions for the author
- Two of the fourteen
sandbox_prefetch_*tests stay insandbox.rs— tradeoff
Movingsandbox_prefetch_requires_artifact_ref/sandbox_prefetch_unknown_artifact_ref_errorsintoartifact_prefetch.rswould require them to callrun_sandbox_tool, which stays insandbox.rs, creating the import cycle the PR is designed to avoid. Keeping them put is the correct call given that constraint, and the PR text states the exception explicitly. - Policy-decision logic and network-IO/download-test-harness code share one file (
artifact_prefetch.rs, ~386 production lines) — non-blocking-improvement
A further split (pure policy gating vs. streaming download + its one-shot test HTTP server) could let the policy rules be unit-tested without any IO harness in scope, but the current file is well under the 1,500-line module-size cap and the current boundary is coherent (everything onlyprefetch_artifact_value_with_policycalls moved together); not necessary now.
Positive signals
- The production code move is verified byte-for-byte: diffing the old
sandbox.rs's prefetch-cluster block against the newartifact_prefetch.rsblock shows exactly one textual difference,fn→pub(crate) fnonprefetch_artifact_value_with_policy(plus one blank-line delta) — nothing else changed in transit. docs/architecture.md's new entries accurately describe the real import graph:artifact_prefetch.rshas nouse crate::sandboxanywhere in the file (confirmed, no back-edge), and a whole-tree search findssandbox.rsis the module's only caller.- The widened visibility (
pub(crate)) is the minimum this move requires, since both modules are declared as direct siblings inlib.rs.
Deployment notes
None — pure internal module split, no behavior, CLI surface, or output change.
What this covered
Read apps/rocmd/src/artifact_prefetch.rs (1001 lines) and apps/rocmd/src/lib.rs (27 lines) completely at the reviewed head, and apps/rocmd/src/sandbox.rs (1034 lines) completely at the reviewed head plus its pre-move state at the PR's merge-base, to directly diff the moved block. Read the PR's own net diff against its base (f7c5fef7a...ef92fc0d4, matching GitHub's reported 4 files / +1009 / -985) in full, including both hunks of the docs/architecture.md change. Searched the whole tree at the reviewed commit for any remaining reference to the moved private helpers or any caller of artifact_prefetch/prefetch_artifact_value_with_policy other than sandbox.rs; found none.
This PR's head advanced (bf9e77c7 → ef92fc0d) since an earlier automated review was posted here, by a routine merge of upstream main. Established directly from git, not taken as a claim: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs and apps/rocmd/src/lib.rs are byte-identical between the two commits, and the docs/architecture.md content this PR itself owns (its diff against its own base) is unchanged; the merge's only effect was pulling in unrelated upstream files and unrelated docs/architecture.md entries for other modules that also exist on current main. This review re-verified the change directly rather than reusing that review's conclusions.
CI at time of review: everything that has reported a conclusion is green, including clippy, Test (affected crates) (both runners), build-and-test (both runners), CodeQL, and the full security-scan suite. Three jobs had not yet concluded (E2E tests, E2E tests (ubuntu-26.04), Windows build and test (debug)) — none had failed, so this is reported as incomplete information rather than a finding. Skill checks (skillscope) and Sphinx docs build are skipped but unrelated (this change touches neither skills/ nor docs/rocm-docs/).
Did not run locally: cargo test -p rocmd, the e2e scenario, or the full workspace suite — relied on CI's result for these. Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference). Design review (Step 7) was reasoned through directly rather than via separate spawned workers, given how narrowly answerable the questions were for a change this size and shape; a prior round of this same review already fanned the code passes out across independent workers over this identical content.
Not reconciled against this PR's own review thread. A prior human reviewer's two open questions (why 12 of 14 tests moved, and asking for the before/after cargo test -- --list diff) are already answered by the PR description's own text, which this review judged independently against the code and found accurate — but this review did not read that thread and this is not a response to it.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
juhovainio
left a comment
There was a problem hiding this comment.
Approving. I compared the removed and added lines mechanically: the production code moved into artifact_prefetch.rs is verbatim. The only other changes are imports, the module doc comment, pub(crate) on the entry function, and the docs/architecture.md entries. artifact_prefetch.rs does not import sandbox.rs, so there is no module cycle. The architecture entry is added and the sandbox.rs entry corrected, and file sizes are well under the module limit.
One note, not blocking: 2 of the 14 sandbox_prefetch_* tests in the ticket stay in sandbox.rs because they drive run_sandbox_tool, and moving them would recreate the cycle. That reasoning holds, but it is only a partial match to the ticket's step.
CI: the substantive checks pass. The Strix Halo WSL2 lane fails at the distro install step, which looks like infrastructure. The Strix Halo Windows E2E lane fails at "Run E2E tests on Strix Halo Windows" and I could not tell why, so a rerun or log check would be worthwhile before merge. I did not build or run tests locally.
…rtifact-prefetch-out-of-sandbox Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…_tool The prefetch tests that moved to artifact_prefetch.rs call the entry point directly, and the two that stay in sandbox.rs stop at the argument checks, so nothing drove a reference that resolves through the recipe registry into the policy decision. Add one test that signs a recipe index in the test, resolves a reference through run_sandbox_tool, and checks that the recipe's model id is used, that the default policy fetches and caches nothing, and that an approved policy reaches the cached-marker answer. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
… env The dispatch test sets the model recipe index, signature and public key environment variables and held a mutex only of its own, while sandbox_prefetch_unknown_artifact_ref_errors resolves a recipe reference through the same registry and read them without it. Run in parallel, that reader could see the index path set before its signature and key, or the variables mid-restore, and fail with a configuration error instead of "was not found". A developer's own exported variables could do the same. Move the guard into test_support as RecipeIndexEnv, with builtin_catalog() for readers and signed_index() for the test that supplies an index. Both take the one lock, the guard sets or clears all four recipe-registry variables under it, and drop restores them. The unknown-reference test now takes the built-in-catalog guard, which also makes it independent of the shell it runs in. No other rocmd test reaches the registry: the cache-warm tests inject their resolver. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head e72d6caa
Review — no blocking findings
Full review of the whole change (5 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, apps/rocmd/src/test_support.rs, docs/architecture.md), diff eb2a8194a...e72d6caa1.
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (public review, internal tracker reference); judged only against what the PR description itself states, which this review found accurate against the diff.
Blocking
None
Non-blocking
- The new env-mutation test guard isn't registered with this repo's own regression check for exactly this bug class —
apps/rocmd/src/test_support.rs:324-391(RecipeIndexEnv, used atapps/rocmd/src/sandbox.rs:938andapps/rocmd/src/sandbox.rs:1011),xtask/src/env_mutation_contract.rs.
This repo has a dedicated static-analysis test,a_test_mutating_the_environment_serializes_itself, that scans every#[test]function body for a bareset_var(/remove_var(/RestoredEnvVar::set(call not preceded by a lock — either a*_TEST_LOCK-suffixed static's.lock(, or a type listed inNAMED_SERIALIZERS(today justScopedTestEnv, documented there as "a type whose whole job is to perform the mutation on a test's behalf").RecipeIndexEnvis structurally the same kind of type asScopedTestEnv— one RAII value that takes the lock and performs the mutation inside its own constructor — but its backing static is namedRECIPE_INDEX_ENV(no_TEST_LOCKsuffix), and neither the type nor its constructors were added toNAMED_SERIALIZERS/MUTATIONS. Because the actualset_var/remove_varcalls live insideRecipeIndexEnv::apply()/Drop::drop()— plain functions, not#[test]bodies — the scanner's text match never fires for either of the two call sites insandbox.rs, so they are not merely passing the guard, they are invisible to it. At runtime the serialization is correct (I traced it directly: both constructors touch all four vars every time,Drop's explicit restore runs before theMutexGuardfield auto-drops releases the lock, and poison recovery viaPoisonError::into_inneris used on both the lock and the restore path), so this is not a live bug — but it means the one guard this repo built specifically for this bug class (its own doc comment citesEAI-8397) provides no safety net here going forward, unlike for the existing pattern.
Confidence 78 · architectural · Fix: addRecipeIndexEnv(or its constructors) toxtask/src/env_mutation_contract.rs'sNAMED_SERIALIZERS, mirroring howScopedTestEnvis already covered.
Decisions for the author
- Two of the fourteen
sandbox_prefetch_*tests stay insandbox.rs— tradeoff
sandbox_prefetch_requires_artifact_refandsandbox_prefetch_unknown_artifact_ref_errorsassert thePrefetchArtifactdispatch arm's argument checks (missing/unknown--artifact-ref), which run insandbox.rsbeforeresolve_model_recipe_artifact/prefetch_artifact_value_with_policyare ever reached. Moving them intoartifact_prefetch.rswould require that module to call back intosandbox.rs'srun_sandbox_tool, recreating the cycle this PR removes. Keeping them beside the dispatch they exercise is the reasoned choice given that constraint; the PR text states the exception explicitly. - The recipe-index configuration is read from process-wide environment variables rather than an injectable seam — non-blocking-improvement
xtask/src/env_mutation_contract.rs's own doc comment states the preferred long-term fix for this whole class of test is to avoid environment mutation entirely ("pass the value in through a test seam", citingnewest_rocm_install_dir_inas the pattern), which would make a lock unnecessary rather than just correctly held. That would mean changingcrates/rocm-core/src/model_catalog.rs'sload_model_recipe_registryto accept an injected override, which is production-code surface this PR (a behavior-preserving move) correctly leaves untouched; flagging it for whoever next touches that resolver.
Positive signals
- The new dispatch test genuinely exercises the
PrefetchArtifactseam end-to-end rather than calling the policy function directly: its first assertion only holds ifrun_sandbox_toolactually resolved the artifact reference through the recipe registry (asserts the resolved recipe's canonical model id), and its second call only lands on"cached"if the caller'sSandboxToolPolicy— not the default — reached the policy decision. RecipeIndexEnv'sDropis ordered correctly: the explicit restore runs to completion before the struct's ownMutexGuardfield is auto-dropped (Rust drops fields in declaration order after the explicitDrop::dropbody), so the lock is never released mid-restore.- The commit history shows the fix converging within the PR itself rather than needing a follow-up round: the dispatch-test commit introduced a lock scoped only to its own new test, and the very next commit recognized the sibling test it left unguarded and generalized the guard to cover both.
Deployment notes
None — pure internal module split plus a test-only change; no behavior, CLI surface, or output change.
What this covered
Read all 5 changed files completely at the reviewed head, diffed the moved production block directly against its pre-move location at the merge-base, and read xtask/src/env_mutation_contract.rs in full to verify the one finding above by tracing its scanner logic rather than asserting it. Two independent fan-out workers: one over the coupled sandbox.rs/artifact_prefetch.rs/lib.rs/test_support.rs group verifying the move and the new test-isolation fix are both correct; one running the agent-instruction/history/prior-changes/code-comment passes against this repo's AGENTS.md and its existing env-var-guard precedent (RestoredEnvVar/ScopedTestEnv). Design review (Step 7) was reasoned through directly given how narrowly answerable the questions were for a change this shape and size.
No Rust toolchain available in this sandbox: did not run cargo test, cargo xtask check-module-size, or the env_mutation_contract test itself. The finding above and the verified positive signals are from reading the relevant source directly (the scanner's own logic, the Drop/lock code) rather than executing it.
CI at time of review: everything that had concluded was green — clippy, both build-and-test runners, Test (affected crates) (ubuntu-26.04), Coverage (rocm-dash crates, ratcheted), Windows release build and lifecycle, the full security-scan suite, CodeQL, lint/docs/signature checks, and copilot-pull-request-reviewer. Still in progress and not yet failed: both E2E tests runners, Build Linux PR binaries, Test (affected crates) (default runner), Windows build and test (debug) (the required job that runs the workspace-wide cargo test --all-targets, which is what actually exercises the env_mutation_contract guard discussed above), and Security scan / codeql / rust. Reported as incomplete information, not a finding.
Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference). Not reconciled against this PR's own review thread or any prior round — this was read blind, with no earlier review content consulted.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
…rtifact-prefetch-out-of-sandbox Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Closes ROCMAI-822.
This moves the artifact-prefetch cluster out of apps/rocmd/src/sandbox.rs into a new apps/rocmd/src/artifact_prefetch.rs. It is a behaviour-preserving move plus one new test that drives the moved tool through its dispatch; the edits that are not a verbatim move are listed below.
What moved: prefetch_artifact_value_with_policy (the
prefetch_artifacttool's policy gating and streamed download) and everything only it calls: declared_source_policy_block_message, huggingface_policy_uri_block_message, artifact_declares_huggingface_authenticated_policy, artifact_is_huggingface, is_huggingface_url, http_url_host, prefetch_blocked_value and artifact_bytes_path_for_marker. ARTIFACT_PREFETCH_TIMEOUT moves with it, which is what keeps the two modules from forming a cycle: artifact_prefetch.rs uses nothing from sandbox.rs, and sandbox.rs only calls the one entry point. The atomic writer is taken from rocm_core::atomic_write, the same import sandbox.rs already used, so there is no sandbox <-> artifact_prefetch cycle and rocmd gains no new dependency (check-crate-edges passes).Tests moved with the code: twelve of the fourteen sandbox_prefetch_* tests, huggingface_url_detection_is_host_scoped, and the serve_one_http_response and sha256_hex test helpers. Shared fixtures are still reached through crate::test_support.
Exception to the ticket's "all fourteen sandbox_prefetch_* tests move": two stay in sandbox.rs, sandbox_prefetch_requires_artifact_ref and sandbox_prefetch_unknown_artifact_ref_errors. They are not tests of the prefetch code. They call run_sandbox_tool and assert the two argument checks in the
PrefetchArtifactarm of that dispatch (a missing--artifact-refand an artifact ref the recipe registry does not know), which run before prefetch_artifact_value_with_policy is reached. The dispatch stays in sandbox.rs, so a move would make artifact_prefetch.rs import sandbox.rs back, the cycle this PR removes, and no shared test helper changes that: the dependency is on run_sandbox_tool itself, not on a fixture. They are the only tests on main that drive run_sandbox_tool forPrefetchArtifact, so keeping them where the dispatch is leaves each test next to the code it covers. A third test now sits beside them for the same reason (see the last commit below).Every edit that is not a verbatim move:
source_policy_requiredwith no network use and no cache marker written, and that, with a marker already present and download approved, the caller's policy is the one applied (cached). It needs no network. Mutating the dispatch turns it red each time: passing the default policy instead of the caller's (thecachedassertion fails), passing the artifact reference as the model (the model-id assertion fails), and not routing the tool (the call returns an error). The production code is untouched by that commit.The other steps of the ticket are not in this PR because they are already done. The shared atomic writer landed in rocm-core with #610, so sandbox.rs no longer owns one. mcp_argv.rs and the watchers directory already exist on main, and the remaining step was delivered earlier under its own ticket.
There is no e2e scenario because this is a purely internal refactor: no command output, exit code, file path or runtime selection changes. The existing scenario artifact-prefetch-01 (@id:artifact-prefetch-failed-marker-leaves-no-temp) drives the prefetch driver end to end and passes after the move (
cargo xtask e2e -- -n "artifact-prefetch-01": 1 scenario, 6 steps passed; run before the latest merge of main, which does not touch the prefetch code).Test list before and after. Before is origin/main at f7c5fef, after is this branch at the merge of that commit, both taken with the same target directory. The leaf names are
cargo test -p rocmd -- --listlines ending: test, with the module path stripped and sorted. The leaf-name diff is empty (261 entries on each side, and the--list --ignoredlist is identical too, 1 entry):The same lists with the module path kept differ only by the thirteen moved tests changing module, from
sandbox::tests::toartifact_prefetch::tests::(nothing is added, removed or renamed apart from that prefix):The two tests that stay, sandbox::tests::sandbox_prefetch_requires_artifact_ref and sandbox::tests::sandbox_prefetch_unknown_artifact_ref_errors, appear in both lists unchanged.
The list comparison above was taken at the first merge and is unchanged by the move; the last commit adds exactly one test to sandbox::tests and nothing else.
Verification, run on this branch after the second merge of origin/main and the new test:
cargo test -p rocmd --no-fail-fastpasses, three runs in a row (279 passed, 1 ignored each; main itself gained tests since the earlier count of 260). The earlier results below were taken before that merge and the new test, except where noted:cargo clippy -p rocmd --all-targets -- -D warnings,cargo fmt --all -- --check,cargo xtask check-architecture-docandcargo xtask check-module-sizewere re-run after the new test and pass.cargo fmt --checkandcargo clippy --locked -p rocmd -p rocm-core --all-targets -- -D warningsare clean on Linux, andcargo clippy --locked --workspace --all-targets --target x86_64-pc-windows-msvc -- -D warningsis clean for the whole workspace, using stub C compiler and archiver scripts for the aws-lc-sys and ring build scripts since the host has no Windows C compiler; that is a type-check and lint, nothing was linked or run on Windows.cargo xtask manifest --check,check-crate-edges,check-architecture-docandcheck-module-sizepass,cargo test -q --locked -p xtask prepushpasses (98 tests), and the commit hooks, including lychee on the touched docs, passed. A conflict-marker grep over apps, crates, docs, xtask, engines, tests and scripts finds only RST heading underlines in docs/rocm-docs/index.rst that were already there. The leak scan of the added lines found only a moved test fixture URL on an example domain.Latest merge with main (after the PR left the merge queue as conflicting): main moved again and docs/architecture.md conflicted in one hunk, where main reworded the common.rs entry directly below the new artifact_prefetch.rs entry. The merge keeps main's common.rs entry unchanged and this PR's artifact_prefetch.rs entry above it; the diff against main is still the same five files with the same line counts as before the merge (artifact_prefetch.rs +1001, lib.rs +1, sandbox.rs +105 -976, test_support.rs +83 -1, architecture.md +2 -1). After it,
cargo test -p rocmd --no-fail-fast(279 passed, 1 ignored),cargo clippy -p rocmd --all-targets -- -D warnings,cargo fmt --all -- --check,cargo xtask check-architecture-docandcargo xtask check-module-sizepass on the merged tree.Second merge with main: the branch predated the change that runs the cucumber suite on a large-stack thread, and its Strix Halo Windows E2E lane stopped with a stack overflow in the e2e harness. Merging origin/main brings that change in; the merge had no conflicts, and the diff against main is still the same four files with the same line counts as before the test commit (+1009 -985).
First merge with main: origin/main moved while this was in review and the branch conflicted in docs/architecture.md, where main reworded the anthropic_proxy.rs entry directly above the new artifact_prefetch.rs entry. The merge keeps both: main's anthropic_proxy.rs entry as it now reads, followed by the new artifact_prefetch.rs entry. Nothing in apps/rocmd other than anthropic_proxy.rs changed on main, so the diff against main is still the same four files.
Production lines (cargo xtask check-module-size counting rules): sandbox.rs was 826 before and is 456 after, and artifact_prefetch.rs is 386, so no file grows past the 1,500 limit and nothing exceeds it. Total lines of sandbox.rs went from 2013 to 1034.
Overlap with #541, which has now merged: it landed on main without changing apps/rocmd/src/sandbox.rs (the only apps/rocmd files that differ between the old merge base and main are anthropic_proxy.rs and Cargo.toml), so the sandbox.rs overlap that was flagged earlier no longer exists. This PR does not touch it.
What was not run: the full workspace test suite,
python scripts/smoke_local.pyandcargo xtask coveragewere not run locally and are left to CI; only the e2e scenario above was run, not the whole e2e suite; nothing was executed on Windows (type-check and lint only, as described); no live network download was made beyond what the existing tests do against a local one-shot HTTP server.