Skip to content

refactor(rocmd): move artifact prefetch out of sandbox.rs into its own module - #1006

Merged
jussielo-amd merged 7 commits into
mainfrom
refactor-rocmd-move-artifact-prefetch-out-of-sandbox
Nov 11, 2026
Merged

jussielo-amd merged 7 commits into
mainfrom
refactor-rocmd-move-artifact-prefetch-out-of-sandbox

Conversation

@jussielo-amd

@jussielo-amd jussielo-amd commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Closes ROCMAI-822.

This moves the artifact-prefetch cluster out of apps/rocmd/src/sandbox.rs into a new apps/rocmd/src/artifact_prefetch.rs. It is a behaviour-preserving move plus one new test that drives the moved tool through its dispatch; the edits that are not a verbatim move are listed below.

What moved: prefetch_artifact_value_with_policy (the prefetch_artifact tool's policy gating and streamed download) and everything only it calls: declared_source_policy_block_message, huggingface_policy_uri_block_message, artifact_declares_huggingface_authenticated_policy, artifact_is_huggingface, is_huggingface_url, http_url_host, prefetch_blocked_value and artifact_bytes_path_for_marker. ARTIFACT_PREFETCH_TIMEOUT moves with it, which is what keeps the two modules from forming a cycle: artifact_prefetch.rs uses nothing from sandbox.rs, and sandbox.rs only calls the one entry point. The atomic writer is taken from rocm_core::atomic_write, the same import sandbox.rs already used, so there is no sandbox <-> artifact_prefetch cycle and rocmd gains no new dependency (check-crate-edges passes).

Tests moved with the code: twelve of the fourteen sandbox_prefetch_* tests, huggingface_url_detection_is_host_scoped, and the serve_one_http_response and sha256_hex test helpers. Shared fixtures are still reached through crate::test_support.

Exception to the ticket's "all fourteen sandbox_prefetch_* tests move": two stay in sandbox.rs, sandbox_prefetch_requires_artifact_ref and sandbox_prefetch_unknown_artifact_ref_errors. They are not tests of the prefetch code. They call run_sandbox_tool and assert the two argument checks in the PrefetchArtifact arm of that dispatch (a missing --artifact-ref and an artifact ref the recipe registry does not know), which run before prefetch_artifact_value_with_policy is reached. The dispatch stays in sandbox.rs, so a move would make artifact_prefetch.rs import sandbox.rs back, the cycle this PR removes, and no shared test helper changes that: the dependency is on run_sandbox_tool itself, not on a fixture. They are the only tests on main that drive run_sandbox_tool for PrefetchArtifact, so keeping them where the dispatch is leaves each test next to the code it covers. A third test now sits beside them for the same reason (see the last commit below).
Every edit that is not a verbatim move:

  • prefetch_artifact_value_with_policy is widened from private to pub(crate) because sandbox.rs now calls it across the module boundary.
  • lib.rs declares the new private module.
  • Import lines changed: sandbox.rs drops the imports only the moved code used and imports the entry point, the new file adds its own, and Path, which a Linux-only bubblewrap test in sandbox.rs used through the old top-level import, is now imported inside that tests module.
  • sha256_hex was a module-level cfg(test) function and now sits in the new tests module, so its attribute is gone with the move.
  • The new file has a module doc comment.
  • One new test, sandbox_prefetch_resolves_artifact_and_applies_policy_through_run_sandbox_tool, in the last commit. A review pointed out that no test drove a reference that resolves through the recipe registry into the policy decision: the moved tests call the entry point directly and the two that stay stop at the argument checks. The built-in catalog lists no artifacts, so the test signs a one-recipe index with rocm_core's own key generation and signing, points the recipe-index environment variables at it through a guard in test_support (RecipeIndexEnv) that serialises every test that reads or sets them and restores them on drop, and calls run_sandbox_tool. The unknown-reference test above takes the same guard with the built-in catalog, so a parallel run cannot see the variables half set; that came from a review finding and is its own commit. It asserts that the recipe's canonical model id is what the decision receives, that the default policy answers source_policy_required with no network use and no cache marker written, and that, with a marker already present and download approved, the caller's policy is the one applied (cached). It needs no network. Mutating the dispatch turns it red each time: passing the default policy instead of the caller's (the cached assertion fails), passing the artifact reference as the model (the model-id assertion fails), and not routing the tool (the call returns an error). The production code is untouched by that commit.
  • docs/architecture.md gains the entry for the new file and the sandbox.rs entry is corrected (and names the new test).

The other steps of the ticket are not in this PR because they are already done. The shared atomic writer landed in rocm-core with #610, so sandbox.rs no longer owns one. mcp_argv.rs and the watchers directory already exist on main, and the remaining step was delivered earlier under its own ticket.

There is no e2e scenario because this is a purely internal refactor: no command output, exit code, file path or runtime selection changes. The existing scenario artifact-prefetch-01 (@id:artifact-prefetch-failed-marker-leaves-no-temp) drives the prefetch driver end to end and passes after the move (cargo xtask e2e -- -n "artifact-prefetch-01": 1 scenario, 6 steps passed; run before the latest merge of main, which does not touch the prefetch code).

Test list before and after. Before is origin/main at f7c5fef, after is this branch at the merge of that commit, both taken with the same target directory. The leaf names are cargo test -p rocmd -- --list lines ending : test, with the module path stripped and sorted. The leaf-name diff is empty (261 entries on each side, and the --list --ignored list is identical too, 1 entry):

$ diff leaf_before.txt leaf_after.txt
$ diff leaf_before_ignored.txt leaf_after_ignored.txt

The same lists with the module path kept differ only by the thirteen moved tests changing module, from sandbox::tests:: to artifact_prefetch::tests:: (nothing is added, removed or renamed apart from that prefix):

$ diff <(sort path_before.txt) <(sort path_after.txt)
42a43,55
> artifact_prefetch::tests::huggingface_url_detection_is_host_scoped
> artifact_prefetch::tests::sandbox_prefetch_blocks_approved_artifact_without_sha256
> artifact_prefetch::tests::sandbox_prefetch_blocks_artifact_larger_than_approved_limit
> artifact_prefetch::tests::sandbox_prefetch_blocks_gated_huggingface_without_source_policy
> artifact_prefetch::tests::sandbox_prefetch_blocks_huggingface_source_policy_without_token
> artifact_prefetch::tests::sandbox_prefetch_blocks_non_direct_non_huggingface_source
> artifact_prefetch::tests::sandbox_prefetch_cached_marker_skips_network
> artifact_prefetch::tests::sandbox_prefetch_downloads_direct_artifact_when_approved
> artifact_prefetch::tests::sandbox_prefetch_never_sends_huggingface_token_over_plain_http
> artifact_prefetch::tests::sandbox_prefetch_never_sends_huggingface_token_to_non_huggingface_url
> artifact_prefetch::tests::sandbox_prefetch_reports_policy_required_without_network
> artifact_prefetch::tests::sandbox_prefetch_respects_declared_huggingface_auth_policy
> artifact_prefetch::tests::sandbox_prefetch_respects_manual_only_source_policy
112d124
< sandbox::tests::huggingface_url_detection_is_host_scoped
117,126d128
< sandbox::tests::sandbox_prefetch_blocks_approved_artifact_without_sha256
< sandbox::tests::sandbox_prefetch_blocks_artifact_larger_than_approved_limit
< sandbox::tests::sandbox_prefetch_blocks_gated_huggingface_without_source_policy
< sandbox::tests::sandbox_prefetch_blocks_huggingface_source_policy_without_token
< sandbox::tests::sandbox_prefetch_blocks_non_direct_non_huggingface_source
< sandbox::tests::sandbox_prefetch_cached_marker_skips_network
< sandbox::tests::sandbox_prefetch_downloads_direct_artifact_when_approved
< sandbox::tests::sandbox_prefetch_never_sends_huggingface_token_over_plain_http
< sandbox::tests::sandbox_prefetch_never_sends_huggingface_token_to_non_huggingface_url
< sandbox::tests::sandbox_prefetch_reports_policy_required_without_network
128,129d129
< sandbox::tests::sandbox_prefetch_respects_declared_huggingface_auth_policy
< sandbox::tests::sandbox_prefetch_respects_manual_only_source_policy

The two tests that stay, sandbox::tests::sandbox_prefetch_requires_artifact_ref and sandbox::tests::sandbox_prefetch_unknown_artifact_ref_errors, appear in both lists unchanged.

The list comparison above was taken at the first merge and is unchanged by the move; the last commit adds exactly one test to sandbox::tests and nothing else.

Verification, run on this branch after the second merge of origin/main and the new test: cargo test -p rocmd --no-fail-fast passes, three runs in a row (279 passed, 1 ignored each; main itself gained tests since the earlier count of 260). The earlier results below were taken before that merge and the new test, except where noted: cargo clippy -p rocmd --all-targets -- -D warnings, cargo fmt --all -- --check, cargo xtask check-architecture-doc and cargo xtask check-module-size were re-run after the new test and pass. cargo fmt --check and cargo clippy --locked -p rocmd -p rocm-core --all-targets -- -D warnings are clean on Linux, and cargo clippy --locked --workspace --all-targets --target x86_64-pc-windows-msvc -- -D warnings is clean for the whole workspace, using stub C compiler and archiver scripts for the aws-lc-sys and ring build scripts since the host has no Windows C compiler; that is a type-check and lint, nothing was linked or run on Windows. cargo xtask manifest --check, check-crate-edges, check-architecture-doc and check-module-size pass, cargo test -q --locked -p xtask prepush passes (98 tests), and the commit hooks, including lychee on the touched docs, passed. A conflict-marker grep over apps, crates, docs, xtask, engines, tests and scripts finds only RST heading underlines in docs/rocm-docs/index.rst that were already there. The leak scan of the added lines found only a moved test fixture URL on an example domain.

Latest merge with main (after the PR left the merge queue as conflicting): main moved again and docs/architecture.md conflicted in one hunk, where main reworded the common.rs entry directly below the new artifact_prefetch.rs entry. The merge keeps main's common.rs entry unchanged and this PR's artifact_prefetch.rs entry above it; the diff against main is still the same five files with the same line counts as before the merge (artifact_prefetch.rs +1001, lib.rs +1, sandbox.rs +105 -976, test_support.rs +83 -1, architecture.md +2 -1). After it, cargo test -p rocmd --no-fail-fast (279 passed, 1 ignored), cargo clippy -p rocmd --all-targets -- -D warnings, cargo fmt --all -- --check, cargo xtask check-architecture-doc and cargo xtask check-module-size pass on the merged tree.

Second merge with main: the branch predated the change that runs the cucumber suite on a large-stack thread, and its Strix Halo Windows E2E lane stopped with a stack overflow in the e2e harness. Merging origin/main brings that change in; the merge had no conflicts, and the diff against main is still the same four files with the same line counts as before the test commit (+1009 -985).

First merge with main: origin/main moved while this was in review and the branch conflicted in docs/architecture.md, where main reworded the anthropic_proxy.rs entry directly above the new artifact_prefetch.rs entry. The merge keeps both: main's anthropic_proxy.rs entry as it now reads, followed by the new artifact_prefetch.rs entry. Nothing in apps/rocmd other than anthropic_proxy.rs changed on main, so the diff against main is still the same four files.

Production lines (cargo xtask check-module-size counting rules): sandbox.rs was 826 before and is 456 after, and artifact_prefetch.rs is 386, so no file grows past the 1,500 limit and nothing exceeds it. Total lines of sandbox.rs went from 2013 to 1034.

Overlap with #541, which has now merged: it landed on main without changing apps/rocmd/src/sandbox.rs (the only apps/rocmd files that differ between the old merge base and main are anthropic_proxy.rs and Cargo.toml), so the sandbox.rs overlap that was flagged earlier no longer exists. This PR does not touch it.

What was not run: the full workspace test suite, python scripts/smoke_local.py and cargo xtask coverage were not run locally and are left to CI; only the e2e scenario above was run, not the whole e2e suite; nothing was executed on Windows (type-check and lint only, as described); no live network download was made beyond what the existing tests do against a local one-shot HTTP server.

Move the artifact-prefetch cluster into its own artifact_prefetch.rs:
prefetch_artifact_value_with_policy and everything only it calls
(declared_source_policy_block_message, huggingface_policy_uri_block_message,
artifact_declares_huggingface_authenticated_policy, artifact_is_huggingface,
is_huggingface_url, http_url_host, prefetch_blocked_value,
artifact_bytes_path_for_marker) together with ARTIFACT_PREFETCH_TIMEOUT, so
the new module needs nothing from sandbox.rs and the two do not form a
cycle. sandbox.rs keeps the sandbox-tool dispatch and calls into the new
module. The atomic writer comes from rocm_core::atomic_write, the same
import sandbox.rs used.

The tests move with the code: twelve sandbox_prefetch_* tests,
huggingface_url_detection_is_host_scoped, and the serve_one_http_response
and sha256_hex test helpers. Shared fixtures are still reached through
crate::test_support. The leaf test names are unchanged, so the leaf-name
test list of the rocmd crate is identical before and after.

Every edit that is not a verbatim move:
- prefetch_artifact_value_with_policy is widened from private to
  pub(crate), because sandbox.rs now calls it across the module boundary.
- lib.rs declares the new private module.
- Import lines: sandbox.rs drops the imports only the moved code used and
  imports the entry point; artifact_prefetch.rs adds its own. Path, which
  sandbox.rs's Linux-only bubblewrap test used through the old top-level
  import, is now imported inside that tests module.
- sha256_hex was a module-level #[cfg(test)] function; it now sits in the
  new tests module, so its #[cfg(test)] attribute is gone with the move.
- Two of the fourteen sandbox_prefetch_* tests stay in sandbox.rs:
  sandbox_prefetch_requires_artifact_ref and
  sandbox_prefetch_unknown_artifact_ref_errors drive run_sandbox_tool, which
  stays, and moving them would make the new module's tests import
  sandbox.rs back.
- A module doc comment on the new file.
- docs/architecture.md gains the entry for the new file and the sandbox.rs
  entry is corrected to say it now hands the artifact over.

No behaviour change. Production lines of sandbox.rs: 826 before, 456 after
(artifact_prefetch.rs: 386).

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the artifact-prefetch refactor.

No open issues 🎉

What changed in this PR

This refactor separates artifact prefetching from rocmd’s sandbox execution code, reducing sandbox.rs’s responsibilities while preserving existing behavior and reusing the shared atomic writer.

Changes:

  • Moves download policy checks, streaming downloads, helpers, and related tests into artifact_prefetch.rs.
  • Keeps sandbox dispatch and artifact-reference validation in sandbox.rs, with a one-way call into the new module.
  • Registers the private module and documents the revised module boundaries.
File Description
docs/​architecture.md Documents the prefetch module and updates sandbox responsibilities.
apps/​rocmd/​src/​sandbox.rs Delegates prefetching and retains dispatch-level validation tests.
apps/​rocmd/​src/​lib.rs Declares the private artifact_prefetch module.
apps/​rocmd/​src/​artifact_prefetch.rs Houses the moved prefetch implementation, helpers, timeout, and tests.

Review effort: Balanced

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor
Security scan found no critical or high findings on lines changed by this pull request

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

@jussielo-amd
jussielo-amd marked this pull request as ready for review November 10, 2026 11:36
@jussielo-amd
jussielo-amd requested a review from a team as a code owner November 10, 2026 11:36
…andbox

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

# Conflicts:
#	docs/architecture.md

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The move itself looks right. Compared against the removed lines in sandbox.rs, the code in artifact_prefetch.rs is identical apart from the changes the PR description lists (imports, the module doc comment, prefetch_artifact_value_with_policy becoming pub(crate), and sha256_hex moving into the tests module). CI is green and the docs/architecture.md entries match the code. I found nothing that breaks behavior.

Two things I'd like addressed before merge:

  1. Test count vs. the ticket. 12 of the 14 sandbox_prefetch_* tests moved; the two that call run_sandbox_tool stay in sandbox.rs to avoid an import cycle. Please either move them too (for example behind a small shared test helper) or state the exception explicitly in the ticket and PR text, so the "all 14 moved" expectation doesn't silently go unmet.
  2. Test-list evidence. The PR description says the cargo test -p rocmd -- --list leaf names are unchanged before and after, but the comparison isn't shown. Please paste the diff of the two lists so reviewers can confirm no test was dropped or renamed.

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · review-code · head bf9e77c7

Review — no blocking findings

Full review of the whole change (4 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, docs/architecture.md).
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (this is a public review and ROCMAI-822 is an internal tracker reference); judged only against what the PR description itself states, which this review found to be accurate.

Blocking

None

Non-blocking

  • No single test exercises the full dispatch seam for prefetch_artifact — apps/rocmd/src/sandbox.rs:221-235, apps/rocmd/src/sandbox.rs:913-947, apps/rocmd/src/artifact_prefetch.rs:875
    run_sandbox_tool's PrefetchArtifact arm resolves the artifact via resolve_model_recipe_artifact and then calls prefetch_artifact_value_with_policy. The two tests kept in sandbox.rs (sandbox_prefetch_requires_artifact_ref, sandbox_prefetch_unknown_artifact_ref_errors) only exercise the validation before that call (missing/unknown artifact ref); every test that actually drives prefetch_artifact_value_with_policy (e.g. sandbox_prefetch_downloads_direct_artifact_when_approved) calls it directly in artifact_prefetch.rs, bypassing run_sandbox_tool and the recipe-registry resolution step. This gap predates the move (same split of behavior already existed in one file), so it isn't introduced by this PR, but splitting the two halves across files makes the gap harder to notice from either file alone.
    Confidence 75 · logic · Fix: a small integration test (can live in either file) that drives a successful or policy-blocked prefetch through run_sandbox_tool end to end.

Decisions for the author

  • Two of the fourteen sandbox_prefetch_* tests stay in sandbox.rs — tradeoff
    Moving sandbox_prefetch_requires_artifact_ref/sandbox_prefetch_unknown_artifact_ref_errors into artifact_prefetch.rs would require them to call run_sandbox_tool, which stays in sandbox.rs, creating the import cycle the PR is designed to avoid. Keeping them put is the correct call given that constraint, and the PR text already states the exception explicitly. Confirm this is still the intended resolution to the open reviewer question about matching the ticket's count of 14, since the two options raised there (move them behind a shared test helper, or state the exception) — this PR already does the latter.
  • Policy-decision logic and network-IO/download-test-harness code share one file (artifact_prefetch.rs, ~387 production lines) — non-blocking-improvement
    A further split (pure policy gating vs. streaming download + its one-shot test HTTP server) could let the policy rules be unit-tested without any IO harness in scope, but the current file is well under the 1,500-line module-size cap and the current boundary is coherent (everything only prefetch_artifact_value_with_policy calls moved together); not necessary now.

Positive signals

  • The docs/architecture.md entries for both files accurately describe the real import graph (artifact_prefetch.rs has no use crate::sandbox — confirmed no back-edge — and sandbox.rs's entry correctly states it is the only caller), rather than the generic restatement these entries sometimes degrade into.
  • prefetch_artifact_value_with_policy's widened visibility is pub(crate), which is the minimal visibility this move requires (both modules are declared as direct siblings in lib.rs), not broader than needed.
  • Every function and test in the claimed move list was verified byte-for-byte identical between the old sandbox.rs and the new artifact_prefetch.rs apart from the explicitly enumerated exceptions (visibility, imports, module doc comment, sha256_hex losing its test-only attribute since it now sits directly inside mod tests).

Deployment notes

None — pure internal module split, no behavior, CLI surface, or output change.

What this covered

Read both files completely in both their before and after states (apps/rocmd/src/sandbox.rs 2013→1034 lines, new apps/rocmd/src/artifact_prefetch.rs 1001 lines), plus the full lib.rs and docs/architecture.md diffs, against a14f82b41e75cc6d88d71b77193c59496bb81493...bf9e77c7427c8656339b705dc375d2d04ddb4c98. Two independent fan-out workers: one on the coupled sandbox.rs+artifact_prefetch.rs group verifying the move is behavior-preserving (4a), one on the combined agent-instruction/history/prior-PR/code-comment passes (4b) against this repo's AGENTS.md. Design review (Step 7) was investigated directly rather than via separate spawned workers, given how narrowly answerable the questions were for a change this size and shape. Also independently re-verified: module-line counts against the 1,500-line cap, the pub(crate) visibility is minimal (both modules are crate-root siblings), no import cycle, a leak scan of the diff (clean — only example.invalid/.evil.example test fixture URLs and the real huggingface.co domain used for policy tests), and that the PR's own merge commit (which recorded a real conflict in docs/architecture.md against a concurrently-landed PR) resolved to exactly the two intended entries with no stray conflict markers or stale text.

CI: all checks green (gh pr view --json statusCheckRollup), including clippy, the affected-crate test jobs, E2E tests, CodeQL, and the security-scan suite; Skill checks (skillscope) and Sphinx docs build are SKIPPED but unrelated (this change touches neither skills/ nor docs/rocm-docs/).

Did not run locally: cargo test -p rocmd, the e2e scenario, or the full workspace suite — relied on CI's green result for these, which the project treats as authoritative. Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference).

Not reconciled against this PR's own review thread. A prior human reviewer left two open questions (why 12 of 14 tests moved, and asking for the before/after cargo test -- --list diff to be pasted) that this blind pass happens to independently speak to: direct comparison of the old and new file contents confirms the moved test set is unchanged and that the two-test exception is correctly reasoned (avoids an import cycle) — but this review did not read that thread and this is not a response to it.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

…andbox

Conflict in docs/architecture.md: main reworded the anthropic_proxy.rs entry
in the apps/rocmd list while this branch added the artifact_prefetch.rs entry
directly below it. Both are kept: main's anthropic_proxy.rs entry as it now
reads, followed by the new artifact_prefetch.rs entry.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the artifact-prefetch extraction, although tests were not rerun in this review environment.

No open issues 🎉

Review effort: Balanced

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · review-code · head ef92fc0d

Review — no blocking findings

Full review of the whole change (4 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, docs/architecture.md).
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (this is a public review and ROCMAI-822 is an internal tracker reference); judged only against what the PR description itself states, which this review found to be accurate.

Blocking

None

Non-blocking

  • No single test exercises the full dispatch seam for prefetch_artifact — apps/rocmd/src/sandbox.rs:221-235, apps/rocmd/src/sandbox.rs:913-947, apps/rocmd/src/artifact_prefetch.rs:875
    run_sandbox_tool's PrefetchArtifact arm resolves the artifact via resolve_model_recipe_artifact and then calls prefetch_artifact_value_with_policy. The two tests kept in sandbox.rs (sandbox_prefetch_requires_artifact_ref, sandbox_prefetch_unknown_artifact_ref_errors) only exercise the validation before that call (missing/unknown artifact ref); every test that actually drives prefetch_artifact_value_with_policy (e.g. sandbox_prefetch_downloads_direct_artifact_when_approved) calls it directly in artifact_prefetch.rs, bypassing run_sandbox_tool and the recipe-registry resolution step. This gap predates the move (the same split of behavior already existed within one file), so it isn't introduced by this PR, but splitting the two halves across files makes the gap harder to notice from either file alone.
    Confidence 75 · logic · Fix: a small integration test (can live in either file) that drives a successful or policy-blocked prefetch through run_sandbox_tool end to end.

Decisions for the author

  • Two of the fourteen sandbox_prefetch_* tests stay in sandbox.rs — tradeoff
    Moving sandbox_prefetch_requires_artifact_ref/sandbox_prefetch_unknown_artifact_ref_errors into artifact_prefetch.rs would require them to call run_sandbox_tool, which stays in sandbox.rs, creating the import cycle the PR is designed to avoid. Keeping them put is the correct call given that constraint, and the PR text states the exception explicitly.
  • Policy-decision logic and network-IO/download-test-harness code share one file (artifact_prefetch.rs, ~386 production lines) — non-blocking-improvement
    A further split (pure policy gating vs. streaming download + its one-shot test HTTP server) could let the policy rules be unit-tested without any IO harness in scope, but the current file is well under the 1,500-line module-size cap and the current boundary is coherent (everything only prefetch_artifact_value_with_policy calls moved together); not necessary now.

Positive signals

  • The production code move is verified byte-for-byte: diffing the old sandbox.rs's prefetch-cluster block against the new artifact_prefetch.rs block shows exactly one textual difference, fn → pub(crate) fn on prefetch_artifact_value_with_policy (plus one blank-line delta) — nothing else changed in transit.
  • docs/architecture.md's new entries accurately describe the real import graph: artifact_prefetch.rs has no use crate::sandbox anywhere in the file (confirmed, no back-edge), and a whole-tree search finds sandbox.rs is the module's only caller.
  • The widened visibility (pub(crate)) is the minimum this move requires, since both modules are declared as direct siblings in lib.rs.

Deployment notes

None — pure internal module split, no behavior, CLI surface, or output change.

What this covered

Read apps/rocmd/src/artifact_prefetch.rs (1001 lines) and apps/rocmd/src/lib.rs (27 lines) completely at the reviewed head, and apps/rocmd/src/sandbox.rs (1034 lines) completely at the reviewed head plus its pre-move state at the PR's merge-base, to directly diff the moved block. Read the PR's own net diff against its base (f7c5fef7a...ef92fc0d4, matching GitHub's reported 4 files / +1009 / -985) in full, including both hunks of the docs/architecture.md change. Searched the whole tree at the reviewed commit for any remaining reference to the moved private helpers or any caller of artifact_prefetch/prefetch_artifact_value_with_policy other than sandbox.rs; found none.

This PR's head advanced (bf9e77c7 → ef92fc0d) since an earlier automated review was posted here, by a routine merge of upstream main. Established directly from git, not taken as a claim: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs and apps/rocmd/src/lib.rs are byte-identical between the two commits, and the docs/architecture.md content this PR itself owns (its diff against its own base) is unchanged; the merge's only effect was pulling in unrelated upstream files and unrelated docs/architecture.md entries for other modules that also exist on current main. This review re-verified the change directly rather than reusing that review's conclusions.

CI at time of review: everything that has reported a conclusion is green, including clippy, Test (affected crates) (both runners), build-and-test (both runners), CodeQL, and the full security-scan suite. Three jobs had not yet concluded (E2E tests, E2E tests (ubuntu-26.04), Windows build and test (debug)) — none had failed, so this is reported as incomplete information rather than a finding. Skill checks (skillscope) and Sphinx docs build are skipped but unrelated (this change touches neither skills/ nor docs/rocm-docs/).

Did not run locally: cargo test -p rocmd, the e2e scenario, or the full workspace suite — relied on CI's result for these. Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference). Design review (Step 7) was reasoned through directly rather than via separate spawned workers, given how narrowly answerable the questions were for a change this size and shape; a prior round of this same review already fanned the code passes out across independent workers over this identical content.

Not reconciled against this PR's own review thread. A prior human reviewer's two open questions (why 12 of 14 tests moved, and asking for the before/after cargo test -- --list diff) are already answered by the PR description's own text, which this review judged independently against the code and found accurate — but this review did not read that thread and this is not a response to it.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I compared the removed and added lines mechanically: the production code moved into artifact_prefetch.rs is verbatim. The only other changes are imports, the module doc comment, pub(crate) on the entry function, and the docs/architecture.md entries. artifact_prefetch.rs does not import sandbox.rs, so there is no module cycle. The architecture entry is added and the sandbox.rs entry corrected, and file sizes are well under the module limit.

One note, not blocking: 2 of the 14 sandbox_prefetch_* tests in the ticket stay in sandbox.rs because they drive run_sandbox_tool, and moving them would recreate the cycle. That reasoning holds, but it is only a partial match to the ticket's step.

CI: the substantive checks pass. The Strix Halo WSL2 lane fails at the distro install step, which looks like infrastructure. The Strix Halo Windows E2E lane fails at "Run E2E tests on Strix Halo Windows" and I could not tell why, so a rerun or log check would be worthwhile before merge. I did not build or run tests locally.

…rtifact-prefetch-out-of-sandbox

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…_tool

The prefetch tests that moved to artifact_prefetch.rs call the entry
point directly, and the two that stay in sandbox.rs stop at the argument
checks, so nothing drove a reference that resolves through the recipe
registry into the policy decision. Add one test that signs a recipe index
in the test, resolves a reference through run_sandbox_tool, and checks that
the recipe's model id is used, that the default policy fetches and caches
nothing, and that an approved policy reaches the cached-marker answer.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new test mutates shared recipe-index environment variables without synchronizing an existing reader, which can cause unexpected configuration errors and flaky parallel test runs.

1 open issue

Review effort: Balanced

Comment thread apps/rocmd/src/sandbox.rs Resolved
@jussielo-amd
jussielo-amd added this pull request to the merge queue Nov 11, 2026
@jussielo-amd
jussielo-amd removed this pull request from the merge queue due to a manual request Nov 11, 2026
… env

The dispatch test sets the model recipe index, signature and public key
environment variables and held a mutex only of its own, while
sandbox_prefetch_unknown_artifact_ref_errors resolves a recipe reference
through the same registry and read them without it. Run in parallel, that
reader could see the index path set before its signature and key, or the
variables mid-restore, and fail with a configuration error instead of
"was not found". A developer's own exported variables could do the same.

Move the guard into test_support as RecipeIndexEnv, with builtin_catalog()
for readers and signed_index() for the test that supplies an index. Both
take the one lock, the guard sets or clears all four recipe-registry
variables under it, and drop restores them. The unknown-reference test now
takes the built-in-catalog guard, which also makes it independent of the
shell it runs in. No other rocmd test reaches the registry: the cache-warm
tests inject their resolver.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The artifact-prefetch refactor has no active review issues.

1 open issue

Review effort: Balanced

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · review-code · head e72d6caa

Review — no blocking findings

Full review of the whole change (5 files: apps/rocmd/src/sandbox.rs, apps/rocmd/src/artifact_prefetch.rs, apps/rocmd/src/lib.rs, apps/rocmd/src/test_support.rs, docs/architecture.md), diff eb2a8194a...e72d6caa1.
Closes ROCMAI-822 per the PR description. Ticket named but not retrieved here (public review, internal tracker reference); judged only against what the PR description itself states, which this review found accurate against the diff.

Blocking

None

Non-blocking

  • The new env-mutation test guard isn't registered with this repo's own regression check for exactly this bug class — apps/rocmd/src/test_support.rs:324-391 (RecipeIndexEnv, used at apps/rocmd/src/sandbox.rs:938 and apps/rocmd/src/sandbox.rs:1011), xtask/src/env_mutation_contract.rs.
    This repo has a dedicated static-analysis test, a_test_mutating_the_environment_serializes_itself, that scans every #[test] function body for a bare set_var(/remove_var(/RestoredEnvVar::set( call not preceded by a lock — either a *_TEST_LOCK-suffixed static's .lock(, or a type listed in NAMED_SERIALIZERS (today just ScopedTestEnv, documented there as "a type whose whole job is to perform the mutation on a test's behalf"). RecipeIndexEnv is structurally the same kind of type as ScopedTestEnv — one RAII value that takes the lock and performs the mutation inside its own constructor — but its backing static is named RECIPE_INDEX_ENV (no _TEST_LOCK suffix), and neither the type nor its constructors were added to NAMED_SERIALIZERS/MUTATIONS. Because the actual set_var/remove_var calls live inside RecipeIndexEnv::apply()/Drop::drop() — plain functions, not #[test] bodies — the scanner's text match never fires for either of the two call sites in sandbox.rs, so they are not merely passing the guard, they are invisible to it. At runtime the serialization is correct (I traced it directly: both constructors touch all four vars every time, Drop's explicit restore runs before the MutexGuard field auto-drops releases the lock, and poison recovery via PoisonError::into_inner is used on both the lock and the restore path), so this is not a live bug — but it means the one guard this repo built specifically for this bug class (its own doc comment cites EAI-8397) provides no safety net here going forward, unlike for the existing pattern.
    Confidence 78 · architectural · Fix: add RecipeIndexEnv (or its constructors) to xtask/src/env_mutation_contract.rs's NAMED_SERIALIZERS, mirroring how ScopedTestEnv is already covered.

Decisions for the author

  • Two of the fourteen sandbox_prefetch_* tests stay in sandbox.rs — tradeoff
    sandbox_prefetch_requires_artifact_ref and sandbox_prefetch_unknown_artifact_ref_errors assert the PrefetchArtifact dispatch arm's argument checks (missing/unknown --artifact-ref), which run in sandbox.rs before resolve_model_recipe_artifact/prefetch_artifact_value_with_policy are ever reached. Moving them into artifact_prefetch.rs would require that module to call back into sandbox.rs's run_sandbox_tool, recreating the cycle this PR removes. Keeping them beside the dispatch they exercise is the reasoned choice given that constraint; the PR text states the exception explicitly.
  • The recipe-index configuration is read from process-wide environment variables rather than an injectable seam — non-blocking-improvement
    xtask/src/env_mutation_contract.rs's own doc comment states the preferred long-term fix for this whole class of test is to avoid environment mutation entirely ("pass the value in through a test seam", citing newest_rocm_install_dir_in as the pattern), which would make a lock unnecessary rather than just correctly held. That would mean changing crates/rocm-core/src/model_catalog.rs's load_model_recipe_registry to accept an injected override, which is production-code surface this PR (a behavior-preserving move) correctly leaves untouched; flagging it for whoever next touches that resolver.

Positive signals

  • The new dispatch test genuinely exercises the PrefetchArtifact seam end-to-end rather than calling the policy function directly: its first assertion only holds if run_sandbox_tool actually resolved the artifact reference through the recipe registry (asserts the resolved recipe's canonical model id), and its second call only lands on "cached" if the caller's SandboxToolPolicy — not the default — reached the policy decision.
  • RecipeIndexEnv's Drop is ordered correctly: the explicit restore runs to completion before the struct's own MutexGuard field is auto-dropped (Rust drops fields in declaration order after the explicit Drop::drop body), so the lock is never released mid-restore.
  • The commit history shows the fix converging within the PR itself rather than needing a follow-up round: the dispatch-test commit introduced a lock scoped only to its own new test, and the very next commit recognized the sibling test it left unguarded and generalized the guard to cover both.

Deployment notes

None — pure internal module split plus a test-only change; no behavior, CLI surface, or output change.

What this covered

Read all 5 changed files completely at the reviewed head, diffed the moved production block directly against its pre-move location at the merge-base, and read xtask/src/env_mutation_contract.rs in full to verify the one finding above by tracing its scanner logic rather than asserting it. Two independent fan-out workers: one over the coupled sandbox.rs/artifact_prefetch.rs/lib.rs/test_support.rs group verifying the move and the new test-isolation fix are both correct; one running the agent-instruction/history/prior-changes/code-comment passes against this repo's AGENTS.md and its existing env-var-guard precedent (RestoredEnvVar/ScopedTestEnv). Design review (Step 7) was reasoned through directly given how narrowly answerable the questions were for a change this shape and size.

No Rust toolchain available in this sandbox: did not run cargo test, cargo xtask check-module-size, or the env_mutation_contract test itself. The finding above and the verified positive signals are from reading the relevant source directly (the scanner's own logic, the Drop/lock code) rather than executing it.

CI at time of review: everything that had concluded was green — clippy, both build-and-test runners, Test (affected crates) (ubuntu-26.04), Coverage (rocm-dash crates, ratcheted), Windows release build and lifecycle, the full security-scan suite, CodeQL, lint/docs/signature checks, and copilot-pull-request-reviewer. Still in progress and not yet failed: both E2E tests runners, Build Linux PR binaries, Test (affected crates) (default runner), Windows build and test (debug) (the required job that runs the workspace-wide cargo test --all-targets, which is what actually exercises the env_mutation_contract guard discussed above), and Security scan / codeql / rust. Reported as incomplete information, not a finding.

Did not retrieve ROCMAI-822 from any tracker (named but not retrieved, per public-review policy for an internal ticket reference). Not reconciled against this PR's own review thread or any prior round — this was read blind, with no earlier review content consulted.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

@jussielo-amd
jussielo-amd added this pull request to the merge queue Nov 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 11, 2026
…rtifact-prefetch-out-of-sandbox

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the change separating artifact prefetching from sandbox execution.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@jussielo-amd
jussielo-amd added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 6c84db9 Nov 11, 2026
41 checks passed
@jussielo-amd
jussielo-amd deleted the refactor-rocmd-move-artifact-prefetch-out-of-sandbox branch November 11, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants