Skip to content

feat(diagnose): measure an APU on the pool its engine allocates from - #1054

Open
r0x0r wants to merge 18 commits into
mainfrom
rocmai-912-unified-memory
Open

r0x0r wants to merge 18 commits into
mainfrom
rocmai-912-unified-memory

Conversation

@r0x0r

@r0x0r r0x0r commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Why

rocm diagnose --model could not answer "will this model fit" on any AMD APU.

An APU has no private VRAM. amd-smi and every Windows adapter query report the fixed BIOS carve-out as the device's memory — 512 MiB measured on a Ryzen 7 4700U, ~4 GiB on a Strix Halo — while the engine allocates out of system memory the GPU is given an aperture onto. Comparing a recipe minimum against the carve-out refuses nearly every curated model on machines that demonstrably serve them, so the classification answered UnifiedMemoryUnreadable and the verdict came back undetermined, on the stated grounds that the real pool "is not readable by anything in this binary today".

It is readable, and that sentence was the defect.

What changes

The aperture is read where the kernel and the driver actually publish it:

source what it reports
Linux mem_info_gtt_total the kernel's GTT aperture cap, beside the mem_info_vram_total the carve-out comes from
Windows DXGI_ADAPTER_DESC1::SharedSystemMemory the WDDM shared budget, which nothing in the PowerShell/CIM surface the other probes use can report

Neither is installed RAM nor a fraction of it. Half of RAM is an assertion about driver policy; BIOS settings and amdgpu.gttsize move the real figure, so it is measured rather than inferred.

A fit against that figure yields Degraded, never Ready. The pool is shared with the operating system, so what was measured is a ceiling rather than an allocation, and nothing here measured how much of it is free. Below the ceiling the verdict still blocks, and says that freeing memory will not raise an aperture the driver will not hand out.

UnifiedMemoryUnreadable survives, narrowed to its true meaning: an APU whose figure was absent or unparseable on this host. It keeps its distance from AcceleratorMemoryUnknown, whose "run amd-smi again" remediation would still be a dead end here.

The aperture is consulted only on the APU branch. Both files exist on a discrete card too, and crediting one with the other would report an RX 7900 as having 24 + 64 GiB, after which everything fits.

The same ambiguity is refused one level down, on both platforms: the figure is per-device, so a host showing more than one AMD adapter yields no figure at all rather than the larger of two. On an APU paired with a discrete Radeon there is nothing at that point that says which adapter is the integrated one, and picking would credit the APU with the discrete card's pool.

A cpu_only recipe is left alone. The shared-pool check is the last thing the assessment does and originally fired on the host alone, so a recipe that never allocates from the accelerator was softened to Degraded and handed a remediation for a device it does not touch — on evidence no earlier branch had put in the report. It is now gated on requires_gpu, as the "no GPU is visible" check beside it already was.

The Degraded plan names no check that can never pass. That verdict is forced by the host whenever the pool is shared and the recipe needs the GPU, so the verify: rocm diagnose --model <model> the plan used to carry reprinted the identical verdict however much the user closed — a loop with no exit, which reads as a check the user failed to satisfy rather than one that cannot be. The plan now names the one thing that does settle it (rocm serve), states in its note that this check cannot read how much of the pool is free so re-asking reports the same thing, and carries no verify at all; the renderer omits an empty one.

The GGUF estimator says the figure it compared against is shared. A fits on an APU means "the machine would allow this at best", not "this much is waiting for it", and the note on the estimate says so — the figure is printed beside the totals with nothing else to tell the two kinds of number apart.

User-facing text that still described the old behaviour was corrected with it: README.md, docs/manual-testing.md, and the --model / --context-size help in apps/rocm/src/cli.rs all said an APU's pool "can't be read" / answers "undetermined by design" / that the comparison is "against the dedicated VRAM of the GPU".

Verification

Full gate on a Linux ROCm box (MI300X, gfx942): fmt --all --check, clippy --workspace --all-targets -D warnings, clippy -p e2e-cucumber --test e2e -D warnings, test -p e2e-cucumber --test feature_naming, xtask check-architecture-doc, xtask check-crate-edges, xtask check-module-size, test -p rocm-core (1584 passed), test -p rocm --bin rocm (1904 passed).

Mutation evidence — each check was confirmed to fail against deliberately broken production code:

broken input check that caught it
aperture credited to a discrete card the_gtt_aperture_never_reaches_a_discrete_card, scenario diagnose-73 (@id:diagnose-model-a-discrete-card-is-never-credited-with-the-aperture)
a cleared pool reported as Ready an_apu_whose_pool_clears_the_minimum_is_degraded_not_ready, an_apu_with_a_readable_pool_is_degraded_never_ready, scenario diagnose-71 (@id:diagnose-model-an-apu-is-measured-on-its-shared-pool)
an unreadable pool given a guessed number scenario diagnose-72 (@id:diagnose-model-an-unreadable-shared-pool-is-said-not-guessed)
an APU credited with a discrete Radeon's budget two_amd_adapters_leave_the_shared_budget_unattributable, two_amd_cards_leave_the_gtt_aperture_unattributable, scenario diagnose-74 (@id:diagnose-model-two-amd-cards-leave-the-pool-unattributable)
the requires_gpu(recipe) && required.is_some() gate removed, softening a cpu_only recipe a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses, scenario diagnose-75 (@id:diagnose-model-a-cpu-only-recipe-is-not-softened-by-a-pool-it-never-uses) — the scenario's first half goes red (left: "degraded", right: "ready")
the dead-end verify restored on the Degraded plan the_shared_pool_plan_names_no_check_that_can_never_clear — the only failure in the suite
the estimator's shared-pool note deleted a_shared_pool_is_compared_against_its_ceiling_and_said_to_be_shared, scenario diagnose-76 (@id:diagnose-model-a-gguf-estimate-says-its-pool-is-shared) — red both when the whole is_shared_pool() block goes and when only the notes.push does, leaving the flag set
that note pushed unconditionally (if true) the same test, via its negative half against Dedicated and Unknown

The last two behaviours that had only unit coverage now have scenarios of their own, diagnose-75 and diagnose-76, both Linux-gated and both running on the GPU-less mock lane like the four APU scenarios above them. diagnose-75 needs a cpu_only recipe, and all 49 shipped recipes are gpu_required or gpu_preferred, so it plants a signed recipe index — the only way the CLI loads another catalog — holding the smallest curated recipe twice, once as shipped and once with its device policy and GPU minimum removed, and asks about both on the same planted APU: the first half is the immunity, the second is the premise that the host would otherwise soften.

Writing it surfaced a harness trap worth naming. scenario_command takes command_env rather than borrowing it, so a Given's fixture reaches exactly one child process; a scenario that asks two questions of the same host silently gets the runner's real machine for the second. The first draft of diagnose-75 answered about this box's 191.9 GiB MI300X out of the built-in catalog while reading as though it answered about the planted APU. ask_again_on_the_same_planted_machine restores the environment between questions, and the first half now asserts the planted 60 GiB aperture so a leak fails loudly instead of changing the subject.

Not verified: the Windows path has never executed. The workspace cannot cross-compile, so it was type-checked and clippy-checked against x86_64-pc-windows-msvc in isolation and has unit coverage only. Its DXGI vtable offsets and COM lifetimes are unexercised by any running binary. That gap is written into the testing guide beside the coverage it qualifies, so it outlives this description: the adapter choice and the implausible-reading rule are pure functions tested on every platform, and the FFI around them is tested by nothing.

Also unverified by construction: the windows-sys feature list. Nothing in the Linux or macOS toolchain here builds the MSVC target, so the merge resolution below was checked as a set operation rather than by compiling.

Scope

windows-sys 0.61 carries no COM interfaces at all — there is no Win32_Graphics_Dxgi feature to enable — so the three vtable slots that are read are hand-declared and dxgi.dll is loaded by absolute System32 path. The alternative was taking on the windows crate for three struct fields.

main added a Win32_Security_Authorization feature to the same list after this branch was cut. The merge keeps both sides: windows-sys features are additive, so the resolution is the strict union (8 + 8 distinct features in, 9 out, with every feature from either side present).

Non-goals, where a reviewer may wonder why this stops where it does:

  • How much of the pool is free is not measured, only how large it can get. That is what makes the verdict Degraded, and why its plan now says so rather than offering a check to re-run.
  • No BIOS carve-out or amdgpu.gttsize recommendation, and no SVM mapping failed detection. ROCMAI-912 also describes those; they are a separate diagnose concern and need none of this.
  • Partial DXGI enumeration is treated as a complete answer. An EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND ends the adapter loop the same way the clean end of the list does, so a second AMD adapter sitting past a transient error would be missed and the first one's budget confidently attributed. NOT_FOUND is what that call returns at the true end of the list in practice, and the Windows path is unexecuted either way, so this is left as known and untested rather than guessed at.

One adjacent fix rides along because the scenarios cannot exist without it: the serve path's DRM read now goes through host_path like every other DRM read, so a simulated host root reaches it. hardware_root's module doc already listed that path as re-rooted, and this was the one caller naming it directly.

`rocm diagnose --model` could not answer "will this model fit" on any AMD
APU. An APU has no private VRAM: `amd-smi` and every Windows adapter query
report the fixed BIOS carve-out as the device's memory -- 512 MiB measured
on a Ryzen 7 4700U, ~4 GiB on a Strix Halo -- while the engine allocates
out of system memory the GPU is given an aperture onto. Comparing a recipe
minimum against the carve-out refuses nearly every curated model on
machines that demonstrably serve them, so the classification answered
`UnifiedMemoryUnreadable` and the verdict came back undetermined, on the
stated grounds that the real pool "is not readable by anything in this
binary today".

It is readable. Linux exposes the kernel's own GTT aperture cap as
`mem_info_gtt_total`, beside the `mem_info_vram_total` the carve-out comes
from; Windows exposes the WDDM shared budget as
`DXGI_ADAPTER_DESC1::SharedSystemMemory`, which nothing in the
PowerShell/CIM surface the other probes use can report. Both are read by a
new `host_gpu/unified_memory.rs` and carried as `AcceleratorMemory::Unified`.
Neither is installed RAM nor a fraction of it: half of RAM is an assertion
about driver policy, and BIOS settings and `amdgpu.gttsize` move the real
figure.

A fit against that figure yields `Degraded`, never `Ready`. The pool is
shared with the operating system, so what was measured is a ceiling rather
than an allocation, and nothing here measured how much of it is free.
Below the ceiling the verdict still blocks, and says that freeing memory
will not raise an aperture the driver will not hand out.

`UnifiedMemoryUnreadable` survives, narrowed to its true meaning: an APU
whose figure was absent or unparseable on this host. It keeps its distance
from `AcceleratorMemoryUnknown`, whose "run `amd-smi` again" remediation
would still be a dead end here.

The aperture is consulted only on the APU branch. Both files exist on a
discrete card too, and crediting one with the other would report an RX 7900
as having 24 + 64 GiB, after which everything fits; a unit test and
scenario diagnose-61 pin that down.

The Windows path cannot be compiled on either available machine and has
unit coverage only; it was type-checked and clippy-checked against
`x86_64-pc-windows-msvc` in isolation, since the workspace cannot
cross-compile. Its DXGI vtables are hand-declared because `windows-sys`
0.61 carries no COM interfaces at all -- there is no `Win32_Graphics_Dxgi`
feature to enable -- and `dxgi.dll` is loaded by absolute System32 path.

The serve path's DRM read now goes through `host_path` like every other
DRM read, so a simulated host root reaches it; `hardware_root`'s module
doc already listed that path as re-rooted and this was the one caller
naming it directly.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
`amd_shared_system_memory` took the largest `SharedSystemMemory` of every
non-software AMD adapter DXGI enumerates. Its reasoning only ever covered a
foreign iGPU beside an AMD card -- hence the vendor filter -- and never two
real AMD adapters, which is an APU paired with a discrete Radeon.

The budget is per-adapter and nothing at that point says which adapter is the
integrated one, so the larger figure wins and the APU is credited with the
discrete card's pool. That is the Windows spelling of the contamination
`the_gtt_aperture_never_reaches_a_discrete_card` pins on Linux, and the reason
`linux_drm_amdgpu_gtt_total_bytes_in` answers `None` above one DRM card rather
than choosing between them. Two adapters is a figure that could not be
attributed, not a figure to pick from.

A software adapter is still dropped before the count rather than after, so WARP
beside a real AMD part does not make it ambiguous.

The new test asserts both enumeration orders: a rule that only holds when the
larger adapter happens to enumerate second is not a rule.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r
r0x0r marked this pull request as ready for review November 12, 2026 10:31
@r0x0r
r0x0r requested a review from a team as a code owner November 12, 2026 10:31
@r0x0r
r0x0r requested a review from rominf November 12, 2026 10:31
@r0x0r r0x0r closed this Nov 12, 2026
@r0x0r r0x0r reopened this Nov 12, 2026

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work here. No single blocker in the design, but a few things need addressing before this can merge.

Needs changes

  1. Windows adapter enumeration can silently break the "exactly one AMD adapter" rule. An EnumAdapters1 error other than NOT_FOUND ends the loop and returns a partial list as if complete; an adapter whose GetDesc1 fails is skipped. If the missing adapter is a second AMD part, the APU gets credited with one adapter's budget, which is the contamination the second commit set out to prevent. Return None on either path (inline comments).
  2. detect_host_unified_memory_gib() runs unconditionally from host_accelerator_memory, so every Windows diagnose --model does a LoadLibrary + DXGI enumeration, discrete-only hosts included. Please compute it only on the APU branch, and confirm the call site is not on an async worker (blocking FFI rule).
  3. The new GGUF behavior is untested. estimate_fit now gives Unified hosts a numeric estimate plus the shared-pool note, but nothing exercises it: deleting the note keeps everything green. Scenarios diagnose-59..61 only cover the curated-recipe path. Add a test that fails when the note is removed, and a scenario or a stated reason there isn't one.

Smaller

  • diagnose-60's !contains("offers 4.0 GiB") is vacuous if wording changes; measured_gpu_gib == None already proves the point.
  • cast_signed needs Rust 1.87; please confirm MSRV.
  • Please grep README, --help and docs/manual-testing.md for claims that an APU verdict is undetermined, and run cargo xtask check-module-size for model_fit.rs/model_readiness.rs.
  • Check that nothing (serve gating, exit codes, chat tool) treats the new Degraded as warn-or-refuse on every APU.

Scope / process

  • The PR is currently CONFLICTING with main and only the docs build has reported, so it needs a rebase and a full CI run. Scenario ids 59-61 should be rechecked after the rebase.
  • ROCMAI-912 asks for the GTT read plus carve-out/gttsize guidance and SVM detection; this does the read and defers the rest, which is fine as stated. The roughly 250-line Windows DXGI COM FFI path is beyond the ticket and has never been executed; consider splitting it into its own PR. The ticket also mentions #386 as overlapping work, so please check for duplication on rebase.

What looks good: the APU-only gating is pinned by tests, the single-adapter rule is applied on both OSes, the DLL loads by absolute path, and scenarios 59-61 assert message and state together.

if hr == DXGI_ERROR_NOT_FOUND {
break;
}
if hr < 0 || adapter.is_null() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An EnumAdapters1 error other than NOT_FOUND ends the loop and the partial list is returned as complete. If a second AMD adapter was not reached, the "exactly one AMD adapter" rule passes wrongly and the APU is credited with one adapter's budget. Return None from enumerate_adapters on this path.

let hr = ((*vtbl).get_desc1)(adapter.cast(), &raw mut desc);
(hr >= 0).then_some(desc)
};
if let Some(desc) = read {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An adapter whose GetDesc1 fails is dropped silently. That also defeats the single-adapter rule: an unreadable second AMD adapter makes the first look unique. Treat an unreadable description as not attributable (return None) instead of skipping.

// classification below stays a pure function of its arguments. It is one
// file read on Linux and one DXGI enumeration on Windows -- no
// subprocess, nothing this path waits on.
rocm_core::detect_host_unified_memory_gib(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now runs a DXGI enumeration on every Windows diagnose --model, including discrete-only hosts, and the "nothing this path waits on" claim is unverified for DXGI. Consider computing it lazily only on the APU branch, and confirm this is not reached from an async worker (blocking FFI rule).

// this at best" rather than "this much is waiting for it". Said on the
// estimate itself, because the figure is printed beside the totals with
// nothing else to tell the two kinds of number apart.
if memory.is_shared_pool() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New user-visible behavior with no test: a Unified host now gets a numeric estimate plus this note. Add a test that fails when the note is removed, and a scenario or a stated reason there isn't one for the GGUF path.

.expect("evidence is not an array")
.iter()
.filter_map(serde_json::Value::as_str)
.any(|line| line.contains("offers 4.0 GiB")),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This negative assertion passes vacuously if the wording ever changes; measured_gpu_gib(&model) == None above already proves no figure is claimed. Drop it or assert on the actual evidence line.

One conflict, in docs/architecture.md: main replaced the rocm-core module
map's bullet list with a table carrying a Class column, and this branch had
edited two of the bullets.

Took main's table, then ported both edits into it rather than keeping either
side whole. `unified_memory.rs` becomes its own `host_gpu/` row, which is how
the table already lists every other child of that module, instead of the
clause it was on this branch inside the `host_gpu.rs` bullet; the
`model_readiness.rs` row gains the sentence about an APU being measured on its
shared pool and so reaching `Degraded` but never `Ready`.

Keeping our side would have reverted the table for the whole crate; taking
theirs would have dropped the only citation of `unified_memory.rs`, which the
architecture-doc hook requires and which is the module this branch adds.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor
Security scan found no critical or high findings on lines changed by this pull request

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

`an_apu_whose_pool_clears_the_minimum_is_degraded_not_ready` and
`classify_accelerator_memory_reports_the_measured_unified_pool_for_an_apu`
chose their recipe by GPU-memory minimum alone. The first builtin recipe above
8 GiB prefers vLLM, which the platform gate rules out on native Windows before
memory is ever consulted, so the verdict came back `Blocked` for a reason that
has nothing to do with the pool and both assertions failed there while passing
everywhere else. The first Windows CI run on this branch is what caught it;
the tests had been wrong since they were written.

Selecting on the engine rather than on the host is deliberate. Filtering to
"ungated here" reads as the obvious fix and is a worse one: it would pick a
vLLM recipe on Linux and a Lemonade recipe on Windows, leaving neither lane
able to catch a regression in what the other runs. Pinning the engine class
keeps both lanes on the same recipe, so the memory arithmetic the tests are
named for is the only thing that varies between them.

The gate call is kept inside the predicate as a live guard rather than reduced
to a comment: if Lemonade ever stops being ungated, the recipe search fails
loudly at its `expect` instead of an assertion failing somewhere less obvious.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Verification pass on this review — three items resolve, two confirmed standing

@juhovainio — this review had not been answered. I have not changed code here (another working session holds this branch and I hit a non-fast-forward on it earlier today), so this is verification rather than fixes: facts for whoever picks it up, with the checkable claims actually checked.

Resolved since the review was written

  • "CONFLICTING with main, only the docs build has reported." Both resolved. The branch merged origin/main at d3e47b008 and is now MERGEABLE, with a full run green — 38 passed, 3 skipping, 0 failures. I also confirmed that merge resolved docs/architecture.md correctly: main had restructured it into per-crate tables, the merge took the table form (not the old bullet list), and host_gpu/unified_memory.rs has its row, so check-architecture-doc passes.
  • cast_signed / MSRV. No problem. The workspace pins rust-version = "1.96" (Cargo.toml:28) and rust-toolchain.toml channel 1.96.0. cast_signed stabilised in 1.87, well under the floor.
  • check-module-size for model_fit.rs / model_readiness.rs. Ran it on the merged tree: 424 source files checked, none over 1500 production lines. Neither file is near the cap.

Confirmed, still standing

  • detect_host_unified_memory_gib() is still unconditional. apps/rocm/src/model_fit.rs:194 calls it in the argument list of classify_accelerator_memory, with a comment arguing the point — that it keeps the classifier "a pure function of its arguments". That is a fair design instinct, but it does not dispose of your objection: purity is preserved just as well by passing None on the non-APU branch, and on Windows the unconditional form is a LoadLibrary + DXGI enumeration on every diagnose --model, discrete-only hosts included, for a value that is then discarded. Your second question — whether the call site sits on an async worker — is the one that decides whether this is merely wasteful or a blocking-FFI violation, and I did not trace it.

  • The GGUF path's shared-pool note is unasserted. I checked rather than took it on faith. The note at model_readiness.rs:647 is gated on host.accelerator_memory.is_shared_pool(), and the only places the phrase appears elsewhere are two comments (model_fit.rs:951, model_readiness.rs:1697) and the diagnose-59..61 scenarios, which exercise the curated-recipe path. Nothing asserts the note on the estimate_fit branch, so deleting it stays green — exactly as you said.

Not checked

The Windows EnumAdapters1 / GetDesc1 partial-enumeration finding (your item 1) is the one I would rank highest, since it defeats the single-adapter safety rule that the rest of the change leans on. I have not verified it: nothing available here builds the MSVC target, and that path has never been executed. It needs someone who can run it, or the None-on-error change made and reasoned about statically.

Leaving CHANGES_REQUESTED in place — items 1, 2 and 3 are all still open.

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — needs work

Full review of the whole change (13 files, +1567/-118).

Implements the APU-memory-measurement portion of ROCMAI-912 (reading the real GTT/WDDM shared-memory pool for APUs and capping the verdict at Degraded); the description itself scopes out free-memory measurement, BIOS/gttsize guidance, and SVM-mapping-failure detection as a separate concern. Ticket ROCMAI-912 named but not retrievable from here.

Blocking

  • A model that needs no GPU memory still gets forced to Degraded with GPU-specific remediation on a shared-memory host — crates/rocm-core/src/model_readiness.rs:658-681
    The final block in assess() downgrades any Ready/Degraded verdict to Degraded whenever host.accelerator_memory.is_shared_pool() is true, with no check on whether this particular recipe's required figure (recipe.min_gpu_mem_gb) was ever Some. A recipe with device_policy == "cpu_only" and no declared GPU-memory minimum falls into the (None, _) match arm (line 618), which leaves the verdict at its initial Ready; the final block then still fires, appending "this host's integrated GPU has no dedicated VRAM..." evidence and a "close what else is using memory before serving" fix for a model that never touches the GPU. cpu_only is a schema-legal device_policy value and no shipped catalog recipe currently combines it with an absent GPU-memory minimum, so this isn't live today — but it's reachable through the public function and untested (the one new test exercising the downgrade, an_apu_with_a_readable_pool_is_degraded_never_ready, only uses a GPU-requiring recipe).
    Confidence 85 · logic · Fix: gate the final block on requires_gpu(recipe) (or required.is_some()) in addition to is_shared_pool().

  • The new "no dedicated VRAM" note in the GGUF size estimator has no test — crates/rocm-core/src/gguf_fit.rs:850-857
    estimate_fit pushes a new note onto estimate.notes whenever memory.is_shared_pool(). No test in the file constructs AcceleratorMemory::Unified { .. }; the parallel UnifiedMemoryUnreadable case has its own test but this one does not. The note is user-visible (read by apps/rocm/src/diagnose_model.rs), so it isn't dead code, but it ships unverified.
    Confidence 85 · logic · Fix: add a test with AcceleratorMemory::Unified{..} asserting the note appears for a shared pool and not for Dedicated/Unknown.

  • An e2e assertion meant to catch the BIOS carve-out leaking into the verdict can never fail — tests/e2e-cucumber/tests/e2e/diagnose_steps.rs:4316
    assert_unreadable_shared_pool_claims_no_figure checks that no evidence line contains "offers 4.0 GiB" (the planted 4 GiB carve-out). format_gib special-cases whole-GiB values and renders them without a decimal ("4 GiB", never "4.0 GiB"), so this substring can never appear regardless of whether the bug it's meant to catch is present. The scenario's other assertions (reason, no measured figure) still work; only this specific guard is vacuous.
    Confidence 85 · logic · Fix: match "offers 4 GiB" (no decimal), or use a non-whole-GiB carve-out value in the fixture.

Non-blocking

None

Decisions for the author

  • vram_carveout_gib is currently write-only — non-blocking-improvement
    AcceleratorMemory::Unified's doc comment says the carve-out is "kept only so a report can show the two side by side," but no rendering path currently reads the field (only measured_gib() destructures it away via ..). Either wire it into a report line, or reword the comment to describe it as reserved for a future report rather than something already shown.
  • Scenario diagnose-60 passes identically with the whole feature reverted — tradeoff
    Pre-PR code already returned UnifiedMemoryUnreadable unconditionally for any APU, so a host whose aperture is absent behaves the same before and after this change; diagnose-60 demonstrates two narrower in-PR regressions (no fabricated fallback figure, no reason collapse) rather than that the feature shipped. That proof is fully carried by diagnose-59 instead, so this isn't a coverage hole, just worth knowing diagnose-60 isn't where the feature's existence is pinned.

Positive signals

  • The hand-written Windows DXGI/COM vtable layouts, the IID_IDXGIFactory1 GUID, and the LoadLibraryW-by-absolute-path loading were checked against external references (SDK IDL mirrors, docs.rs, a third-party Go DXGI binding) and matched; COM object release discipline traces clean on every path, including early returns.
  • The exact failure mode this module exists to avoid — crediting an APU with a discrete card's memory when two AMD adapters are present — was introduced, caught, and fixed within this same branch (commit d16e5d3, after the initial 9c3677f) before this review even started, and is now pinned by a test asserting both enumeration orders on both the Linux (DRM) and Windows (DXGI) sides.
  • the_gtt_aperture_never_reaches_a_discrete_card and its multi-GPU variant are genuinely adversarial: they guard the single most dangerous failure mode (summing or substituting the shared pool into a discrete card's VRAM) on both the single- and multi-adapter paths.
  • docs/architecture.md got a correct new-file entry for unified_memory.rs, and docs/testing.md accurately describes the new scenarios and explicitly states the Windows FFI coverage gap, rather than leaving it silent.
  • The e2e test fixture's planted sysfs layout (vendor, mem_info_gtt_total, mem_info_vram_total, KFD properties) was cross-checked field-by-field against the actual Linux readers and matches exactly, with no typo'd paths.

Deployment notes

None required. The change is additive (new enum variant, new function, serde-compatible via the existing snake_case renaming) and needs no migration. It does change rocm diagnose --model's output for APU hosts whose pool is readable: these now report a measured figure and Degraded (never Ready) instead of the previous Undetermined/unified_memory_unreadable — an intended, user-visible behavior change rather than a risk, but worth knowing if anything scripts against the old value.

What this covered

Full diff between 8f086ad9c0fe85e7bab2bd8272dc88638938e4fe and d6a44d2f69b7e65ff48da5d4f1fa63774103b42b (13 files, +1567/-118), read completely by three independent passes grouped by coupling (the Linux/Windows measurement layer and its FFI; the classification/verdict/estimator layer plus its docs; the e2e scenarios and step definitions), each also reading the surrounding callers/callees and cross-checking doc comments against behavior. Supplemented directly: git history of the touched files (confirming the in-branch self-correction in commit d16e5d3), the shipped model catalog (to confirm which device_policy/min_gpu_mem_gb combinations exist today), and the repository's own contribution guide for the subprocess/FFI, platform-gated-test, and architecture-doc rules applied above.

Did not run: a local build/test/clippy pass — relied on the PR's CI, which is green across every listed check, including a native Windows build-and-test job (consistent with the PR's own claim that the Windows FFI path has unit coverage but no integration run against real hardware). No prior review from this account exists at this head. Not reconciled against any other reviewer's comments or discussion on this PR.

@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

The MI300X lane's 8 failures are a serving timeout, not the APU change

This lane ran to completion, so it needs a real answer rather than an infrastructure note:

414 scenarios (402 passed, 12 failed)
Reconciliation: 4 xfail, 2 XPASS (2 flaky), 8 unexpected failure(s).

The eight: bench-load-real-serve, chat-end-to-end-local-model, serve-vllm-inference, runtime-lifecycle-activate-restart-services-succeeds, and four serve-anthropic-*.

One dominant symptom underneath almost all of them:

endpoint http://127.0.0.1:11435/v1/models did not serve model Qwen3.5-0.8B
after 1 attempt(s) of 300s each
… after 2 attempt(s) of 300s each
… after 1 attempt(s) of 90s each

The model never becomes ready. Everything downstream follows from that: no served model means no anthropic endpoint: row, which is exactly how the four serve-anthropic-* scenarios fail, and it equally explains the chat and bench scenarios. They are one failure wearing eight hats, not eight independent regressions.

Why this is unlikely to be this PR. The diff is the APU unified-memory read (host_gpu/unified_memory.rs, the GTT/SharedSystemMemory ceiling) and the diagnose --model classification over it. It does not touch serving, readiness polling, or the Anthropic endpoint. And the host here is an MI300X — a discrete data-center part, not an APU — so the new code path is not even the one being taken.

Two things I checked rather than assumed:

Not re-run, not fixed. A 300-second readiness timeout that elapsed twice is not something a re-run clarifies, and spending MI300X time to watch it fail again would take GPU capacity from the lane-config work currently queued. What would settle it is a completed MI300X run on main; if the same eight fail there, this is a main-side or environment problem and should be tracked separately from this PR.

@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

MI300X: the 8 failures are a broken prewarmed vLLM runtime, not this PR

I previously said the identical 8-failure set on #1036 and #1054 pointed upstream of both but that I had no main-side run to prove it. I now have the root cause from the logs themselves.

Every one of the 8 scenarios fails the same way — the served endpoint never comes up — and the service log tail says why:

RuntimeError: operator torchvision::nms does not exist

raised while vLLM imports torch, inside the prewarmed wheel runtime:

release-wheel-multi-arch-10-1-0-34152a156777d77a
/home/runner/_work/rocm-cli/e2e-prewarm-multi-arch-v2/data/runtimes/wheel/.../python3.14/site-packages/torch/library.py

That is a torch/torchvision ABI mismatch in the prewarm tree on the runners, not anything either PR changes.

Evidence it is environmental:

#1036 #1054
runner mi300x-1 mi300x-0
failing scenarios 8 8 — byte-identical list
torchvision::nms does not exist 11 occurrences 11 occurrences
runtime id release-wheel-multi-arch-10-1-0-34152a156777d77a same

The two PRs have different bases (#1036 sits on the #678 stack, #1054 on main) and ran on different machines, yet fail the same 8 scenarios for the same reason against the same prewarmed runtime id. The failing set is exactly the vLLM-serving scenarios: bench-load-real-serve, chat-end-to-end-local-model, runtime-lifecycle-activate-restart-services-succeeds, serve-vllm-inference, and the four serve-anthropic-* ones.

Main-side baseline. Main's last E2E self-hosted run that reached a verdict is 43015368683 (2026-11-11 20:34). Its MI300X lane reports Reconciliation: 4 xfail, 2 XPASS, 0 unexpected failure(s) — test-clean. The job is still marked failure, but only because the artifact upload died: Failed to CreateArtifact: ... Unexpected token '<'. So main was clean before this runtime state appeared, and has produced no verdict since — every main E2E run today is either cancelled or still queued.

Fix is on the runners, not in these branches. The prewarm tree needs to be rebuilt with a torchvision matching the pinned torch. Nothing I can push to #1036 or #1054 will clear this, and re-running the lane will reproduce it as long as the cached runtime stays.

Both sides appended three scenarios to diagnose.feature and both numbered
them 59/60/61. main's --symptom-file scenarios (#1048) keep 59/60/61 since
they are already merged and referenced elsewhere; this branch's APU memory
scenarios become 62/63/64 in the same relative order, as
tests/feature_naming.rs requires a scenario's number to equal its position
in the file.

The @id: tags are unchanged on both sets -- expectations.toml and the step
bindings key on those, not on the numbers.

References to this branch's renumbered scenarios updated in docs/testing.md
(the ROCMAI-912 section) and in a comment in e2e/diagnose_steps.rs. The
references to main's 59/60/61 in symptom_file_steps.rs, e2e.rs,
docs/architecture.md and the docs/testing.md --symptom-file section are
left alone.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Merged main into this branch (merge only — no review items addressed)

main was merged in as 63283cd. One conflict: tests/e2e-cucumber/features/diagnose.feature, where both sides appended three scenarios and both numbered them 59/60/61.

Renumbering

tests/e2e-cucumber/tests/feature_naming.rs requires a scenario's number to equal its position in the file, so the two sets could not both keep 59/60/61. main's keep their numbers — they are already merged and referenced from several places outside this branch. This branch's move to 62/63/64, in the same relative order, and are placed after main's.

@id: was now from
diagnose-a-piped-paste-is-diagnosed-as-pasted 59 59 main (#1048)
diagnose-a-paste-at-the-terminal-is-explained-and-diagnosed 60 60 main (#1048)
diagnose-an-unreadable-error-file-is-a-usage-error 61 61 main (#1048)
diagnose-model-an-apu-is-measured-on-its-shared-pool 59 62 this PR (ROCMAI-912)
diagnose-model-an-unreadable-shared-pool-is-said-not-guessed 60 63 this PR (ROCMAI-912)
diagnose-model-a-discrete-card-is-never-credited-with-the-aperture 61 64 this PR (ROCMAI-912)

The @id: tags are unchanged on both sets — expectations.toml and the step bindings key on those, not on the numbers, so nothing bound to a scenario had to move. The file now runs 01–64 with no gaps or duplicates. Against origin/main the feature file is a pure addition (54 insertions, 0 deletions): none of main's scenarios was dropped or reworded.

References updated

Only references to this branch's scenarios were renumbered:

  • docs/testing.md — the ROCMAI-912 APU memory pool section: "Scenarios diagnose-59, diagnose-60 and diagnose-61 plant a simulated host" → diagnose-62, diagnose-63, diagnose-64.
  • tests/e2e-cucumber/tests/e2e/diagnose_steps.rs — the comment in assert_unmeasured_machine_is_told_why noting that a readable APU pool never reaches that step: diagnose-59 → diagnose-62.

References to main's 59/60/61 were deliberately left alone, since those scenarios kept their numbers: tests/e2e-cucumber/tests/e2e/symptom_file_steps.rs, tests/e2e-cucumber/tests/e2e.rs, docs/architecture.md, and the --symptom-file section of docs/testing.md (including its cargo xtask e2e -- -n 'diagnose-(59|60|61) ' filter).

Gates (Linux, x86_64)

Gate Result
cargo fmt --all --check pass
cargo clippy --workspace --all-targets -- -D warnings pass
cargo clippy -p e2e-cucumber --test e2e -- -D warnings pass
cargo test -p e2e-cucumber --test feature_naming pass (4/4 — the renumbering proof)
cargo xtask check-architecture-doc pass
cargo xtask check-crate-edges pass
cargo xtask check-module-size pass
cargo test -p rocm-core --lib pass (1512)
cargo test -p rocm --bin rocm pass (1867)

feature_naming is the gate that proves the renumbering: all four of its tests pass, including scenario_names_are_indexed_sequentially_per_feature and scenario_indexes_are_unique_across_the_suite.

One flake on the first -p rocm --bin rocm run: service_launch::spawn::tests::launch_releases_the_leased_port_before_spawning_the_child failed with Address already in use (os error 98) under concurrent load on the shared box. It passed in isolation and on two subsequent full-suite runs.

Review findings are NOT addressed

This is a merge only. No production code was changed, and the three blocking findings from the 2026-11-12 review remain open:

  1. A cpu_only recipe with no GPU-memory minimum is still forced to Degraded with GPU-specific remediation, because the final block in assess() gates only on is_shared_pool().
  2. The new "no dedicated VRAM" note in gguf_fit.rs's estimate_fit has no test.
  3. The e2e assertion !contains("offers 4.0 GiB") is vacuous: format_gib renders whole values without a decimal ("4 GiB"), so the assertion can never fail.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See full report above.

@juhovainio

Copy link
Copy Markdown
Collaborator

(Full text of review #1054 (review) — its body was unintentionally truncated by a tool error when submitting; this comment carries the actual content.)

Review — needs work

Full review of the whole change (13 files, +1567/-118), fanned out across four independent passes grouped by coupling.
Implements the APU-memory-measurement portion of ticket ROCMAI-912 (reading the real GTT/WDDM shared-memory aperture for AMD APUs and capping the resulting verdict at Degraded); the description itself scopes out free-memory measurement, BIOS/gttsize guidance, and SVM-mapping-failure detection as a separate concern. Ticket ROCMAI-912 is an internal identifier not retrievable from here. No other ticket or issue is named in the description.

Blocking

  • A model with no GPU-memory requirement still gets forced to Degraded with GPU-specific remediation on a shared-memory host — crates/rocm-core/src/model_readiness.rs:658-681 (related: requires_gpu() at line 355, the (None, _) match arm around line 618)
    The final block in assess() downgrades any Ready/Degraded verdict to Degraded whenever host.accelerator_memory.is_shared_pool() is true, gated only on that plus the current verdict — never on requires_gpu(recipe) (recipe.device_policy != "cpu_only"), unlike the earlier "no GPU visible" gate a few dozen lines above, which does check it. A recipe with device_policy == "cpu_only" and no declared GPU-memory minimum takes the (None, _) arm, which leaves the verdict at its initial Ready with no GPU-specific evidence; the final block then still fires, appending "this host's integrated GPU has no dedicated VRAM..." evidence and a "close what else is using memory before serving" fix for a model that never touches the GPU. cpu_only is a real, schema-validated device_policy (confirmed in the catalog schema and used elsewhere in the codebase); no recipe in the shipped catalog currently combines it with an absent GPU-memory minimum, so this isn't live today, but it's reachable through the public assess() function (including via an externally supplied recipe index) and untested — no test in the diff exercises a cpu_only recipe on a shared-pool host.
    Confidence 85 · logic · Fix: gate the final block on requires_gpu(recipe) (or required.is_some()) in addition to is_shared_pool().

  • An e2e assertion meant to catch the BIOS carve-out leaking into the verdict can never fail — tests/e2e-cucumber/tests/e2e/diagnose_steps.rs:4393-4401
    assert_unreadable_shared_pool_claims_no_figure asserts no evidence line contains "offers 4.0 GiB" (the fixture's planted 4 GiB carve-out). Verified directly: format_gib (model_readiness.rs:370-376) special-cases whole-GiB values and renders them with no decimal — format_gib(8.0) == "8 GiB", pinned by the function's own unit test — so "offers 4.0 GiB" can never be produced for this value. The guard is doubly unreachable: the actual branch this scenario drives (AcceleratorMemory::UnifiedMemoryUnreadable, around model_readiness.rs:556-564) doesn't use the "...offers {}" template at all, it uses an entirely different sentence. The scenario's other assertions (reason, no measured figure) still work; only this specific guard is vacuous.
    Confidence 85 · mechanical/logic · Fix: match "offers 4 GiB" (no decimal), or use a non-whole-GiB carve-out value in the fixture; either way the guard should target the sentence the UnifiedMemoryUnreadable branch actually emits.

  • The new Linux-side "ambiguous multi-adapter attribution" invariant has no e2e scenario and no stated justification for the gap — absence confirmed across tests/e2e-cucumber/features/diagnose.feature and docs/testing.md
    The repo's own AGENTS.md:92-101 states plainly: "User-observable behavior needs a scenario, not only a unit test... a unit test asserting the internal helper does NOT discharge this; it proves the function, not the behavior" — and requires that where a scenario is genuinely absent, the PR say so. This PR introduces exactly such user-observable behavior on Linux (a host showing more than one AMD DRM card gets no aperture figure at all, for either adapter), covered only by a unit test (two_amd_cards_leave_the_gtt_aperture_unattributable). docs/testing.md's new section is careful to name and justify the Windows-FFI coverage gap in three places, but says nothing about the Linux multi-adapter case, and no scenario in diagnose.feature plants two AMD DRM cards. This is plausibly simulable via the same host-root seam the new scenarios already use.
    Confidence 85 · architectural · Fix: add a scenario planting two AMD DRM cards (mirroring two_amd_adapters_leave_the_shared_budget_unattributable's Windows-side unit test), or add the same kind of explicit justification to docs/testing.md that the Windows gap already gets.

Non-blocking

  • AcceleratorMemory::Unified::vram_carveout_gib is populated but never read anywhere — crates/rocm-core/src/model_readiness.rs:134-138 (doc comment), apps/rocm/src/model_fit.rs:278-285 (construction)
    The doc comment claims it's "kept only so a report can show the two side by side." A repository-wide search for reads of this field (outside its own construction and test-literal sites) returns nothing: no text renderer, and the enum isn't even Serialize, so it can't reach --json either. The claim is false as the code stands today.
    Confidence 85 · architectural · Fix: wire the field into an actual report path, or reword the comment to describe it as reserved for a future report rather than something already shown.

  • gguf_fit::EstimateOutcome::Fits/DoesNotFit doc comments still say "dedicated memory" after this PR routes shared/ceiling pools through the same comparison — crates/rocm-core/src/gguf_fit.rs:176-181
    This PR's own diff in the same file updates the neighboring CannotEstimateReason::UnifiedMemoryUnreadable doc to reflect the new unified-memory handling, but leaves Fits/DoesNotFit describing only "the GPU's dedicated memory," even though estimate_fit now also compares against a Unified host's shared pool (appending a mitigating note rather than changing the enum's own doc).
    Confidence 70 · mechanical · Fix: extend the doc comments to acknowledge the shared/ceiling-pool case, mirroring the sibling update already made in this PR.

Decisions for the author

  • The Windows aperture read runs unconditionally on every rocm diagnose --model call, not only on the APU branch — tradeoff
    host_accelerator_memory (apps/rocm/src/model_fit.rs) calls detect_host_unified_memory_gib() unconditionally; only its result is consulted later, inside classify_accelerator_memory, on the APU branch. On a discrete-GPU Windows host this means a full LoadLibraryW → CreateDXGIFactory1 → EnumAdapters1 loop → GetDesc1×N → release-everything sequence runs and is discarded every time. This is explicitly justified in a comment (keeps the classification a pure function of its arguments; the call site is fully synchronous, confirmed by inspection — no async executor is involved, so this isn't the "blocking call on an async worker" hazard it might otherwise be). Cheap in absolute terms and a reasonable choice for testability, but worth confirming it's the intended cost/risk tradeoff rather than gating the read itself to the APU branch.
  • DXGI enumeration treats a failed GetDesc1 on one adapter as skip-and-continue, but any EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND as stop-the-loop-and-return-what-you-have — tradeoff
    In the narrow case where an earlier-indexed adapter enumerates fine and the APU's own later-indexed adapter trips a non-NOT_FOUND EnumAdapters1 error, the "exactly one AMD adapter" that survives could be the wrong one. This requires an enumeration-failure pattern real EnumAdapters1 essentially never produces outside the natural end-of-list boundary, so it isn't confirmed to fire in practice (hence not scored as a finding) — flagged for the author's awareness rather than as a defect.
  • The ~250-line hand-written Windows DXGI/COM FFI path ships in the same PR as the independently complete, hardware-verified Linux path — tradeoff
    The Windows path is new, unsafe, and has never executed against real hardware — it's type-checked/clippy-checked cross-compiled and unit-tested for its pure logic only, a gap the PR discloses honestly in three places. Both paths are individually sound on inspection, but a reviewer evaluating this as one unit is asked to accept unverified unsafe/FFI code to get the (fully verified) Linux feature on the same timeline. Author's call whether splitting is worthwhile; not a defect in either path.
  • Scenario diagnose-64's Gherkin text claims less than its step definition asserts — non-blocking-improvement
    The scenario's prose ("the memory the answer was reached against is the card's own VRAM alone") doesn't mention a verdict, but its backing step also asserts the overall verdict is ready — true and useful (it rules out the aperture having replaced rather than just not-summed-into VRAM), but undercuts a reader's ability to tell what's checked from the feature file alone, unlike scenario-62 which has an explicit separate verdict step.

Positive signals

  • The hand-written Windows DXGI/COM vtable layouts (IdxgiFactory1Vtbl, IdxgiAdapter1Vtbl, DXGI_ADAPTER_DESC1 field order), IID_IDXGIFactory1, DXGI_ERROR_NOT_FOUND, and DXGI_ADAPTER_FLAG_SOFTWARE were independently checked against the real DXGI/COM ABI by two separate passes and found correct; COM reference-counting (every factory/adapter released exactly once, module freed last) traces clean on every path, including early returns.
  • The "exactly one AMD device may answer" ambiguity rule is implemented independently on Linux (DRM/sysfs) and Windows (DXGI), unit-tested in both adapter/card enumeration orders on each platform, and pinned end-to-end on Linux by the_gtt_aperture_never_reaches_a_discrete_card.
  • docs/testing.md's new section is unusually candid about the coverage gap it does disclose (Windows FFI has unit coverage only), explaining why in three separate places (feature file, step file, this doc) rather than leaving it silent.
  • Real-hardware figures cited in doc comments (a Ryzen 7 4700U measured at 536870912 bytes BIOS carve-out vs. 7895 MiB DXGI/GTT shared budget) are reproduced verbatim in the unit tests, tying the comment's claim to an executable check.
  • The one genuine defect in this feature's own development history — an early version of the adapter-selection logic that picked the larger of two AMD adapters' budgets instead of refusing to attribute, which would have credited an APU with a paired discrete Radeon's budget — was introduced and then caught and fixed within this same branch (commit 9c3677fa → d16e5d32) before this review, and is now exactly what two_amd_adapters_leave_the_shared_budget_unattributable exists to prevent from recurring.

Deployment notes

This changes rocm diagnose --model's output for APU hosts whose pool is readable: they now get a measured figure and Degraded (never Ready), where they previously got Undetermined/unified_memory_unreadable. This is additive (new enum variant, new function) and needs no data migration, but it is a user-visible behavior change — worth flagging if anything downstream scripts against the old verdict value.

What this covered

Full diff between 20914c735464c8b3f60a7bbcee4e0432732af563 (main) and 63283cd326e13e50f8ec56c20bdf3b04e26779ee (the PR's current head, a merge-of-main commit whose merge-base equals main's tip — a plain two-point diff, not a stacked branch) — 13 files, +1567/-118 — read completely across four independent passes grouped by coupling: (1) the Linux/Windows memory-reading layer and its FFI (host_gpu.rs, host_gpu/linux.rs, host_gpu/unified_memory.rs, lib.rs, Cargo.toml), (2) the classification/verdict/estimator layer plus architecture/testing docs (model_readiness.rs, gguf_fit.rs, model_fit.rs, gpu_policy.rs, docs/architecture.md, docs/testing.md), (3) the e2e scenarios and step definitions (diagnose.feature, diagnose_steps.rs), and (4) a source-reading pass covering agent-instruction adherence (AGENTS.md; no CLAUDE.md exists in this repo), git history/blame, prior-review patterns, and doc-comment-vs-behavior consistency. Each pass read the surrounding callers/callees of changed symbols; the orchestrating pass additionally re-verified directly against source: the requires_gpu/shared-pool interaction, format_gib's whole-number rendering and the exact evidence template each branch emits, the absence of any Linux multi-adapter scenario plus the precise AGENTS.md rule it bears on, and the zero-read-sites claim for vram_carveout_gib. CI was read directly rather than assumed: every check at this head reports success except one hardware-specific job (E2E tests (MI300X, current)) still queued at review time — not a failure, so not treated as blocking.

Did not run: a local build/test/clippy pass (relied on the PR's own CI, confirmed green above); no Windows cross-compile toolchain was available to execute the DXGI path, matching the PR's own disclosed gap. This repository's internal ticket tracker is not reachable from here, so ROCMAI-912 is reported as named-but-unretrievable rather than verified against its own acceptance criteria. Two earlier reviews exist on this PR, but both are anchored to earlier, now-superseded commits (not the current head), so nothing was narrowed against them; their text was read only to confirm they were superseded, never passed to the fan-out workers, and every finding above was independently re-derived and verified against the current source rather than carried over. Not reconciled against any reviewer's comments or discussion on this PR.


🤖 Automated by an AI agent

Two conflicts, both from the same cause: main and this branch each appended
to the end of `diagnose.feature` and `diagnose_steps.rs`, so the two sets of
additions collided without overlapping in meaning.

tests/e2e-cucumber/features/diagnose.feature: both sides added a
`diagnose-62`, `-63` and `-64`. Main's six scenarios (62-67, the
pasted-error-contribution and fix-plan ones) keep their numbers; this
branch's three APU-memory scenarios become 68, 69 and 70 and move after
them, so each number still equals its position in the file.

Two references followed the renumber: `diagnose_steps.rs`'s note that an APU
with a readable pool is covered elsewhere, and `docs/testing.md`'s ROCMAI-912
paragraph. The other five references to `diagnose-62/-63/-64` in the tree
point at main's pasted-error scenarios and are deliberately left alone.

tests/e2e-cucumber/tests/e2e/diagnose_steps.rs: two independent step-definition
sections appended at the same place. Both are kept. Each side's final
assertion was left unterminated by the split, sharing one `);`/`}` tail
between them, so the tail is reproduced for each block rather than closing
only the one that happened to come last.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The last block of `assess()` downgraded any Ready/Degraded verdict to
Degraded whenever the host's accelerator memory was a shared pool, gated
on that alone. A recipe with `device_policy = "cpu_only"` and no GPU
memory minimum takes the `(None, _)` arm and arrives there Ready with no
GPU evidence at all, and was then told "this host's integrated GPU has no
dedicated VRAM" and handed "close what else is using memory before
serving" -- a remediation for a device the model never touches.

Gate it on `requires_gpu(recipe)`, as the "no GPU is visible" check a few
dozen lines above already is. `cpu_only` is a schema-validated
device_policy, so this was reachable through the public
`assess_model_readiness`; no shipped recipe pairs it with an absent
minimum today, which is why it was untested. The new unit test
synthesises the pair from a catalog recipe and asserts the GPU-requiring
twin on the same host is still softened, so it cannot pass against a
check that was simply deleted.

Also reword two docs the same review flagged: `Unified::vram_carveout_gib`
now says plainly that nothing reads it (no renderer, and the enum carries
no Serialize, so it cannot reach --json either) and that it is reserved
for a side-by-side report; and `EstimateOutcome::Fits`/`DoesNotFit` now
say the comparison is against a shared pool's ceiling as well as a
discrete card's VRAM, mirroring the `UnifiedMemoryUnreadable` wording
already updated in that file.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…card host

Two gaps the review found in the ROCMAI-912 coverage.

The guard in `assert_unreadable_shared_pool_claims_no_figure` asserted
that no evidence line contained "offers 4.0 GiB". `format_gib` renders a
whole-GiB figure with no decimal at all (its own
`format_gib_renders_a_whole_number_with_no_decimal` pins that), and the
branch this scenario drives does not use the "...and this host offers {}"
template in the first place, so the string was unmatchable twice over and
the assertion could not fail. It now anchors on the sentence the
`UnifiedMemoryUnreadable` branch really emits and derives the forbidden
figure from `CARVE_OUT_BYTES` through `as_rendered`, which mirrors both
of `format_gib`'s branches. `CARVE_OUT_BYTES` becomes 4.5 GiB so the
rendered form cannot collide with a recipe minimum of the same size, and
`carve_out_as_rendered` asserts it stays fractional.

Mutation evidence: with the production evidence line changed to quote the
carve-out, the old literal still passed while the new assertion failed
naming "4.5 GiB".

diagnose-71 covers the Linux multi-adapter invariant, which AGENTS.md
says needs a scenario and had only the unit test
`two_amd_cards_leave_the_gtt_aperture_unattributable`. It is not
reachable through `ROCM_CLI_TEST_HOST_ROOT` alone: the sysfs VRAM
fallback's own older ordinal-ambiguity guard withholds every row once a
second AMD card appears, so the host is classified "GPU memory
unmeasured" before the aperture rule is consulted, and a scenario planted
that way would pass while exercising nothing. The fixture therefore adds
a stub `amd-smi` reporting the single compute device -- the APU-beside-a-
non-enumerated-AMD-adapter shape where this rule is the one that decides
-- with `ROCM_E2E_KFD_DEVICE` (planted twice, since `kfd_readable`
requires the node to open both as written and re-rooted) and `HOME`
pointed at the scenario's own root, because `resolve_amd_smi_binary`
prefers `$HOME/.rocm/runtimes/registry` over PATH and the harness does
not isolate HOME. The step asserts the reason is
`unified_memory_unreadable`, not merely "undetermined", so a fixture that
silently stopped reaching the APU branch fails instead of passing.

Mutation evidence: dropping the `cards == 1` guard in
`linux_drm_amdgpu_gtt_total_bytes_in` turns the verdict into `degraded`
on 60 GiB -- one card's aperture attributed to the host -- and the
scenario goes red.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @juhovainio — all three blocking findings are fixed, plus both non-blocking ones. Commits 8412853 and 59d7638, on top of a main merge (0403a56) that renumbered things; see the note at the end before you re-read the diff.

Finding 1 — a cpu_only recipe forced to Degraded with GPU remediation — fixed

The final block in assess() is now gated on requires_gpu(recipe) as well as is_shared_pool(), matching the "no GPU is visible" gate above it. A cpu_only recipe with no GPU minimum takes the (None, _) arm, stays Ready, and gets no GPU evidence and no fix.

Pinned by model_readiness::tests::a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses. It synthesises the recipe from a catalog one (no shipped recipe pairs cpu_only with an absent minimum, which is why this was untested) and includes the non-vacuity half: a gpu_required twin on the same Unified host must still come back Degraded, so the test cannot pass against a check that was simply deleted.

Finding 2 — the unfalsifiable e2e assertion — fixed, with mutation evidence

You were right on both counts, and so was your note about 4 GiB being ambiguous.

  • The guard now anchors on the sentence the UnifiedMemoryUnreadable branch actually emits ("…the figure for the memory pool its engine allocates from could not be read here"), asserted present first, so the absence check below it is made against a report that exists.
  • The forbidden figure is derived from CARVE_OUT_BYTES via a new as_rendered, which mirrors both branches of format_gib.
  • CARVE_OUT_BYTES is now 4.5 GiB (it is shared with diagnose-68, so the change is consistent across both; diagnose-70 uses its own 8 GiB literal and is untouched). carve_out_as_rendered asserts it stays fractional, so nobody can quietly make it whole again and re-introduce the collision with a 4 GiB recipe minimum.

Mutation test. I changed the production evidence line to quote the carve-out, and temporarily left the old literal guard in place ahead of the new one:

✔  Then the CLI reports that it could not determine the answer
✘  And the reason given is that the shared pool could not be read, and no memory figure is claimed
   Step panicked: the BIOS carve-out (4.5 GiB) is not this host's pool and must not be reported as one

The old "offers 4.0 GiB" assertion passed over that report; the new one caught it. Both reverted.

Finding 3 — the Linux multi-adapter invariant — scenario added (diagnose-71), and the investigation turned up something worth knowing

It cannot be driven through ROCM_CLI_TEST_HOST_ROOT alone. read_drm_vram_usage in gpu_policy.rs has its own, older ordinal-ambiguity guard that withholds every VRAM row as soon as a second AMD card is found. So a two-card host planted through the seam is classified accelerator_memory_unknown before classify_accelerator_memory ever consults the aperture — the scenario would have passed while exercising nothing. (My first draft did exactly that; the undetermined_reason assertion I added is what caught it.)

The host where this rule is the one that decides is an APU beside an AMD adapter ROCm does not enumerate: amd-smi reports one compute device while sysfs shows two AMD DRM cards. diagnose-71 plants that, which needed two things worth flagging:

  • a stub amd-smi plus the ROCM_E2E_KFD_DEVICE seam — planted twice, because kfd_readable requires the node to open both as written and re-rooted under the simulated root (the fake root may make the gate decline, never allow);
  • HOME pointed at the scenario's own root, because resolve_amd_smi_binary looks in $HOME/.rocm/runtimes/registry before PATH and the harness deliberately does not isolate HOME. Without this the scenario passes on GPU-less lanes and reports the runner's real card on every self-hosted GPU lane. (Caught on the MI300X box: the verdict came back ready on 191.9 GiB.)

The step asserts the reason is unified_memory_unreadable, not merely "undetermined", so a fixture that stops reaching the APU branch fails loudly instead of passing quietly.

Mutation test. Dropping the cards == 1 guard in linux_drm_amdgpu_gtt_total_bytes_in:

✘  Then the CLI reports that it could not determine the answer
   left: "degraded"   right: "undetermined"

— card0's 60 GiB aperture attributed to the host. Restored; all four scenarios green.

docs/testing.md's ROCMAI-912 section now names diagnose-71 and explains the amd-smi/HOME requirement and why a host-root-only two-card scenario would be vacuous. The Windows FFI gap keeps its existing justification; nothing changed there.

Non-blocking

  • vram_carveout_gib doc — fixed. Verified by search: the field is written in one place (classify_accelerator_memory) and read nowhere outside tests, no renderer touches it, and AcceleratorMemory carries no Serialize, so it cannot reach --json. Reworded to say plainly that nothing reads it today and that it is reserved for a side-by-side report.
  • EstimateOutcome::Fits/DoesNotFit docs — fixed. Both now say the comparison is against the memory the host's engine allocates from — a discrete card's VRAM or a Unified host's shared-pool ceiling — with the caveat that clearing a ceiling is weaker than clearing a card (Fits) and that over-ceiling is as firm as over-VRAM (DoesNotFit). Mirrors the CannotEstimateReason::UnifiedMemoryUnreadable wording already in that file.

Note on the scenario numbers

main was merged in (0403a56). Both sides had appended scenarios numbered diagnose-62/-63/-64. Main's six (62–67) keep their numbers; this branch's three APU scenarios became 68, 69, 70, since feature_naming requires a scenario's number to equal its position in the file. The @id: tags are unchanged, so expectations.toml and the step bindings are unaffected. Any diagnose-62/-63/-64 reference you find in the tree now points at main's pasted-error scenarios, not at these. The new one is diagnose-71.

Verification

On a Linux devbox (this Mac cannot build rocm-engine-lemonade): cargo fmt --all --check; cargo clippy --workspace --all-targets -D warnings; cargo clippy -p e2e-cucumber --test e2e -D warnings; cargo test -p e2e-cucumber --test feature_naming; cargo xtask check-architecture-doc / check-crate-edges / check-module-size; cargo test -p rocm-core; cargo test -p rocm --bin rocm. All green — process::tests::loopback_lease_blocks_a_second_bind_until_released failed once under parallel load on the shared box and passes alone; it is unrelated and pre-existing. Scenarios diagnose-68/69/70/71 were also run for real against a built e2e-test-hooks binary: 4 passed, 19 steps.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes. The APU shared-pool measurement itself looks sound: the aperture is only consulted on the APU branch, discrete and multi-GPU hosts are pinned by a test, and the scenarios assert the message together with the state. The blockers below are about docs, one remediation, and test/PR-text coverage.

Blocking

  1. User-facing docs still describe the old behavior. README.md (~l.1021 and ~l.1119: "the CLI can't yet read the memory pool... the answer stays undetermined"), docs/manual-testing.md (~l.1715: "an APU answers undetermined by design"), and the --context-size / --kv-cache-type help in apps/rocm/src/cli.rs (~l.148, l.172: "against the dedicated VRAM of the GPU") are all false now for an APU with a readable aperture. Please update them in this change.
  2. The Degraded fix on a Unified host ("close what else is using memory", then verify with rocm diagnose --model) can never clear its condition, because a Unified host is capped at Degraded by construction. No test asserts the fix either. Drop it (evidence only) or reword it and assert it.
  3. The new GGUF estimate behavior (shared-pool note and the Unified arm in gguf_fit.rs) has no unit test and no scenario, and the PR text names neither. A Fits/DoesNotFit outcome against the ceiling is user-observable.
  4. PR body: the mutation table cites diagnose-60/61, but the scenarios added are diagnose-68..71. It also doesn't mention the gguf_fit change or the cpu_only carve-out in assess.

Non-blocking

  • DXGI enumeration can return a partial list as complete (see inline), which defeats the "exactly one AMD adapter" rule.
  • vram_carveout_gib has no reader and its mask filter is untested.
  • diagnose-70 asserts a strict ready, which may depend on runner RAM.
  • "since this PR" in a doc comment.

CI is green at this head.

.to_owned(),
);
if was_ready {
readiness.fix = Some(Fix {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fix tells the user to close apps and re-run rocm diagnose --model, but a Unified host can never leave Degraded, so the advice cannot clear the condition that printed it, and no test asserts fix here. Drop it (evidence-only) or reword and assert it.

// this at best" rather than "this much is waiting for it". Said on the
// estimate itself, because the figure is printed beside the totals with
// nothing else to tell the two kinds of number apart.
if memory.is_shared_pool() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new shared-pool note and the Unified arm have no test (unit or scenario). Please add an estimate_fit test with Unified fits / does-not-fit asserting outcome and note, plus a scenario or a stated reason in the PR text.

pub enum EstimateOutcome {
/// The estimated total is within the GPU's dedicated memory. The total
/// The estimated total is within the memory the host's engine would
/// allocate from: a discrete card's dedicated VRAM, or — since this PR —

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"since this PR" has no meaning once merged; describe the behavior instead.

if hr == DXGI_ERROR_NOT_FOUND {
break;
}
if hr < 0 || adapter.is_null() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A failed EnumAdapters1 other than NOT_FOUND breaks here and returns a partial list as if complete. If the unreached adapter is a second AMD part, amd_shared_system_memory sees exactly one and attributes it, which is the contamination this module says it refuses. Return None on an enumeration error.

let hr = ((*vtbl).get_desc1)(adapter.cast(), &raw mut desc);
(hr >= 0).then_some(desc)
};
if let Some(desc) = read {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An adapter whose GetDesc1 fails is dropped silently, which can likewise hide a second AMD adapter. Treat an unreadable adapter as making the answer unattributable.

/// [`ModelVerdict::Ready`] -- see `assess_model_readiness`.
Unified {
gtt_total_gib: f64,
vram_carveout_gib: Option<f64>,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vram_carveout_gib has no reader (the doc says so). Drop it until a renderer exists.

.iter()
.filter(|usage| usable_indices.is_none_or(|indices| indices.contains(&usage.index)))
.map(|usage| usage.total_mb as f64 / 1024.0)
.reduce(f64::max),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mask filter and reduce(f64::max) feeding the unused carve-out are not covered by a test: removing the filter or swapping max stays green. Test under a mask with two rows, or drop the field.

);
assert_eq!(
model_verdict(&model),
"ready",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strict ready depends on runner system RAM; assert_fitting_model_is_ready in this file tolerates degraded for that reason. Accept ready|degraded and assert the shared-pool evidence is absent instead.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full review of the current head (59d76384) against main (f15f9b4b), 13 files, +1984/-120. This implements the APU-shared-memory-measurement portion of ROCMAI-912 (reading the real GTT/WDDM aperture for AMD APUs and capping the verdict at Degraded); the description itself scopes out free-memory measurement, BIOS/gttsize guidance and SVM-mapping-failure detection as separate work.

The core measurement logic is sound: the aperture is only consulted on the APU branch, the "exactly one AMD adapter" rule is implemented and tested on both Linux and Windows, and the e2e scenarios plant realistic sysfs/DXGI fixtures. However, several things need addressing before merge — largely the same ground @juhovainio's latest review already covers, which I independently re-derived and confirm:

Blocking

  1. The branch currently conflicts with main (crates/rocm-core/Cargo.toml, inline comment) — confirmed locally via git merge-tree: a windows-sys feature-list conflict between this PR's Win32_System_LibraryLoader addition and a Win32_Security_Authorization feature main added since this branch was cut. Needs a rebase with both features kept.
  2. User-facing docs still describe the old "undetermined" behavior. README.md (~1016-1021, ~1119-1121) and docs/manual-testing.md (~1715) both say an APU's memory "can't yet be read" / "answers undetermined by design." That's no longer true for an APU with a readable aperture — these files aren't part of this diff and need updating.
  3. CLI help text makes the same stale claim. apps/rocm/src/cli.rs (the --model flag's doc comment around line 148, and --context-size around line 172) describes the comparison as "against the dedicated VRAM of the GPU," which ignores the new shared-pool/Unified case.
  4. The Degraded remediation text can never be satisfied (inline comment on model_readiness.rs:690) — re-running the suggested verify command reproduces the identical result forever, since only a ceiling is measured, never free memory. Untested.
  5. The new GGUF shared-pool note has zero test coverage (inline comment on gguf_fit.rs:859) — no unit test constructs AcceleratorMemory::Unified in this file, and no scenario exercises it.
  6. The PR description is stale against its own branch. The mutation-evidence table cites diagnose-60/diagnose-61, but a main merge renumbered this branch's scenarios to diagnose-68-71 (60/61 now belong to unrelated, pre-existing scenarios). The description also predates and doesn't mention the later cpu_only-recipe fix or the gguf_fit gap above.

Non-blocking

  • DXGI partial-enumeration handling (inline comment on unified_memory.rs:350) — a real, untested gap, but I agree with the existing call to treat it as non-blocking given how rarely a non-NOT_FOUND EnumAdapters1 error fires in practice and that the Windows path is unexecuted either way.

On CI: the failing MI300X/E2E-consolidated checks are a torch/ROCm version mismatch in the prewarmed runtime image (confirmed via job logs: "torch built for a different ROCm version than the installed SDK"), consistent with the extensive infra investigation already on this thread — not something this diff introduces. The merge conflict above is the actual blocker to mergeability right now.

What this covered: the full diff read across two independent passes (the Linux/Windows measurement + classification layer; the verdict/estimator/e2e layer), plus direct verification of CI logs, a local git merge-tree against current main, and the specific claims in the existing reviews on this PR. Not reconciled against discussion beyond confirming which prior findings still hold at this head.

🤖 Automated by an AI agent

Comment thread crates/rocm-core/Cargo.toml Outdated
# There is no `Win32_Graphics_Dxgi` feature to use instead: windows-sys carries
# no COM interfaces at all, so the three vtable slots that read are declared in
# that module.
windows-sys = { version = "0.61", features = ["Win32_Foundation", "Win32_Security", "Win32_Storage_FileSystem", "Win32_System_LibraryLoader", "Win32_System_Registry", "Win32_System_SystemInformation", "Win32_System_Threading", "Win32_UI_WindowsAndMessaging"] }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking (mergeability): this branch currently conflicts with main right here. git merge-tree shows a content conflict on this windows-sys feature list — main has since added a Win32_Security_Authorization feature (for owner_only_file::owner_only_dacl_mismatch, per the comment above it on main), while this PR's version drops that feature and adds Win32_System_LibraryLoader. The resolution needs both features present: ["Win32_Foundation", "Win32_Security", "Win32_Security_Authorization", "Win32_Storage_FileSystem", "Win32_System_LibraryLoader", "Win32_System_Registry", "Win32_System_SystemInformation", "Win32_System_Threading", "Win32_UI_WindowsAndMessaging"]. Small, but it means the PR isn't mergeable as-is and needs a rebase.

Comment thread crates/rocm-core/src/model_readiness.rs Outdated
);
if was_ready {
readiness.fix = Some(Fix {
summary: "close what else is using memory before serving; the engine, not this \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this remediation can never clear. verify re-runs rocm diagnose --model {model_ref}, but the Degraded verdict it's meant to verify away is forced by this very block whenever is_shared_pool() && requires_gpu(recipe) — unconditionally, since only the aperture ceiling is measured here, never actual free memory. Re-running the verify command after "closing what else is using memory" reproduces the identical Degraded result every time; there's no state the user can reach that clears it. No test in this file asserts on .fix for this path, so the dead-end isn't caught anywhere. Either drop the fix here (evidence-only, matching how the rest of the PR frames this as a hard ceiling) or reword it to not imply the check itself can be satisfied.

// this at best" rather than "this much is waiting for it". Said on the
// estimate itself, because the figure is printed beside the totals with
// nothing else to tell the two kinds of number apart.
if memory.is_shared_pool() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this note has no test coverage. No test in this file's #[cfg(test)] mod tests constructs AcceleratorMemory::Unified (only UnifiedMemoryUnreadable is exercised), and no scenario in diagnose.feature drives a GGUF-backed estimate against a Unified host. This is user-visible text read by diagnose_model.rs, not dead code — deleting this block would currently leave every test green. Please add a unit test asserting the note appears for Unified and not for Dedicated/Unknown.

if hr == DXGI_ERROR_NOT_FOUND {
break;
}
if hr < 0 || adapter.is_null() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking, but worth tracking: an EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND breaks this loop identically to the clean end-of-list case (line 347), silently truncating the adapter list rather than signaling "enumeration was unreliable." If a second real AMD adapter sits past the index where a transient error occurs, this returns exactly one AMD adapter and confidently attributes its budget — the same contamination the single-adapter rule exists to prevent, reached via a failure path rather than a successful one. This was flagged as blocking in an earlier review on this PR and downgraded to non-blocking on the reasoning that EnumAdapters1 essentially only returns NOT_FOUND at the true end of the list in practice — I think that's a reasonable call given the Windows path is still unexecuted on real hardware either way, but since this is the one part of the file with zero test coverage (confirmed: no test drives a non-NOT_FOUND EnumAdapters1 error or a GetDesc1 failure through here), I'd still suggest returning None for the whole call on any error besides DXGI_ERROR_NOT_FOUND, rather than treating it as a complete answer, before this ships to real Windows APU+dGPU hardware.

Resolves the `windows-sys` feature-list conflict in `crates/rocm-core/Cargo.toml`
as a strict union: `main` added `Win32_Security_Authorization` for
`owner_only_file` while this branch added `Win32_System_LibraryLoader` for the
DXGI shared-memory read. windows-sys features are additive, so keeping only one
side would fail to compile on Windows.

Signed-off-by: Roman Sirokov <roman@flowrl.com>
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…r pass

The `Degraded` an APU gets for a GPU recipe is forced by the host, not by
anything the user is doing: the branch fires whenever the pool is shared and
the recipe needs the GPU, because only the aperture is measured and free memory
never is. Its plan carried `verify: rocm diagnose --model <model>`, which
therefore reprints the identical verdict however much the user closes -- a loop
with no exit, read as a check they failed to satisfy rather than one that
cannot be.

Say the limit out loud instead. The plan now names the one thing that does
settle it (`rocm serve`), its note states that this check cannot read how much
of the pool is free so re-asking reports the same thing, and it carries no
`verify` at all -- the renderer omits an empty one, so nothing is promised that
cannot be delivered.

`the_shared_pool_plan_names_no_check_that_can_never_clear` pins it, running the
premise rather than asserting it from the source: the same question asked twice
of the same host gives the same answer. Restoring the old `verify` line fails
that test and nothing else.

Signed-off-by: Roman Sirokov <roman@flowrl.com>
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The note was user-visible text with nothing holding it: no test in `gguf_fit`
constructed `AcceleratorMemory::Unified` at all, so deleting the block outright
left every test in the file green.

`a_shared_pool_is_compared_against_its_ceiling_and_said_to_be_shared` pins the
three things that can break separately -- that a readable pool is estimated
rather than declined, that the figure compared against is the pool and not the
carve-out beside it (the planted carve-out is below the model's own weights, so
a `Fits` proves which was used), and that the note is present for `Unified` and
absent for a dedicated card and an unreadable one.

Mutation evidence: replacing the `is_shared_pool()` guard with `false` fails
this test and no other; replacing it with `true` fails this test too, so the
negative half is load-bearing rather than decoration.

Signed-off-by: Roman Sirokov <roman@flowrl.com>
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
README, `docs/manual-testing.md` and the `--model`/`--context-size` help all
still described the old behaviour: an integrated GPU's pool "can't yet be
read", an APU answers "undetermined by design", the comparison is "against the
dedicated VRAM of the GPU". None of that holds for an APU whose pool is
readable, which is the case this branch added.

Each now says what actually happens: the comparison is made against the shared
memory pool the engine allocates from rather than the BIOS carve-out; the
figure is a ceiling the operating system draws on too, so a model under it
answers `degraded` rather than `ready` and one over it is `blocked` with the
note that freeing memory will not raise it; and `undetermined` is now the
narrower statement that the pool could not be read on this particular host.

Signed-off-by: Roman Sirokov <roman@flowrl.com>
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Replying to review 5935006825. All six blocking items addressed at c8e0b2f; nothing here was already fixed at 59d76384, so each one was a real finding. Taking them in order.

1. Conflict with main — fixed. Confirmed exactly as described (git merge-tree gave one content conflict, crates/rocm-core/Cargo.toml). Resolved by merging origin/main rather than rebasing, keeping both features. The resolution is a strict union, checked as a set operation rather than by eye: 8 features on each side, 9 out, every feature from either side present, nothing invented. Win32_Security_Authorization and Win32_System_LibraryLoader are both there, and a comment on the dependency now says why dropping either breaks a Windows build. mergeStateStatus has gone from conflicted to MERGEABLE.

One correction to the inline comment, which does not change the resolution: on main that windows-sys line carries no preceding comment about owner_only_file — it is bare. The feature list you gave was right.

2. Stale user-facing docs — fixed. README.md and docs/manual-testing.md both now describe what the code does. The undetermined bullet in the README is narrowed to "the shared memory pool its engine allocates from could not be read on this host", and a new paragraph says a readable pool is judged on the pool rather than the carve-out, that this is a ceiling and so answers degraded rather than ready, and that a model over it is blocked with the note that freeing memory will not raise it. The GGUF paragraph's "where the CLI can't read the memory pool" is replaced with the shared-pool comparison and its caveat, and the "compared with the total dedicated VRAM" sentence now names the APU case. In docs/manual-testing.md §12 the "a discrete card: an APU answers undetermined by design" parenthetical is gone, replaced with what an APU actually prints, and the expected-result bullet names the shared pool.

3. CLI help — fixed. Both doc comments in apps/rocm/src/cli.rs now carry the Unified case: --model says the estimate is made against the dedicated VRAM "or, on an integrated GPU, against the shared memory pool its engine allocates from, which is a ceiling the operating system draws on too", and --context-size says the comparison is against that pool "rather than the BIOS carve-out beside it", and in neither case against what other processes have left free. cargo test -p rocm --bin rocm passes, so the advised_commands lint still parses every command these render.

4. The Degraded remediation — fixed, and reworded rather than dropped. Your diagnosis is right and the dead end is exactly as you describe. I did not take either of the two options you offered, though; here is the reasoning, and please push back if you disagree.

Dropping the fix outright matches house precedent — assess already says "An APU's unreadable pool has no command that changes the answer, and gets none" — but it leaves a user who asked "can I run this?" with a softened verdict and no next step at all. There is a next step; it is just not this check. So the plan now:

  • carries no verify (the renderer omits an empty one, so nothing is promised that cannot be delivered),
  • names the one command that genuinely settles it, rocm serve <model>,
  • and states the limit out loud in a note rather than leaving the reader to discover it by re-running: "this check compared the model against the pool's ceiling and cannot read how much of the pool is free, so asking it again reports the same thing however much you close; the load is what settles it."

The summary is now "nothing here is broken: serve it and let the engine answer, having closed what else is using memory" — the freeing advice survives, because it does improve the odds, but it is no longer framed as something this check will confirm.

Pinned by the_shared_pool_plan_names_no_check_that_can_never_clear. It runs the premise rather than asserting it from the source: it asks the same question twice of the same host and asserts the verdict did not change, so the test fails loudly if that assumption ever stops holding and it is guarding nothing. It then asserts the verify is empty, that "diagnose" has not come back by another route, that a rocm serve command is offered, and that the note explains why there is nothing to re-run — with a dedicated-card non-vacuity half that is Ready with no plan at all. Mutation evidence: putting verify: format!("rocm diagnose --model {model_ref}") back fails this test and no other in the suite.

5. The GGUF shared-pool note — fixed. Confirmed: no test in gguf_fit.rs constructed AcceleratorMemory::Unified. a_shared_pool_is_compared_against_its_ceiling_and_said_to_be_shared pins the three things that can break separately — that a readable pool is estimated rather than declined (it must not fall into the measured_gib() == None arm), that the figure compared against is the pool and not the carve-out beside it, and that the note is present for Unified and absent for Dedicated and Unknown. The second is made falsifiable by planting a carve-out below the model's own weights, so a Fits can only have come from the pool.

Mutation evidence: replacing the is_shared_pool() guard with false — your "deleting this block would currently leave every test green" — fails this test and no other. Replacing it with true also fails it, so the negative half is load-bearing rather than decoration.

I did the unit test you asked for and did not add a diagnose.feature scenario for it. The branch is a pure function of AcceleratorMemory and a GGUF header, and an e2e scenario would need a planted GGUF plus a host fixture to reach the same assertion through two more layers. Say the word if you want the scenario anyway and I will add it.

6. PR description — fixed. Confirmed stale: diagnose-60/61 are now "Pasting the error at the terminal is explained" and "An error file that cannot be read… is refused", nothing to do with this branch. The mutation table now cites diagnose-68–71 by their real subjects, and gains rows for the cpu_only fix and the gguf_fit gap — each with the mutation that catches it and whether it was the only failure in the suite. The body also now describes the cpu_only-recipe gate, the remediation rework above, the merge resolution, and the DXGI gap below. The three test names the old table cited all still exist and were re-checked; only the scenario numbers were wrong.

Non-blocking — DXGI partial enumeration: acknowledged, not changed. I agree with your reasoning and with the earlier downgrade, and I have not touched it. It is now written into the PR description's non-goals rather than living only in review history, stated as what it is: a non-NOT_FOUND EnumAdapters1 error ends the loop like a clean end-of-list, so a second AMD adapter past a transient error would be missed and the first one's budget confidently attributed. Returning None on any error but NOT_FOUND is the right change and I would rather make it when the Windows path can be executed against it, since it is untestable here and would ship unexercised either way.

Gates. All green on a Linux ROCm box (MI300X, gfx942), each run on its own: fmt --all --check; clippy --workspace --all-targets -D warnings; clippy -p e2e-cucumber --test e2e -D warnings; test -p e2e-cucumber --test feature_naming; xtask check-architecture-doc; xtask check-crate-edges; xtask check-module-size; test -p rocm-core (1584 passed); test -p rocm --bin rocm (1904 passed).

Not verified. The windows-sys feature change cannot be compile-checked in this environment — nothing here builds the MSVC target — which is why the union was verified as a set operation and is called out in the description. And I agree with you on CI: the MI300X and consolidated-E2E failures are the torch/ROCm image mismatch reproducing on every PR in the repo, not this diff.

Not re-requesting review; leaving that to the author.

@r0x0r
r0x0r requested a review from juhovainio November 17, 2026 08:42

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — needs work

Full review of the whole change (base 9101286a → head c8e0b2fb, 16 files, +2206/-135), read completely via three independent fan-out passes. Implements the APU-shared-memory-measurement portion of ROCMAI-912 (reading the real GTT/WDDM shared-memory aperture for AMD APUs and capping the verdict at Degraded); the description itself scopes out free-memory measurement, BIOS/gttsize guidance, and SVM-mapping-failure detection as separate work, and that boundary matches what the diff actually contains. Ticket ROCMAI-912 named but not retrievable from here (no tracker access).

This is a fresh review at the current head, after an earlier round reviewed an older commit (59d76384) on this same PR. Several commits since then (be444821, 71545456, 81788fa5, c8e0b2fb) directly answer that round's blocking points, and I independently re-verified each is actually fixed rather than taking that on faith: the windows-sys/Cargo.toml merge conflict is resolved (clean git merge-tree against current origin/main); the stale "undetermined by design" / "dedicated VRAM" claims in README.md, docs/manual-testing.md, and the --model/--context-size help in apps/rocm/src/cli.rs are gone, replaced with accurate shared-pool language; the Degraded plan's dead-end verify was removed (no verify printed, matched by a new test); and the GGUF estimator's shared-pool note now has a unit test checking both presence (Unified) and absence (Dedicated/Unknown). The four findings below are new ground, not repeats of what already landed.

Blocking

  • The shared-pool "forces Degraded" gate checks requires_gpu(recipe) but not whether a GPU-memory comparison actually happened, so a gpu_required/gpu_preferred recipe with no declared min_gpu_mem_gb is downgraded from Ready on fabricated evidence — crates/rocm-core/src/model_readiness.rs:673-679 (gate), :361-363 (requires_gpu), :1872 (the one new test, which doesn't cover this shape)
    requires_gpu is recipe.device_policy != "cpu_only". A recipe with device_policy: "gpu_required" and min_gpu_mem_gb: None takes the (None, _) match arm, leaves the verdict at Ready, and then this gate still fires because it only checks requires_gpu, not required.is_some(). The shipped catalog has exactly this shape today — openai/gpt-oss-20b, openai/gpt-oss-120b, zai-org/GLM-4.7, zai-org/GLM-5.2, MiniMaxAI/MiniMax-M3, meta-llama/Llama-3.1-405B-Instruct, deepseek-ai/DeepSeek-V4-Flash-0731 (verified directly against crates/rocm-core/src/model_catalog.json) — so on an APU host with a readable shared pool, each is silently stamped Degraded with the invented "this host's integrated GPU has no dedicated VRAM" evidence and a remediation for a comparison that was never made. Same bug class the PR's own cpu_only fix (commit 8412853f) addressed; left open for the broader "no declared minimum" case.
    Confidence 85 · logic · Fix: gate on required.is_some() in addition to/instead of requires_gpu(recipe).

  • The cpu_only-recipe-not-degraded-by-a-shared-pool behavior has no e2e scenario, only a unit test — crates/rocm-core/src/model_readiness.rs:1872, tests/e2e-cucumber/features/diagnose.feature (absent)
    This is new, user-observable CLI behavior (whether the verdict is downgraded to degraded). AGENTS.md §3 is explicit that a unit test does not discharge this for user-observable CLI behavior, and requires a Gherkin scenario or a stated reason for its absence in the PR text. Neither exists: no cpu_only match anywhere under tests/e2e-cucumber/, and the PR body's mutation table cites only the unit test's name for this row with no scenario id and no justification.
    Confidence 78 · architectural (process/coverage) · Fix: add a diagnose.feature scenario covering this, or state in the PR text why one is impractical.

  • The GGUF estimator's "shared pool is a ceiling, not free memory" note has no e2e scenario, only a unit test — crates/rocm-core/src/gguf_fit.rs:1677, tests/e2e-cucumber/features/diagnose.feature (absent)
    This note is printed, user-visible text for a shared-pool host and needs the same AGENTS.md §3 coverage. A prior review round already flagged this as "no unit test and no scenario"; the unit test was added since, but the scenario side was not — this note's wording never appears in diagnose.feature/diagnose_steps.rs (only the unrelated curated-recipe evidence text, a different code path, is scenario-covered).
    Confidence 78 · architectural (process/coverage) · Fix: add a scenario driving a GGUF-outside-catalog estimate on a shared-pool host and asserting the note appears.

  • skills/rocm-doctor/reference.md's description of available_gib is now stale for the case this PR adds — skills/rocm-doctor/reference.md:81 (not part of this diff; could not be posted as an inline PR comment because GitHub only anchors review comments to lines inside the diff)
    "available_gib is the total dedicated VRAM... not what is free" was accurate enough pre-PR, when an APU essentially never populated this field. This PR makes it populate with the shared-pool ceiling for a readable APU — which, per this PR's own evidence text elsewhere, explicitly is not "dedicated VRAM". Every other surface making this same claim (README.md, docs/manual-testing.md, apps/rocm/src/cli.rs) was correctly updated in this PR; this assistant-facing contract file, which feeds the rocm-doctor LLM skill as ground truth, was missed. AGENTS.md §5 requires grepping the same claim's wording across all such surfaces when a behavior claim changes.
    Confidence 78 · logic (doc accuracy) · Fix: add the same shared-pool/ceiling caveat here that README.md/cli.rs now carry.

Non-blocking

None.

Decisions for the author

  • DXGI partial enumeration (a non-NOT_FOUND EnumAdapters1 error, or a GetDesc1 failure on an enumerated adapter) is treated as if the adapter list were complete, which could in principle defeat the "exactly one AMD adapter" safety invariant — answered-by-intent
    PR body, Scope section: "An EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND ends the adapter loop the same way the clean end of the list does, so a second AMD adapter sitting past a transient error would be missed... NOT_FOUND is what that call returns at the true end of the list in practice, and the Windows path is unexecuted either way, so this is left as known and untested rather than guessed at." The code's own doc comment makes the same call for the GetDesc1 case: "Never fails as a whole: an adapter whose description cannot be read is skipped, because one uncooperative adapter must not hide the budget of the one the question is about." Would otherwise have been a blocking architectural finding (it is a real, confirmed gap in the module's central safety invariant under a low-probability but real failure mode); retired because the author already named and reasoned about it. Confirm this remains the intended tradeoff before the Windows path gets real execution coverage.
  • vram_carveout_gib is currently write-only — non-blocking-improvement
    The field's own doc comment says it is "kept only so a report can show the two side by side," and nothing renders or serializes it today (AcceleratorMemory has no Serialize, and every match on Unified destructures it away via ..). The comment documents the gap with a reason, so this isn't blocking dead code, but either wire it into a report line now or make the "reserved for later" framing more prominent so the next reader doesn't mistake it for an oversight.
  • ~250 lines of hand-written Windows DXGI/COM FFI, entirely unexecuted by CI (type/clippy-checked cross-compiled only), ships in the same PR as the Linux path that CI does exercise — tradeoff
    The PR body already raises and answers this itself ("the alternative was taking on the windows crate for three struct fields"), and the vtable layouts/GUID/release discipline were checked against external references with no mismatch found. Not a defect, but worth the author/maintainer explicitly confirming the scope call, since this is real, un-exercised FFI shipping without integration coverage.

Positive signals

  • The "exactly one AMD adapter" invariant is applied, documented, and tested consistently on both the Linux (mem_info_gtt_total/sysfs) and Windows (DXGI) sides, each citing the other's test by name in its own doc comments — a genuinely cross-platform-consistent safety design, not two independently-drifted implementations.
  • COM object lifetime discipline in the new unified_memory.rs (factory/adapter Release() on every path, including early returns and GetDesc1 failure) was traced and found clean with no leaks or double-releases.
  • docs/architecture.md got specific, accurate entries for the new host_gpu/unified_memory.rs file and the updated model_readiness.rs, not boilerplate — each correctly states the ceiling/ never-Ready behavior.
  • User-facing prose updates (README.md, docs/manual-testing.md, apps/rocm/src/cli.rs help text) are thorough and consistent with the new shared-pool/ceiling language everywhere I checked except the one doc gap noted above.

Deployment notes

This changes rocm diagnose --model's observable output for APU hosts whose shared pool is readable: previously undetermined/unified_memory_unreadable, now a measured figure with the verdict capped at degraded (never ready). Anything scripting against the old undetermined value for these hosts will see different output. No migration needed; the change is additive (new enum variant, serde-compatible).

What this covered

Full diff between 9101286acfe1d071b2d1ae43739f937593589ad7 and c8e0b2fb43e09ff80858f2a6047006966e5686bf (16 files, +2206/-135), read completely across three independent fan-out passes grouped by coupling (the Linux/Windows measurement layer and its FFI; the classification/verdict/estimator layer plus its user-facing docs; the e2e scenarios and step definitions), each reading the full diff plus the surrounding callers/callees and doc comments. Supplemented directly, by me: the shipped model catalog JSON (to confirm which device_policy/min_gpu_mem_gb combinations exist today), commit history and message quality (12 commits, descriptive, no AI-attribution footers), a local git merge-tree against current origin/main (clean — the previously-flagged conflict is resolved), and docs/architecture.md/skills/rocm-doctor/reference.md outside the diff for impact analysis.

CI: 45 of 46 checks completed and green at this head; E2E tests (MI300X, current) was still queued at review time (a self-hosted/hardware-gated lane) — noted as unresolved rather than assumed fine, not treated as a failure.

Did not run: a local build/test/clippy pass (relied on CI). History and prior-changes were not run as fully separate dedicated passes beyond what's described above.

Disclosure: this PR's own claim-check step (confirming no other PENDING review existed at this head) required a get_reviews call that returned the full text of four prior reviews before my blind fan-out began; my three review passes themselves were never shown that content (each received only a file scope and explicit instructions not to read the PR description or discussion), but I did see it. Every finding above was independently verified against the code (and, for Finding 1, the shipped catalog) by me before inclusion, not carried over from that content. Reconciliation: the DXGI retirement above corroborates what a prior round already called non-blocking/accepted; the GGUF-scenario finding partially overlaps a prior round's point (the unit-test half of that gap has since been fixed, the scenario half has not); the other three findings are new.


🤖 Automated by an AI agent

// no GPU minimum takes the `(None, _)` arm and reaches here `Ready` with no
// GPU figure at all).
if host.accelerator_memory.is_shared_pool()
&& requires_gpu(recipe)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — the shared-pool "forces Degraded" gate checks requires_gpu(recipe) but not whether a GPU-memory comparison actually happened, so a recipe with device_policy other than cpu_only but no declared min_gpu_mem_gb is downgraded from Ready on evidence that was never produced.

requires_gpu (line 361-363) is recipe.device_policy != "cpu_only". A recipe with device_policy: "gpu_required" and min_gpu_mem_gb: None takes the (None, _) arm at line ~618 ("this recipe declares no GPU memory minimum"), leaves readiness.verdict at its initial Ready, and then this gate still fires because it only checks requires_gpu, not required.is_some(). The shipped catalog has exactly this shape today — openai/gpt-oss-20b, openai/gpt-oss-120b, zai-org/GLM-4.7, zai-org/GLM-5.2, MiniMaxAI/MiniMax-M3, meta-llama/Llama-3.1-405B-Instruct, deepseek-ai/DeepSeek-V4-Flash-0731 (verified directly against crates/rocm-core/src/model_catalog.json) — so on an APU host with a readable shared pool, each of these is silently stamped Degraded with the fabricated "this host's integrated GPU has no dedicated VRAM" evidence and a "free memory, then rocm serve" fix for a comparison that was never made. This is the same bug class the PR's own cpu_only fix (commit 8412853) addressed, left open for the broader "no declared minimum" case. The new test a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses (line 1872) only constructs cpu_only vs. gpu_required-with-a-minimum recipes, so this gap is untested.

Confidence 85 · logic · Fix: gate the block on required.is_some() (or equivalently, only fire when the earlier match actually produced a GPU-memory comparison) in addition to/instead of requires_gpu(recipe).

/// absent minimum today, which is why the recipe is synthesised from a real
/// one rather than looked up -- and why this was untested.
#[test]
fn a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — the cpu_only-recipe-not-degraded-by-a-shared-pool behavior has no e2e scenario, only this unit test.

This is new, user-observable CLI behavior: whether rocm diagnose --model reports ready/blocked or forces degraded for a cpu_only recipe on an APU host differs by this fix. AGENTS.md §3 is explicit that "a unit test asserting the internal helper does NOT discharge this" for user-observable CLI behavior, and requires either a Gherkin scenario in tests/e2e-cucumber/features/ or a stated reason in the PR text for its absence. Neither exists here — grep -rn "cpu_only" over tests/e2e-cucumber/ at this head returns nothing, and the PR body's mutation-evidence table cites only this unit test's name for this row, with no scenario id and no stated justification for the gap.

Confidence 78 · architectural (process/coverage) · Fix: add a diagnose.feature scenario driving rocm diagnose --model against a cpu_only (or no-GPU-minimum) recipe on a host with a readable shared pool, asserting the verdict is not downgraded to degraded — or state in the PR text why one is impractical here.

/// before this one existed, and a test that only asserted presence would
/// still pass against a block that pushed the note unconditionally.
#[test]
fn a_shared_pool_is_compared_against_its_ceiling_and_said_to_be_shared() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — the GGUF estimator's "shared pool is a ceiling, not free memory" note has no e2e scenario, only this unit test.

This note is printed, user-visible text in rocm diagnose --model <gguf> output for a shared-pool host, so per AGENTS.md §3 it needs Gherkin coverage (or a stated reason it's impractical), not just this unit test. A prior review round on this PR already flagged this exact gap as "no unit test and no scenario"; the unit test was since added (this one), but the e2e scenario side was not — grep for this note's wording ("shares with the operating system", "no dedicated VRAM") across tests/e2e-cucumber/features/diagnose.feature and diagnose_steps.rs only matches the curated-recipe evidence text in model_readiness.rs (a different code path, already scenario-covered), never this estimator-specific note.

Confidence 78 · architectural (process/coverage) · Fix: add a scenario driving a GGUF-outside-catalog estimate against a shared-pool APU host and asserting this note appears in the output.

Main landed the fix-step timeout work (#1062), which took `diagnose-68`
through `diagnose-70`. This branch's four APU-memory scenarios move to
`diagnose-71` through `diagnose-74`, so each number still equals its position.

Five references follow them: the ROCMAI-912 scenario list and the
`diagnose-74` note in `docs/testing.md`, and three comments in
`diagnose_steps.rs`. The other `diagnose-68`/`-69`/`-70` references in those
same files -- the fix-4 helper, the one-minute timeout, fix-2 and fix-9 --
belong to main's new scenarios and are left alone.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…ed against

Review finding: the shared-pool gate asked `requires_gpu(recipe)` but not
whether a GPU-memory comparison had actually been made. `requires_gpu` is
only `device_policy != "cpu_only"`, so a `gpu_required` recipe that declares
no `min_gpu_mem_gb` passes it, takes the `(None, _)` arm above, and arrives
`Ready` with no GPU figure in the report -- and was then stamped `Degraded`
with "this host's integrated GPU has no dedicated VRAM" and a remediation for
a comparison nothing performed.

Live, not latent: seven recipes in the shipped catalog have exactly that
shape, so on an APU with a readable pool every one of them was mis-reported.
The earlier `cpu_only` fix closed the neighbouring case and left this one.

The gate now also requires `required.is_some()`. The two terms answer
different questions -- "does it use the GPU" and "did we weigh it against the
pool" -- and only the second licenses the verdict.

Pinned by a test driven off the shipped catalog rather than a synthetic
recipe, with an explicit premise assertion so it fails loudly if no such
recipe remains rather than passing vacuously. Mutation-checked: dropping
`required.is_some()` fails it, naming a real catalog model and the evidence
line it was given.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — this round found something live, and I verified it before fixing it.

Blocking 1 — gpu_required with no declared minimum is softened on evidence nothing produced — fixed (60c29d572).

You were right, including the count. I checked model_catalog.json directly rather than taking it on trust: exactly seven shipped recipes carry device_policy other than cpu_only with min_gpu_mem_gb: None. On an APU with a readable pool, each was stamped Degraded with "this host's integrated GPU has no dedicated VRAM" and handed a remediation for a comparison that never happened.

The irony is that the code comment immediately above the gate already described this shape — "a recipe with no GPU minimum takes the (None, _) arm and reaches here Ready with no GPU figure at all" — as the reason for the requires_gpu gate, while the gate itself didn't exclude it. The earlier cpu_only fix closed the neighbouring case and left this one.

The gate now also requires required.is_some(). The two terms answer different questions — does it use the GPU and did we weigh it against the pool — and only the second licenses the verdict.

Pinned by a_gpu_recipe_with_no_declared_minimum_is_not_degraded_by_a_pool_it_was_never_weighed_against, driven off the shipped catalog rather than a synthetic recipe, since the point is that the shape is live; it asserts the premise first, so it fails loudly if no such recipe remains rather than passing vacuously. Mutation evidence — dropping required.is_some():

deepseek-ai/DeepSeek-V4-Flash-0731: the report claimed the pool was too small for
a model it never sized: [..., "this recipe declares no GPU memory minimum (device
policy `gpu_required`)", "this host's integrated GPU has no dedicated VRAM: ..."]

Those two evidence lines sitting in one report is the defect stated plainly.

Blocking 4 — skills/rocm-doctor/reference.md:81's available_gib description — open. You are right that it is now stale and right that it is the one surface I missed when the others were updated; it feeds the skill as ground truth, so it matters more than an ordinary doc. Not fixed in this push only because I wanted the live mis-report out first. Taking it next.

Blocking 2 and 3 — no e2e scenario for the cpu_only behaviour or the GGUF shared-pool note — open, and I am not going to pretend otherwise. Both are new user-observable output covered only by unit tests, and AGENTS.md §3 is explicit that a unit test does not discharge that. The honest position is that these need scenarios rather than a justification, since nothing makes them impractical to simulate — the existing APU scenarios already plant a host through the same seam. I would rather add them properly than write a reason they are absent.

Decisions: the DXGI partial-enumeration tradeoff and the unexecuted Windows FFI scope call both stand as the PR body describes them — confirmed intended. vram_carveout_gib is still write-only; I will make the "reserved" framing more prominent rather than wiring a report line in this PR.

So: one blocking item fixed with mutation evidence, three still open. Leaving CHANGES_REQUESTED standing.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — needs work

Full review of the whole change (16 files, +2262/-135, base origin/main → head 60c29d57).

Implements the GTT/DXGI shared-memory-pool measurement portion of ROCMAI-912 (reading the real pool an AMD APU's engine allocates from and capping the verdict at Degraded). Ticket ROCMAI-912 retrieved directly: its own comments confirm the BIOS carve-out/amdgpu.gttsize recommendation and SVM mapping failed detection are deliberately deferred as separate work, and record a further Windows-adapter-disambiguation idea that was considered and explicitly rejected for now — both match what this diff actually contains, and the PR does not claim to close the ticket (correct: it remains "In Progress").

This is a fresh review at the current head, after four prior rounds on this PR (three by this account) found and closed several real issues across earlier commits — a merge conflict, stale "undetermined"/"dedicated VRAM" docs, a dead-end remediation, a vacuous e2e assertion, and (most recently, at c8e0b2fb) a gap in the shared-pool Degraded gate. The branch's very last commit (60c29d57, titled as a review finding) closes that last gap: the gate now requires required.is_some() in addition to requires_gpu(recipe), verified directly against the shipped catalog (exactly the 7 recipes the commit message names have this shape). I independently re-derived this rather than taking the commit message on faith. Three findings from that last round are still open at this head (confirmed independently, not carried over); one new defect was found that four prior rounds missed.

Blocking

  • A GGUF-outside-the-catalog estimate on an APU prints self-contradictory text in the same report — crates/rocm-core/src/gguf_fit.rs:1151 (evidence_lines, unchanged by this PR) vs :859-863 (the new shared-pool note)
    evidence_lines unconditionally renders "the GPU with the most dedicated VRAM that ROCm can see has {N} MiB..." whenever estimate.available_gib.is_some(), which is also true for the new Unified case. model_readiness.rs:992 extends readiness.evidence with this text, and render_model_readiness_text prints evidence bullets before estimate.notes in the same report. So on a shared-pool host the output reads "...has 24576 MiB of dedicated VRAM..." followed a few lines later by "...has no dedicated VRAM...". No test exercises evidence_lines with AcceleratorMemory::Unified (the new test only checks .notes).
    Confidence 90 · logic · Fix: branch evidence_lines's wording on memory.is_shared_pool(), the same way the note itself is gated.

  • The GGUF estimator's shared-pool note has no Gherkin scenario, only a unit test — crates/rocm-core/src/gguf_fit.rs:859 (behavior), tests/e2e-cucumber/features/diagnose.feature (absent)
    This is new, user-observable, serialized output (ContextEstimate.notes, reaches --json). AGENTS.md §3 is explicit a unit test does not discharge this. Verified independently: zero occurrences of this note's wording anywhere under tests/e2e-cucumber/; no existing scenario combines a GGUF model with an APU host. A prior round flagged "no unit test and no scenario" — the unit-test half has since been added, the scenario half has not.
    Confidence 85 · architectural (process/coverage) · Fix: add a scenario, or state in the PR text why one is impractical.

  • The cpu_only-recipe immunity to the shared-pool downgrade has no Gherkin scenario, only a unit test — crates/rocm-core/src/model_readiness.rs:681 (behavior), tests/e2e-cucumber/features/diagnose.feature (absent)
    Covered only by a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses. Verified independently: zero occurrences of "cpu_only"/"cpu-only" under tests/e2e-cucumber/; the PR body's mutation table names only this unit test for this row, with no scenario id and no stated reason for the gap.
    Confidence 85 · architectural (process/coverage) · Fix: add a scenario, or state in the PR text why one is impractical.

  • skills/rocm-doctor/reference.md's description of available_gib is stale for the case this PR adds — skills/rocm-doctor/reference.md:81 (not part of this diff; could not be posted as an inline comment for that reason)
    "available_gib is the total dedicated VRAM of the GPU with the most of it that ROCm can see, not what is free" was accurate when an APU essentially never populated this field. This PR makes it populate with the shared-pool ceiling for a readable APU, which per this PR's own evidence text elsewhere explicitly is not dedicated VRAM. Every other surface making this same claim (README.md, docs/manual-testing.md, apps/rocm/src/cli.rs) was correctly updated in this PR; this assistant-facing contract file — which feeds the rocm-doctor skill as ground truth — was missed. Confirmed directly: the lines between this file's state at the previous review round and now are unrelated (an unconnected exit-code-table edit), so this specific sentence is still untouched and still stale.
    Confidence 85 · logic (doc accuracy) · Fix: add the same shared-pool/ceiling caveat here that README.md/cli.rs now carry.

Non-blocking

  • Wrong scenario cross-reference in a feature-file comment — tests/e2e-cucumber/features/diagnose.feature:1126 (inline)
  • docs/architecture.md's gguf_fit.rs entry (line ~314) is stale, still describing the comparison as only "against VRAM" (inline, anchored nearby in the same hunk)

Decisions for the author

  • DXGI partial enumeration (a non-NOT_FOUND EnumAdapters1 error, or a GetDesc1 failure) is treated as a complete adapter list, which could in principle defeat the "exactly one AMD adapter" invariant — answered-by-intent
    PR body, Scope section: "An EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND ends the adapter loop the same way the clean end of the list does... NOT_FOUND is what that call returns at the true end of the list in practice, and the Windows path is unexecuted either way, so this is left as known and untested rather than guessed at." Independently re-traced in unified_memory.rs's collect (confidence 65-85 depending on how the gap is weighted) and confirmed real; retired here because the author already named and reasoned about it. Confirm this remains the intended tradeoff before the Windows path gets real execution coverage.
  • vram_carveout_gib is currently write-only — non-blocking-improvement
    Its own doc comment says it's "kept only so a report can show the two side by side," and nothing renders it today. Not blocking dead code since the gap is documented with a reason, but worth wiring in or re-framing as "reserved."
  • requires_gpu() treats gpu_preferred identically to gpu_required — tradeoff
    If "preferred" is ever meant to allow a CPU-fallback path, the shared-pool ceiling would be irrelevant there; if GPU presence is effectively mandatory for both policies today, the logic is fine and this is just a naming note.

Positive signals

  • The in-branch self-correction on required.is_some() (above) is real engineering working as intended: caught by review, fixed with a test driven off the live catalog rather than a synthetic fixture.
  • cpu_only exclusion is tested with genuine non-vacuity: a cpu_only recipe and a gpu_required sibling are asserted on the same host in the same test.
  • The dead-end Degraded remediation a prior round flagged is gone and stays gone: no verify is offered when the downgrade is pool-forced, pinned by the_shared_pool_plan_names_no_check_that_can_never_clear.
  • Windows DXGI/COM vtable layouts, the DXGI_ADAPTER_DESC1 struct layout, and COM reference-counting discipline (every AddRef/CreateDXGIFactory1 success matched by a Release, including early-return paths) were traced and found correct; SharedSystemMemory is read as a pointer-sized SIZE_T, correctly avoiding the 32-bit WMI fields that saturate at 4 GiB.
  • The Linux GTT reader is a single-pass, no-early-exit design that is structurally immune to the Windows side's partial-enumeration gap (see Decisions above) — a more robust design for the same invariant.
  • format_gib's non-vacuity fix from a prior round is real and well-executed: the e2e fixture's carve-out value is deliberately non-whole-GiB so the "no fabricated figure" assertion can't vacuously pass.
  • Commit messages are clean and accurate at branch head, with proper DCO trailers and no AI-attribution footers.

Deployment notes

Changes rocm diagnose --model's observable output for APU hosts whose shared pool is readable: previously undetermined/unified_memory_unreadable, now a measured figure with the verdict capped at degraded (never ready). Anything scripting against the old undetermined value for these hosts will see different output. No migration needed; the change is additive (new enum variant, serde-compatible).

What this covered

Full diff origin/main...60c29d57 (16 files, +2262/-135), read completely across seven independent blind fan-out passes grouped by coupling: (1) the Linux/Windows measurement layer and its FFI (host_gpu.rs, host_gpu/linux.rs, host_gpu/unified_memory.rs, lib.rs, Cargo.toml, gpu_policy.rs); (2) the verdict/classification layer plus CLI help text (model_readiness.rs, cli.rs); (3) the GGUF/model-fit estimator (model_fit.rs, gguf_fit.rs); (4) user-facing docs (README.md, docs/architecture.md, docs/manual-testing.md, docs/testing.md), cross-checked against code via a repo-wide stale-claim grep sweep; (5) e2e scenarios and step definitions; (6) an AGENTS.md compliance sweep (module size, architecture-doc entries, FFI/subprocess checklist, scenario-coverage requirement, doc-claim accuracy, commit-message quality); (7) commit history (self-correction patterns, pre-existing bug-class recurrence, merge-commit sanity). Each pass read its files completely plus callers/callees; none were shown the PR description, ticket, or other reviews' content. I supplemented directly: the shipped model catalog (to confirm the required.is_some() fix's catalog claim), skills/rocm-doctor/reference.md (outside the diff, impact analysis), and the exact diff hunks available for each finding's location (to confirm which could and couldn't be posted inline).

CI: all 40 reported checks completed green at this head (build-and-test, clippy, both Windows jobs, both E2E lanes, all security scans, docs build) — no required check is red or pending.

Did not run: a local build/test/clippy pass (relied on CI, consistent with the PR's own disclosure that the Windows DXGI/FFI path has unit coverage only, no integration run against real hardware). To check for an existing marked review at this exact head before starting, I read the prior review bodies on this PR via the API; the seven blind fan-out passes above were never shown that content. Not reconciled against any discussion on this PR beyond that marker check.


🤖 Automated by an AI agent

// nothing else to tell the two kinds of number apart.
if memory.is_shared_pool() {
estimate.notes.push(
"this host's integrated GPU has no dedicated VRAM: the memory compared against is \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — self-contradictory output: this note says "no dedicated VRAM," but the evidence line printed just above it in the same report still says "the GPU with the most dedicated VRAM."

evidence_lines (this file, ~line 1145-1151, unchanged by this PR) unconditionally renders:

"the GPU with the most dedicated VRAM that ROCm can see has {offered} MiB ({}) in total; memory other processes already hold is not subtracted"

whenever estimate.available_gib.is_some() — which is also true for the new Unified case this note handles. model_readiness.rs:992 extends readiness.evidence with evidence_lines(&estimate), and render_model_readiness_text prints evidence bullets ( - …) before printing estimate.notes ( note: …) in the same report (verified by reading both call sites). So on an APU with a readable shared pool, a GGUF-outside-the-catalog estimate currently prints both:

   - the GPU with the most dedicated VRAM that ROCm can see has 24576 MiB (24.0 GiB) in total; ...
   note: this host's integrated GPU has no dedicated VRAM: the memory compared against is the ceiling on a pool it shares with the operating system, ...

— directly contradicting itself in the same output. No test exercises evidence_lines with AcceleratorMemory::Unified (the new unit test for this note only checks .notes, not .evidence).

Confidence 90 · logic · Fix: branch the wording in evidence_lines on memory.is_shared_pool() (or thread a flag through ContextEstimate alongside conservative_ceiling), the same way this note itself is gated.

// this at best" rather than "this much is waiting for it". Said on the
// estimate itself, because the figure is printed beside the totals with
// nothing else to tell the two kinds of number apart.
if memory.is_shared_pool() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — this user-visible note has no Gherkin scenario, only a unit test.

This "shared pool is a ceiling, not free memory" note is new, user-observable text (ContextEstimate.notes is a public, serialized field reaching --json), covered only by the new unit test a_shared_pool_is_compared_against_its_ceiling_and_said_to_be_shared. AGENTS.md §3 is explicit that a unit test does not discharge this for user-observable CLI behavior — it proves the function, not the behavior. Verified independently: zero occurrences of this note's wording ("shares with the operating system", "no dedicated VRAM... ceiling") anywhere under tests/e2e-cucumber/; the existing GGUF scenarios (diagnose-49/-50) never combine a GGUF-outside-the-catalog model with an APU/shared-pool host. No PR-text justification for the omission is present either.

Confidence 85 · architectural (process/coverage) · Fix: add a scenario driving a GGUF estimate on a shared-pool host and asserting this note appears, or state in the PR text why one is impractical.

// terms answer different questions -- "does it use the GPU" and "did we
// weigh it against the pool" -- and only the second licenses this verdict.
if host.accelerator_memory.is_shared_pool()
&& requires_gpu(recipe)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking — the cpu_only-recipe immunity this gate grants has no Gherkin scenario, only a unit test.

This requires_gpu(recipe) condition (together with required.is_some(), added by the final commit on this branch to close the previous review round's finding) is what keeps a cpu_only recipe from being softened to Degraded on a shared-pool host — new, user-observable CLI behavior (whether the verdict is downgraded). It's covered only by the unit test a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses. Per AGENTS.md §3, a unit test does not discharge the Gherkin-scenario requirement for user-observable behavior. Verified independently: zero occurrences of "cpu_only"/"cpu-only" anywhere under tests/e2e-cucumber/, and the PR body's mutation-evidence table cites only this unit test's name for this row, with no scenario id and no stated justification for its absence.

Confidence 85 · architectural (process/coverage) · Fix: add a diagnose.feature scenario covering this, or state in the PR text why one is impractical.

# withholds every row first, and the answer is "GPU memory unmeasured" before
# the aperture is ever consulted -- so the scenario plants a stub `amd-smi`
# reporting the one compute device, alongside the two cards in sysfs. The
# premise it rests on is diagnose-68: the same APU with the same readable

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking — wrong scenario cross-reference: this says diagnose-68, but the scenario actually being described is diagnose-71.

diagnose-68 ("A fix whose helper command never finishes says the change may be partial...") is unrelated to APUs/memory. The scenario this comment means — "the same APU with the same readable 60 GiB aperture, alone, does get a figure" — is diagnose-71 ("An integrated GPU is judged on the pool it allocates from, not its carve-out"), confirmed at line 1074. The matching comment in diagnose_steps.rs (~line 532-535) correctly says diagnose-71. This is a copy/renumbering slip confined to this one comment; the scenario IDs themselves have no gaps or duplicates.

Confidence 95 · mechanical · Fix: change "diagnose-68" to "diagnose-71" here.

Comment thread docs/architecture.md
| [`host_gpu/linux.rs`](../crates/rocm-core/src/host_gpu/linux.rs) | Linux sysfs, KFD and DRM probes: the KFD topology, the DRM ip-discovery gfx target, the amdgpu device and driver version, the primary GPU name. | domain |
| [`host_gpu/os_facts.rs`](../crates/rocm-core/src/host_gpu/os_facts.rs) | Kernel version, distro name, installed RAM, and the CPU model and vendor for the host summary. One reader returns the CPU model and vendor together for both the summary and `Examination::probe`: Linux from `/proc/cpuinfo`, Windows from the CPU row of the system inventory. `detect_kernel_version` is now the only `uname -r` reader; [`examine/host_env.rs`](../crates/rocm-core/src/examine/host_env.rs)'s `probe_os` calls it for the examination's `kernel_release`. | domain |
| [`host_gpu/render.rs`](../crates/rocm-core/src/host_gpu/render.rs) | The `rocm examine` text rendering of an `ExamineSummary`. | domain |
| [`host_gpu/unified_memory.rs`](../crates/rocm-core/src/host_gpu/unified_memory.rs) | The memory pool an APU's engine actually allocates a model from, which is neither the BIOS VRAM carve-out every adapter query reports nor installed RAM: `mem_info_gtt_total` from the amdgpu DRM device directory on Linux, `DXGI_ADAPTER_DESC1::SharedSystemMemory` on Windows — read through hand-written DXGI vtable declarations, because `windows-sys` carries no COM interfaces — behind `detect_host_unified_memory_gib`. The adapter choice and the "a reading too small to be a pool is a failed measurement" rule are pure functions tested on every platform. | domain |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking — the gguf_fit.rs entry a few lines above this hunk (line ~314, unchanged by this PR) is now stale.

It still reads "Pure arithmetic on whether a GGUF model fits in GPU memory at a context size: weights, KV cache, compute buffer and reserve against VRAM." This PR's own updated doc comments inside gguf_fit.rs now describe the comparison as being against "the memory the host's engine would allocate from: a discrete card's dedicated VRAM, or — since this PR — the ceiling on the shared pool an APU draws on." The architecture.md row wasn't updated to match, unlike the model_readiness.rs entry in this same hunk, which was. cargo xtask check-architecture-doc only checks presence of an entry for a file, not that its description stays true (AGENTS.md §6 notes this explicitly is still on the author).

Confidence 72 · mechanical · Fix: append something like "...against VRAM, or an APU's shared-pool ceiling" to the gguf_fit.rs row.

Review finding, and one four earlier rounds missed: `evidence_lines`
rendered "the GPU with the most dedicated VRAM that ROCm can see has {N}
MiB" whenever `available_gib` was set, which is also true for the new
`Unified` case. `model_readiness` prints evidence bullets above the
estimate's notes in one report, so an APU got that line a few lines above
"this host's integrated GPU has no dedicated VRAM".

The two renderers were deriving the same fact separately and only one of them
had been taught about shared pools. `ContextEstimate` now carries
`available_is_shared_pool`, set where the note is pushed, so both describe
the figure from one source rather than two. The evidence line says "the pool
this host's integrated GPU allocates from ... a ceiling it shares with the
operating system" in that case.

Pinned by `the_evidence_line_and_the_note_cannot_disagree_about_what_the_
figure_is`, which asserts the relation between the two renderers across both
memory kinds rather than two literal strings, so rewording either cannot
restore the contradiction. Mutation-checked: neutering the branch reproduces
it verbatim.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

The new finding is real and I have fixed it (9c5c0c1df). Thank you for catching something four rounds had walked past.

Blocking 1 — the self-contradicting report — fixed. evidence_lines rendered "the GPU with the most dedicated VRAM that ROCm can see has {N} MiB" whenever available_gib was set, which the new Unified case also satisfies. Since model_readiness prints evidence bullets above the estimate's notes in one report, an APU got that line a few lines above "this host's integrated GPU has no dedicated VRAM".

The root of it is that the two renderers were deriving the same fact independently and only one had been taught about shared pools. Rather than branch the wording in a second place and leave the same trap for the next renderer, ContextEstimate now carries available_is_shared_pool, set where the note is pushed, so both descriptions come from one source. The evidence line now reads "the pool this host's integrated GPU allocates from … a ceiling it shares with the operating system" for that case.

Pinned by the_evidence_line_and_the_note_cannot_disagree_about_what_the_figure_is, which asserts a relation between the two renderers across both memory kinds — not two literal strings — so rewording either cannot quietly restore the contradiction. Mutation evidence, neutering the branch:

shared=true: one report called the same figure dedicated VRAM and not dedicated VRAM.

Blocking 2 and 3 — the two missing Gherkin scenarios — still open, and still the honest answer. The GGUF shared-pool note and the cpu_only immunity are both user-observable and covered only by unit tests. I said last round these need scenarios rather than a justification, and that has not changed: nothing makes them impractical, since the existing APU scenarios already plant hosts through the same seam.

Blocking 4 — skills/rocm-doctor/reference.md:81 — still open. You flagged it last round, I said I would take it next, and I did not. It is a one-line caveat and it feeds the skill as ground truth, so it should not keep slipping behind the code findings. Taking it with the scenarios.

Non-blocking: the wrong scenario cross-reference in diagnose.feature and the stale gguf_fit.rs row in docs/architecture.md (still "against VRAM") — both noted, both going in the same pass as the above.

Decisions: DXGI partial enumeration remains the intended tradeoff, as the PR body states. vram_carveout_gib is still write-only; I will re-frame it as reserved. On requires_gpu() treating gpu_preferred like gpu_required — today GPU presence is effectively mandatory for both, so the logic is right; it is a naming note rather than a defect, and worth saying out loud if gpu_preferred ever grows a CPU-fallback path.

One fixed, four open. CHANGES_REQUESTED stands.

… claimed

Three stale descriptions of the figure this change made shared-pool-capable,
all flagged across earlier review rounds and all deferred behind the code
findings until now.

skills/rocm-doctor/reference.md: `available_gib` was described as "the
total dedicated VRAM of the GPU with the most of it". It now also carries an
integrated GPU's shared-pool ceiling, which by this change's own evidence text
is explicitly not dedicated VRAM. This file is the rocm-doctor skill's ground
truth, so a wrong claim here is one an assistant repeats. It now names both
cases and points at `available_is_shared_pool` as the only way to tell them
apart from the number alone.

docs/architecture.md: the `gguf_fit.rs` row still said the comparison is
"against VRAM".

tests/e2e-cucumber/features/diagnose.feature: a comment named diagnose-68 as
the premise its two-card scenario rests on; after the merge that scenario is
diagnose-71, and diagnose-68 is main's unrelated fix-timeout scenario.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Closing the three doc items I had been deferring (6ad4f1562).

Blocking 4 — skills/rocm-doctor/reference.md:81 — fixed. You flagged this twice and I pushed it behind the code findings twice, which was the wrong order: this file is the rocm-doctor skill's ground truth, so a stale claim here is one an assistant repeats to users. It said available_gib is "the total dedicated VRAM of the GPU with the most of it that ROCm can see". It now names both cases and points at available_is_shared_pool as the only way to tell them apart from the number alone — that field is new in the commit before this one, added so the evidence line and the note stop deriving the same fact separately.

Non-blocking — docs/architecture.md's gguf_fit.rs row — fixed. Still said the comparison is "against VRAM"; it now names both the dedicated and shared-pool cases.

Non-blocking — the scenario cross-reference — fixed. The comment named diagnose-68 as the premise its two-card scenario rests on; after the main merge that scenario is diagnose-71, and diagnose-68 is now main's unrelated fix-timeout scenario, so the reference pointed somewhere actively misleading.

Verified on Linux: cargo fmt --all --check, clippy --workspace --all-targets, clippy -p e2e-cucumber --test e2e, feature_naming, the three xtask checks, and cargo xtask catalog --check (since reference.md is checked against the catalog).

Still open, and these are the real remainder: the two missing Gherkin scenarios — the GGUF shared-pool note and the cpu_only immunity. Both are user-observable and unit-tested only, and as I said last round, nothing makes them impractical to simulate, so a stated justification would be dodging rather than answering. They are the next piece of work on this PR.

CHANGES_REQUESTED stands.

…note as behaviour

Both were new user-observable behaviour with only a unit test behind them,
which proves the function and not the behaviour. Neither was impractical to
simulate: the APU scenarios beside them already plant a host through
ROCM_CLI_TEST_HOST_ROOT.

diagnose-75 needs a cpu_only recipe and the shipped catalog has none -- all 49
entries are gpu_required or gpu_preferred -- so it plants a signed recipe
index holding the smallest curated recipe twice, once as shipped and once with
its device policy and GPU minimum removed, and asks about both on one planted
APU. The second half is the premise: the same host must still soften a recipe
that does need the GPU, or the first half would pass against a check someone
had deleted outright.

diagnose-76 is the same planted APU and diagnose-49's tiny GGUF. It asserts
the aperture is the figure compared against before it looks for the note: a
host that fell back to cannot_estimate carries no figure at all, and a
scenario that only looked for a wording would pass against that while
exercising nothing.

Writing them turned up a trap in the harness. scenario_command takes
command_env rather than borrowing it, so a Given's fixture reaches exactly one
child process; a scenario that asks two questions of the same host gets the
runner's real machine for the second. The first draft of diagnose-75 answered
about a dev box's 191.9 GiB MI300X out of the built-in catalog while reading
as though it answered about the planted APU.
ask_again_on_the_same_planted_machine restores the environment between
questions, and the first half now asserts the planted 60 GiB aperture so a
leak fails loudly instead of changing the subject.

Mutation-tested: dropping the requires_gpu/required.is_some() gate turns
diagnose-75 red (degraded, expected ready); dropping the estimator's
is_shared_pool note turns diagnose-76 red, both when the whole block goes and
when only the notes.push does, leaving the flag set.

The signed-index keygen/sign dance is factored out of the RAM-degraded fixture
rather than copied a third time.

ROCMAI-912

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Both findings are closed with scenarios rather than a stated reason. You were right that a justification would have been dodging — nothing made either impractical to simulate.

Finding A — the cpu_only immunity.
diagnose-75 - A CPU-only recipe is not softened by a shared pool it never allocates from, @id:diagnose-model-a-cpu-only-recipe-is-not-softened-by-a-pool-it-never-uses, @requires-os:linux, so it runs on the GPU-less mock lane beside diagnose-71–diagnose-74.

The shipped catalog has no cpu_only recipe — all 49 entries are gpu_required or gpu_preferred — so the scenario plants a signed recipe index (the only way the CLI loads another catalog, same route as model-catalog-03) carrying the smallest curated recipe twice: once as shipped, once with device_policy: "cpu_only" and its GPU minimum removed. Both are asked on the same planted APU in the one scenario. The second half is the premise — that host must still soften a recipe that does need the GPU — because the first assertion alone would pass just as happily against a shared-pool check deleted outright.

Mutation: dropping requires_gpu(recipe) && required.is_some() from the gate in model_readiness.rs turns the first half red — left: "degraded", right: "ready". Restored, green.

Finding B — the GGUF estimator's shared-pool note.
diagnose-76 - A GGUF estimate on an integrated GPU is told the figure is a shared ceiling, not free memory, @id:diagnose-model-a-gguf-estimate-says-its-pool-is-shared, @requires-os:linux, same lane. Same planted APU as diagnose-71, same tiny GGUF fixture as diagnose-49.

It asserts the 60 GiB aperture is the figure compared against before it looks for the note, for the reason diagnose-74 taught us: a host that fell back to cannot_estimate carries no available_gib at all, and a scenario that only looked for a wording would pass against that while exercising nothing.

Mutation, twice: removing the whole if memory.is_shared_pool() block turns it red on the flag; removing only the notes.push and leaving available_is_shared_pool = true turns it red on the note — so the note assertion is falsifiable on its own, not riding on the flag. Restored, green.

One thing the work turned up that is worth flagging beyond this PR. scenario_command does std::mem::take(&mut world.command_env), so everything a Given plants reaches exactly one child process. A scenario that asks two questions of the same host gets the planted machine for the first and the runner's own for the second — not a failure a reader would recognise, a silent change of subject. The first draft of diagnose-75 answered about the dev box's real 191.9 GiB MI300X out of the built-in catalog while reading as though it answered about the planted APU. ask_again_on_the_same_planted_machine restores the environment between questions, and the first half now also asserts the planted 60 GiB aperture so a leak fails loudly. Any future multi-When scenario that plants a fixture has the same trap waiting.

Also folded in: the signed-index keygen/sign dance is factored out of the RAM-degraded fixture rather than copied a third time, and the mutation-evidence table in the PR body now names @id: tags instead of scenario numbers — four of its rows had gone stale against renumbering (diagnose-68–diagnose-71 are now diagnose-71–diagnose-74).

Gates, all on a Linux ROCm box: fmt --all --check; clippy --workspace --all-targets -D warnings; clippy -p e2e-cucumber --test e2e -D warnings; test -p e2e-cucumber --test feature_naming; xtask check-architecture-doc; xtask check-crate-edges; xtask check-module-size; test -p rocm-core (1659 passed, with the known uv install-path flake, which passes re-run alone). Scenarios run for real, not just the naming gate: diagnose-25, -49, -50, -71…-76 all pass.

One pre-existing red on that box I did not cause and did not touch: diagnose-70 (@id:diagnose-fix-igpu-aborts-on-unreadable-user-scope) fails identically on this branch's previous head with my two files reverted to 6ad4f15. It is a Windows-fixture scenario carrying no @requires-os:windows tag; separate from this work.

@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

One correction to my previous comment, before it misleads anyone.

I reported diagnose-70 as a pre-existing red caused by "a Windows fixture missing @requires-os:windows". That is wrong on both counts and there is no pre-existing failure here.

diagnose-70 (@id:diagnose-fix-igpu-aborts-on-unreadable-user-scope) is tagged @requires-os:windows. What produced the failure was the way I ran it: cargo xtask e2e -- -n "diagnose-70". The -n filter deliberately bypasses the harness's own applicability gating — docs/testing.md says so in two places, including "Cucumber name filtering bypasses the harness's custom applicability/expectation" — so naming a Windows-only scenario on a Linux box forces it to run somewhere it was never meant to. The resulting "was expected to pass on this host but FAILED — a regression" line is an artifact of the filter, not a defect.

The check that settles it: this PR's CI reports no failures, and diagnose-70 is correctly skipped on every Linux lane.

Worth recording as a trap, since -n is the documented way to run a scenario locally: it is safe for same-platform scenarios and actively misleading for cross-platform ones. The scenarios added in 97afa1ff2 are both @requires-os:linux and were run on Linux, so their results stand.

The two new scenarios and their mutation evidence are unaffected by this correction.

@r0x0r
r0x0r requested a review from juhovainio November 17, 2026 16:54
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Every finding from the latest round is now addressed, so I am re-requesting review. Verified each against the code rather than against my own commit messages:

Finding Where it landed
Blocking 1 — report called one figure both "dedicated VRAM" and "no dedicated VRAM" 9c5c0c1df — ContextEstimate::available_is_shared_pool now carries the fact, so the evidence line and the note derive from one source; pinned by a relation test, mutation-checked
Blocking 2 — GGUF shared-pool note had no scenario 97afa1ff2 — diagnose-76, @id:diagnose-model-a-gguf-estimate-says-its-pool-is-shared
Blocking 3 — cpu_only immunity had no scenario 97afa1ff2 — diagnose-75, @id:diagnose-model-a-cpu-only-recipe-is-not-softened-by-a-pool-it-never-uses
Blocking 4 — skills/rocm-doctor/reference.md stale on available_gib 6ad4f1562
Non-blocking — docs/architecture.md's gguf_fit.rs row said "against VRAM" 6ad4f1562
Non-blocking — feature comment named diagnose-68 as its premise 6ad4f1562 — now diagnose-71

Both new scenarios are @requires-os:linux and run on the GPU-less mock lane. Mutation evidence: dropping requires_gpu(recipe) && required.is_some() turns diagnose-75 red (left: "degraded", right: "ready"); dropping the is_shared_pool() block turns diagnose-76 red, and dropping only the notes.push while keeping the flag still turns it red — so the note assertion stands on its own rather than riding on the figure assertion.

Two things worth flagging rather than burying:

No shipped recipe is cpu_only — all 49 are gpu_required/gpu_preferred — so diagnose-75 plants a signed recipe index, the same route model-catalog-03 uses, carrying the smallest curated recipe twice: as shipped, and with device_policy: "cpu_only" and no GPU minimum. Both are asked on one planted APU, and the shipped half is the in-scenario premise, so the scenario cannot pass by the gate having been deleted outright.

A trap we hit and fixed in the first draft: scenario_command does std::mem::take(&mut world.command_env), so a Given's planted fixture reaches exactly one child process. The first version of diagnose-75 silently answered about the runner's real MI300X from the built-in catalog while reading as though it described the planted APU — green, and testing nothing. It now re-plants for the second question and asserts the aperture figure, so a leak fails loudly. Any future multi-When scenario has the same hazard.

The decisions you raised stand as answered: DXGI partial enumeration remains the intended tradeoff, vram_carveout_gib stays reserved rather than wired into a report in this PR, and requires_gpu() treating gpu_preferred like gpu_required is correct today since GPU presence is effectively mandatory for both.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — needs work

Full review of the whole change (17 files, +2653/-194, base ca170c9965cdafacf5ded70b9c85265bd3c25910 → head 97afa1ff2cc4bd0c9b668eadb59ddaaf4b084b11), read completely across six fan-out passes.

Implements the GTT/DXGI shared-memory-pool measurement portion of ROCMAI-912 (reading the real pool an AMD APU's engine allocates from and capping the verdict at Degraded); the description itself scopes out free-memory measurement, BIOS/gttsize guidance, and SVM-mapping-failure detection as separate work, and that boundary matches what the diff actually contains. Ticket ROCMAI-912 named but not retrieved (ticket-tracker lookups are out of scope for this review on a public repo).

This PR has a long review history on this account; this is a fresh pass at a new head (97afa1ff2) that advanced past the last marked review (60c29d572). I independently re-verified, against the current code rather than against commit messages, that every blocking item from the most recent prior round is genuinely fixed here: the gpu_required-with-no-declared-minimum gate now requires required.is_some() in addition to requires_gpu(recipe) (model_readiness.rs:680), confirmed against the shipped catalog; the GGUF shared-pool note and the cpu_only exemption each now have their own Gherkin scenarios (diagnose-75, diagnose-76); the evidence_lines-vs-notes self-contradiction is fixed via a threaded available_is_shared_pool flag with a relational test; and skills/rocm-doctor/reference.md's stale available_gib claim is corrected. The two non-blocking leftovers from that round (a stale scenario cross-reference in a feature-file comment, and a stale docs/architecture.md row for gguf_fit.rs) are also fixed. The two findings below are new ground past what those rounds covered.

Blocking

  • gguf_fit.rs's module-level doc comment still claims the comparison is always against "dedicated VRAM" — crates/rocm-core/src/gguf_fit.rs:8 (not touched by this diff's hunks, so this cannot be anchored as an inline comment)
    The module doc reads: "Weights plus KV cache plus a compute buffer plus a runtime reserve, compared against the dedicated VRAM the readiness check already reads." This is exactly the claim the rest of this PR corrects: estimate_fit now also compares against AcceleratorMemory::Unified's shared-pool ceiling (gated by is_shared_pool() at gguf_fit.rs:876-884), and every other doc comment in this same file that makes this claim — EstimateOutcome::Fits/DoesNotFit (lines 176-190), ContextEstimate::available_gib/available_is_shared_pool (lines 285-301) — was carefully updated in this PR to say "a discrete card's dedicated VRAM, or an APU's ceiling." This one line was missed and now directly contradicts the code it summarizes, one screen below the up-to-date docs for the type it describes.
    Confidence 85 · mechanical · Fix: reword the module doc's opening line to match the updated EstimateOutcome doc.

  • The branch that preserves an existing system-RAM remediation when the shared-pool gate also fires has no test — crates/rocm-core/src/model_readiness.rs:680-723 (inline comment posted on line 688)
    See inline comment for the full trace: no test combines a shared pool with a host already Degraded from the system-RAM check before this gate runs, so the if was_ready { ... } guard protecting that remediation from being overwritten is unexercised.
    Confidence 80 · logic · Fix: add a test with AcceleratorMemory::Unified{..} ample for the recipe's GPU minimum, combined with low system_ram_gib, asserting Degraded with the system-RAM verify preserved.

Non-blocking

None.

Decisions for the author

  • DXGI partial enumeration (a non-NOT_FOUND EnumAdapters1 error, or a GetDesc1 failure on an enumerated adapter) is treated as if the adapter list were complete — answered-by-intent
    PR body, Scope section: "Partial DXGI enumeration is treated as a complete answer. An EnumAdapters1 error other than DXGI_ERROR_NOT_FOUND ends the adapter loop the same way the clean end of the list does, so a second AMD adapter sitting past a transient error would be missed and the first one's budget confidently attributed. NOT_FOUND is what that call returns at the true end of the list in practice, and the Windows path is unexecuted either way, so this is left as known and untested rather than guessed at." This also covers the structurally identical GetDesc1-failure path (an adapter silently dropped from consideration rather than making the result conservative, unlike the sibling Linux GTT reader added in the same PR, which counts AMD-card identity independently of telemetry-read success). Would otherwise have been a blocking architectural finding — it is a real, confirmed gap in the module's central "exactly one AMD adapter" safety invariant, with no test exercising either early-exit path. Retired because the author already named and reasoned about the general shape of this gap. Confirm this remains the intended tradeoff before the Windows path gets real execution coverage.
  • vram_carveout_gib is currently write-only — non-blocking-improvement
    Computed and stored on every AcceleratorMemory::Unified value, but nothing in production reads it (measured_gib() discards it via ..). Its own doc comment says it's kept only so a report can show the two figures side by side, and that is not in this PR's stated scope one way or the other, so this isn't retired as answered-by-intent — but it's a reasonable, low-severity completeness gap rather than dead code proper (it is written and carried, just never rendered). Either wire it into an actual report line, or state it's reserved for a specific future change.

Positive signals

  • The shared-pool Degraded gate's is_shared_pool() && requires_gpu(recipe) && required.is_some() guard correctly handles the "GPU-requiring recipe with no declared minimum" edge case, backed by a catalog-driven (not synthetic) regression test verified directly against the seven shipped recipes with exactly that shape.
  • gguf_fit.rs's available_is_shared_pool field is threaded through so evidence_lines and the shared-pool note in estimate.notes cannot independently derive and then disagree about the same figure — backed by a relational (not string-literal) test whose own comment documents the earlier bug it exists to catch.
  • the_shared_pool_plan_names_no_check_that_can_never_clear is genuinely mutation-aware: it asks the same question twice to prove the "no verify line" premise rather than asserting it from the source.
  • The e2e harness's command_env-consumed-by-value trap (a Given's fixture reaching only the first of two When questions in one scenario) was caught and fixed with ask_again_on_the_same_planted_machine, which re-asserts the planted figure on the second question so a future leak fails loudly instead of silently answering about the runner's own hardware.
  • The Windows DXGI/COM FFI — GUIDs, struct layout, vtable order, reference-counting on every path including early returns and the GetDesc1-failure branch, and the pointer-width read of SharedSystemMemory avoiding 32-bit saturation — is correct on close inspection, despite having zero execution coverage.
  • User-facing docs (README.md, docs/architecture.md, docs/manual-testing.md, docs/testing.md, skills/rocm-doctor/reference.md) are thoroughly and consistently updated everywhere the old "undetermined"/"dedicated VRAM" claim lived; a full-file (not just diff) sweep found no remaining stale occurrence.

Deployment notes

Changes rocm diagnose --model's observable output for APU hosts whose shared pool is readable: previously undetermined/unified_memory_unreadable, now a measured figure with the verdict capped at degraded (never ready). Anything scripting against the old undetermined value for these hosts will see different output. No migration needed; the change is additive (new enum variant, new serde-compatible available_is_shared_pool field).

What this covered

Full diff ca170c9965cdafacf5ded70b9c85265bd3c25910...97afa1ff2cc4bd0c9b668eadb59ddaaf4b084b11 (17 files, +2653/-194), read completely across six fan-out passes, each blind (no PR description, ticket, or review-thread content): (1) the Linux/Windows measurement layer and its FFI (host_gpu.rs, host_gpu/linux.rs, host_gpu/unified_memory.rs, lib.rs, Cargo.toml, gpu_policy.rs); (2) the classification/verdict/estimator layer plus CLI help text (model_readiness.rs, gguf_fit.rs, model_fit.rs, cli.rs); (3) the e2e scenarios and step definitions (diagnose.feature, diagnose_steps.rs); (4) user-facing docs; (5) an AGENTS.md-adherence and code-comment-accuracy sweep; (6) commit/version history across all 17 commits. Passes (1), (2), (4), (5) and (6) ran as independent background workers; pass (3) could not be spawned as an independent worker after repeated attempts failed on a concurrency limit shared across other sessions on this host, so I read it myself in the same context as this report — flagged here as not independent, per the method's own fallback for a worker that cannot be spawned.

Supplemented directly: the shipped model catalog JSON (confirming the "seven recipes" and "cpu_only" claims), direct re-verification of the required.is_some() gate, the evidence_lines/notes relation test, and the two previously-flagged non-blocking doc items from the prior round. CI: all 40 reported checks are green at this head (build-and-test and clippy on both Linux and Windows, both E2E lanes, the Windows release build/lifecycle job, Sphinx docs build, commit-signature verification, security scans, skill checks) — no required check is red or pending.

Did not run: a local build/test/clippy pass (relied on CI, consistent with the PR's own disclosure that the Windows DXGI/FFI path has unit coverage only, no integration run against real hardware). Reconciliation against the five prior review rounds' threads on this PR was done only after the findings above were locked blind, per the method's ordering; it is reflected in the "genuinely fixed" statement above rather than in the findings themselves.


🤖 Automated by an AI agent

ModelVerdict::Ready | ModelVerdict::Degraded
)
{
let was_ready = readiness.verdict == ModelVerdict::Ready;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the "preserve the system-RAM remediation" branch (if was_ready { ... } below) has no test combining a shared pool with a host already Degraded from the system-RAM check before this gate runs.

Every existing test that reaches this gate (an_apu_with_a_readable_pool_is_degraded_never_ready, the_shared_pool_plan_names_no_check_that_can_never_clear, a_cpu_only_recipe_is_not_degraded_by_a_shared_pool_it_never_uses, a_gpu_recipe_with_no_declared_minimum_is_not_degraded_by_a_pool_it_was_never_weighed_against) sets system_ram_gib comfortably above the recipe's recommendation, so was_ready stays true in all of them. The sibling test for the system-RAM path alone, a_host_below_the_recommended_system_ram_is_degraded_not_blocked, uses AcceleratorMemory::Dedicated(64.0), not a shared pool. So nothing exercises the case this comment block exists to handle: a shared-pool host that was already Degraded from the system-RAM check reaching this gate with was_ready == false. Deleting or inverting the if was_ready condition — e.g. always overwriting readiness.fix with the "just serve it" plan, silently dropping the system-RAM remediation's verify: rocm diagnose --model ... — would not be caught by any test in the suite today.

Confidence 80 · logic · Fix: add a test with AcceleratorMemory::Unified{..} ample enough to clear the recipe's GPU minimum, combined with system_ram_gib below the recipe's recommended_system_ram_gb, asserting the verdict is Degraded and that fix still carries the system-RAM check's verify/summary rather than the shared-pool "nothing is broken, just serve it" plan.

… RAM fix

Two review findings.

crates/rocm-core/src/gguf_fit.rs:8 still opened by saying the comparison is
"against the dedicated VRAM the readiness check already reads" -- the exact
claim the rest of this change corrects. Every other doc in the file saying it
was updated; this one was missed, so the module summary contradicted the types
documented one screen below it.

The `if was_ready` guard in `model_readiness.rs` had no test. It stops the
shared-pool gate writing its own `fix` over one an earlier check already
wrote, which matters because the gate's `Degraded` deliberately carries no
`verify` -- replacing a system-RAM remediation with it takes away the one
actionable step the user had. Nothing combined the two: every shared-pool test
had ample RAM and every system-RAM test had a dedicated card.

`a_host_already_softened_for_system_ram_keeps_that_remediation` plants an
ample shared pool with low system RAM so both softenings fire, and asserts
both premises as well as the outcome. Mutation-checked: forcing the guard true
replaces the fix with the gate's own, empty-`verify` one, which is the defect
stated plainly.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r

r0x0r commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Both findings fixed in 07a8ce250.

Blocking 1 — gguf_fit.rs:8's module doc — fixed. You are right that this is the exact claim the rest of the change corrects, and right that it is worse for being one screen above the types whose docs were updated. Worth noting how it survived: I corrected the gguf_fit.rs row in docs/architecture.md two rounds ago and the EstimateOutcome/available_gib docs in this file, and still missed the module summary at the top of the same file. That is the second time in this session a stale count or claim has sat directly adjacent to one I had just fixed. The opening line now says "the memory the engine allocates from … a discrete card's dedicated VRAM, or an integrated GPU's ceiling on the pool it shares with the operating system", matching EstimateOutcome.

Blocking 2 — the untested if was_ready guard — fixed. Your trace was right, including why it matters: the gate's own Degraded deliberately carries no verify, so overwriting an earlier remediation takes away the one actionable step the user had. Nothing combined the two conditions before — every shared-pool test had ample RAM, every system-RAM test had a dedicated card.

a_host_already_softened_for_system_ram_keeps_that_remediation plants an ample shared pool (so the GPU comparison itself passes and the gate is the only thing that could soften it) with system RAM below the recipe's recommendation. It asserts both premises — that the verdict really is Degraded and that the shared-pool gate really did fire — as well as the outcome, so it cannot pass by one of the two softenings having quietly stopped happening.

Mutation evidence, forcing the guard true:

the shared-pool gate overwrote the system-RAM remediation with its own:
Fix { summary: "nothing here is broken: serve it and let the engine answer, having
closed what else is using memory", ..., verify: "" }

An empty verify where the user previously had rocm diagnose --model <ref> is the regression in one line.

Decisions: DXGI partial enumeration remains the intended tradeoff, as the PR body states — confirmed. vram_carveout_gib is still write-only; I will state it as reserved for a specific future change rather than wire a report line in this PR, since rendering it is not in scope here either way.

Verified on Linux: cargo fmt --all --check, clippy --workspace --all-targets, clippy -p e2e-cucumber --test e2e, feature_naming, the three xtask checks, and cargo test -p rocm-core.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants