Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
80ceb08
ROCMAI-556: warn when an AIM catalog entry hasn't been verified recently
juhovainio Oct 29, 2026
f1732f9
ROCMAI-556: add the AMD Enterprise AI AIM catalog to the model catalog
juhovainio Oct 29, 2026
4282f16
ROCMAI-556: use the real model names as AIM canonical ids
juhovainio Oct 29, 2026
ce59ace
ROCMAI-556: flag AIM models that need a Hugging Face token
juhovainio Oct 29, 2026
b22835b
ROCMAI-556: list AIM as an engine on the AMD Instinct catalog platform
juhovainio Oct 29, 2026
86e80b8
ROCMAI-556: match AIM images to the GPUs their profiles support
juhovainio Oct 29, 2026
8c9407a
ROCMAI-556: enforce HF tokens, warn on other boards, estimate VRAM fo…
juhovainio Oct 29, 2026
fd75c39
ROCMAI-556: refuse multi-GPU AIM images before launch
juhovainio Oct 29, 2026
2994651
ROCMAI-556: address review feedback on AIM catalog routing, docs link…
juhovainio Oct 30, 2026
fe3bb42
ROCMAI-556: cover the gated AIM HF-token refusal with e2e scenarios
juhovainio Oct 30, 2026
840fe28
ROCMAI-556: fix clippy lint and model-catalog-01 for the renamed AIM …
juhovainio Nov 2, 2026
629ee8a
ROCMAI-556: use map_or in assert_no_service_started
juhovainio Nov 2, 2026
ed0701c
ROCMAI-556: describe AIM in `rocm model --help` and spell out its name
juhovainio Nov 2, 2026
d951886
ROCMAI-556: refuse a gated AIM image for its token before the GPU check
juhovainio Nov 2, 2026
6718ca2
ROCMAI-556: address review on AIM engine selection, warnings and cove…
juhovainio Nov 2, 2026
1c72890
ROCMAI-556: address review on planner engine choice, AIM pre-flight a…
juhovainio Nov 3, 2026
df87051
ROCMAI-556: fall back to the recipe's next engine, reject empty --eng…
juhovainio Nov 3, 2026
fbd0a83
ROCMAI-556: unset min_gpu_mem_gb where a single-GPU MXFP4 profile con…
juhovainio Nov 3, 2026
0ce3cc7
ROCMAI-556: probe Docker once per model listing, make chat and action…
juhovainio Nov 3, 2026
a8debdb
ROCMAI-556: keep a vLLM-first recipe on vLLM on native Windows
juhovainio Nov 3, 2026
ba41cc9
Merge remote-tracking branch 'origin/main' into fix-794
juhovainio Nov 3, 2026
ee5eec0
ROCMAI-556: probe Docker only after serve's AIM pre-flight refusals
juhovainio Nov 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,11 @@ Every local (relative-path) Markdown link and `#anchor` fragment in the reposito

Since only links are checked, cite a specific repository file as a link relative to the citing file, `[fix.rs](../crates/rocm-core/src/fix.rs)` from `docs/`, rather than a bare backtick path, which nothing checks. The full rule, including the link forms that fail the check and the places it doesn't apply (`.github/`, links leaving a `skills/` folder, and `docs/rocm-docs/` along with the files it includes, which are this file and README.md), is in AGENTS.md section 5 ("Investigate rocm-cli Before Editing"); it applies to citations you add or edit.

### Updating the AIM catalog

AIM model entries are hand-maintained in `crates/rocm-core/src/model_catalog.json`;
see `docs/aim-catalog.md` for the refresh process and staleness warning.

### Test commands

| Component | Command |
Expand Down
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -821,7 +821,14 @@ rocm model [--verbose]
`rocm model` (alias `rocm models`) lists the recommended local models, grouped by
the hardware they target. Pass `--verbose` to list every recipe, including ones
hidden from the default list, with details such as aliases, data type, minimum
GPU memory, and preferred engines.
GPU memory, and preferred engines. The AMD Inference Microservices (AIM) Docker
images are among the hidden recipes: `--verbose` shows each one's supported GPU
families, the GPU count it needs per family, and whether it needs a Hugging Face
token (`requires_hf_token=yes`), and
`rocm serve <model> --engine aim` runs it (needs a reachable Docker daemon).
AIM entries are hand-maintained and record when they were last verified;
`rocm model --verbose` warns when one is over 180 days old (see
`docs/aim-catalog.md`).

### Model serving

Expand Down Expand Up @@ -912,7 +919,10 @@ instead of silently substituting a different model.

Some models (such as Llama) are gated and require Hugging Face authentication.
Log in with `huggingface-cli login` or set `HF_TOKEN` in your environment
before serving gated models.
before serving gated models. The exception is `--engine aim`, whose container
does not read your environment on its own: export `HF_TOKEN` and pass
`--engine-env HF_TOKEN` (see above), or `rocm serve` refuses a gated AIM image
before pulling anything.

`--gpu` selects which AMD GPU the server runs on. `auto` (the default) probes
per-GPU VRAM (through `amd-smi`, or the amdgpu DRM sysfs counters when `amd-smi`
Expand Down
44 changes: 43 additions & 1 deletion apps/rocm/src/chat_host_facts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,23 @@ impl HostFacts {
self.os_name != "windows"
}

/// Whether any catalogued AIM image has a profile for this GPU's family.
/// `None` when the family is unknown, where `rocm serve` enforces nothing.
fn aim_catalog_covers_gpu(&self) -> Option<bool> {
let family = self
.gfx_target
.as_deref()
.or(self.therock_family.as_deref())
.and_then(rocm_core::normalize_therock_family)?;
Some(rocm_core::builtin_model_recipes().iter().any(|recipe| {
recipe
.engine_recipes
.iter()
.filter(|record| record.engine.eq_ignore_ascii_case("aim"))
.any(|record| record.supported_gfx_families.contains(&family))
}))
}

/// How to describe the operating system to a non-technical user. WSL is
/// called out because it is the case the old prompt got wrong: it asserted
/// "on native Windows, vLLM is skipped" at a user who was on WSL, where
Expand Down Expand Up @@ -123,12 +140,18 @@ impl HostFacts {
"- Serving engines that run here: Lemonade. vLLM and AIM are skipped on native Windows; \
tell the user to use WSL2 or Linux for those ROCm GPU engines.\n"
};
let aim_note = if self.vllm_available() && self.aim_catalog_covers_gpu() == Some(false) {
"- No catalogued AIM model has a profile for this GPU family, and `rocm serve` \
refuses them here; do not suggest an AIM catalog model.\n"
} else {
""
};
format!(
"Facts about the machine you are running on, detected by ROCm CLI. Trust these \
over anything you remember about ROCm:\n\
- Operating system: {os}\n\
- AMD GPU: {gpu}\n\
{family}{engines}\
{family}{engines}{aim_note}\
ROCm and ROCm CLI support this operating system. Never tell the user ROCm is \
unavailable on their platform, and never suggest CUDA, DirectML, or CPU fallback.",
os = self.operating_system(),
Expand Down Expand Up @@ -175,6 +198,11 @@ mod tests {
block.contains("Lemonade, vLLM and AIM"),
"vLLM and AIM are available under WSL:\n{block}"
);
// Strix Halo has no profile in any catalogued AIM image.
assert!(
block.contains("do not suggest an AIM catalog model"),
"{block}"
);
assert!(
!block.contains("ROCm/TheRock family"),
"a family identical to the target adds nothing:\n{block}"
Expand Down Expand Up @@ -204,6 +232,20 @@ mod tests {
);
}

#[test]
fn aim_catalog_note_only_appears_where_no_catalogued_image_covers_the_gpu() {
let block = |target: Option<&str>| {
HostFacts::from_gpu_summary("linux", false, gpu(None, target, target)).render()
};
let note = "do not suggest an AIM catalog model";
assert!(!block(Some("gfx942")).contains(note));
assert!(!block(Some("gfx1100")).contains(note));
assert!(!block(Some("gfx1201")).contains(note));
assert!(block(Some("gfx1151")).contains(note));
// No detected GPU: nothing is enforced, so nothing is claimed.
assert!(!block(None).contains(note));
}

#[test]
fn bare_linux_is_not_reported_as_wsl() {
let facts =
Expand Down
4 changes: 3 additions & 1 deletion apps/rocm/src/dash.rs
Original file line number Diff line number Diff line change
Expand Up @@ -190,10 +190,12 @@ fn model_recipe_summaries() -> Vec<ModelRecipeSummary> {
builtin_model_recipes()
.into_iter()
.map(|r| ModelRecipeSummary {
// Same rule as the chat planner: a host-gated AIM first choice is
// not pre-selected when the recipe lists another engine.
preferred_engine: crate::freeform_planner::planner_recipe_engine(&r).map(str::to_owned),
id: r.canonical_model_id,
aliases: r.aliases,
task: r.task,
preferred_engine: r.preferred_engines.into_iter().next(),
})
.collect()
}
Expand Down
Loading
Loading