Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 98 additions & 2 deletions apps/rocm/src/chat/mcp_bridge.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,9 @@ use crate::{
build_codex_bridge_gpu_snapshot, build_codex_bridge_snapshot,
builtin_codex_bridge_engine_inventory, cli_version_string, record_cli_audit_event,
render_config_text, render_engine_inventory_text_with_paths, render_examine_text,
render_examine_text_with_paths, render_model_registry_verbose_text_with_context_and_host,
render_setup_status_text, therock_install_version_selector,
render_examine_text_with_paths, render_model_registry_text_with_context_and_host,
render_model_registry_verbose_text_with_context_and_host, render_setup_status_text,
therock_install_version_selector,
};
use crate::{comfyui, providers, therock};
use anyhow::{Context, Result, bail};
Expand Down Expand Up @@ -497,8 +498,28 @@ fn run_rocm_read_only_in_process(paths: &AppPaths, args: &[String]) -> Result<St
{
Ok(format!("rocm-cli {}\n", cli_version_string()))
}
// `model` / `models` render what `rocm model` prints in a terminal —
// the curated recommendation list. They used to render the
// `--verbose` diagnostics instead: every recipe with its aliases,
// engine recipes, flags and policy notes, measured at 204,470
// characters (~58k tokens) on a developer host against 864 characters
// for plain `rocm model`. That is more than a locally served model's
// whole context, so the one tool call a small model reaches for when
// asked what it can run ended the turn with a provider error. The
// verbose dump is still reachable, but only when it was asked for.
[command]
if command.eq_ignore_ascii_case("model") || command.eq_ignore_ascii_case("models") =>
{
Ok(render_model_registry_text_with_context_and_host(
Some(paths),
None,
None,
))
}
[command, flag]
Comment on lines 511 to +519

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this dispatch change silently disables the CLI chat's deterministic "ROCm CLI summary" block for model questions.

run_rocm_read_only_in_process's ["model"]/["models"] arm now returns render_model_registry_text_with_context_and_host's curated output. Its header comes from model_catalog_header (apps/rocm/src/model_catalog_text.rs), which prints "Recommended models - run one with \rocm serve `"(or the signed-index variant) — never the string"model recipes". Only the verbose renderer (render_model_registry_verbose_text_with_context_and_host) still emits writeln!(output, "model recipes")`.

deterministic_model_tool_summary (apps/rocm/src/chat/summaries.rs:320-321) is strictly gated on tool_text.contains("model recipes"). run_chat_read_only_tool's "rocm_command" arm (this file, ~line 708) feeds exactly this dispatch's output into turn.rs's follow_up_text, which is then passed straight to deterministic_chat_tool_summary (apps/rocm/src/chat/turn.rs:326).

So on rocm chat --prompt --tools — a sibling surface this same function serves — the "ROCm CLI summary" block (Recommended local assistant / Fit / Engine) that used to render whenever the model asked what it could run no longer appears, not because the feature was removed but because the one string it keys on is no longer produced by the new default path. Every existing test of deterministic_model_tool_summary/deterministic_chat_tool_summary (e.g. deterministic_model_tool_summary_suppresses_extra_local_model_follow_up in turn.rs, and the fixtures in summaries.rs) hand-writes a string containing "model recipes" literally rather than calling the real renderer or dispatch, so none of them catch this; no e2e scenario exercises it either.

Confidence 88 — traced end to end at the PR head commit: confirmed the new default output never contains the gate string, confirmed the consumer's exact gate, and confirmed the call chain connecting this dispatch to that consumer.

Fix: either make the summary recognize the curated output too (key off a stable marker both renderers share, or off the tool name/args rather than parsing renderer-specific text), or if this deterministic summary is deliberately being retired for model/models, remove it (and its now-stale fixtures) and say so in the PR description.

if (command.eq_ignore_ascii_case("model")
|| command.eq_ignore_ascii_case("models"))
&& flag.eq_ignore_ascii_case("--verbose") =>
{
Ok(render_model_registry_verbose_text_with_context_and_host(
Some(paths),
Expand Down Expand Up @@ -1177,6 +1198,81 @@ mod tests {
assert!(text.lines().any(|line| line == version_line), "{text}");
}

/// What the chat `model` tool hands back has to be what `rocm model`
/// prints, not the `--verbose` diagnostics. The verbose dump is every
/// recipe with its aliases, engine recipes, flags and policy notes; on a
/// developer host it measured 204,470 characters against 864 for plain
/// `rocm model`, which is more than a locally served model's whole
/// context. Asking a small model "which LLMs can this machine support"
/// makes it call exactly this, so the one question the tool exists for
/// used to end in a provider context-overflow error.
#[test]
fn chat_model_tool_returns_the_curated_list_rocm_model_prints() {
let (_root, paths) = test_paths("mcp-model-curated");

let result = run_internal_mcp_call(
&paths,
"rocm_command",
serde_json::json!({ "args": ["model"] }),
false,
)
.expect("read-only rocm_command model should run");
let text = mcp_tool_result_text(&result);

// The same recipes, from the same renderer `rocm model` calls.
let curated = render_model_registry_text_with_context_and_host(Some(&paths), None, None);
assert!(
text.contains(curated.trim()),
"the tool must return what `rocm model` prints:\n{text}"
);
// And not the per-recipe diagnostics, which are what made it huge.
assert!(
!text.contains("aliases=["),
"the verbose dump must not be the default:\n{text}"
);
// Comfortably inside the dash's 16 KiB per-result budget
// (`TOOL_RESULT_CHAR_BUDGET` in `crates/rocm-dash-tui/src/agent/tools.rs`),
// so a chat turn carrying it is never truncated on the built-in
// catalog. Asserted here rather than imported so the bin does not
// depend on a dash constant.
assert!(
text.len() < 16 * 1024,
"the curated list is {} characters, past the dash budget",
text.len()
);
}

/// The diagnostics are still reachable — the model has to ask for them,
/// the same way a person types `rocm model --verbose`.
#[test]
fn chat_model_tool_still_renders_the_verbose_dump_on_request() {
let (_root, paths) = test_paths("mcp-model-verbose");

let result = run_internal_mcp_call(
&paths,
"rocm_command",
serde_json::json!({ "args": ["model", "--verbose"] }),
false,
)
.expect("read-only rocm_command model --verbose should run");
let text = mcp_tool_result_text(&result);

assert!(text.contains("model recipes"), "{text}");
assert!(
text.contains("aliases=["),
"--verbose must still render the per-recipe diagnostics:\n{text}"
);
// The two really are different sizes; without this the test above
// could pass against a renderer that lost its diagnostics entirely.
let curated = render_model_registry_text_with_context_and_host(Some(&paths), None, None);
assert!(
text.len() > curated.len(),
"the verbose dump ({}) should be longer than the curated list ({})",
text.len(),
curated.len()
);
}

#[test]
fn natural_language_plan_returns_structured_mutating_action() {
let (_root, paths) = test_paths("nl-plan-structured-mutating");
Expand Down
Loading
Loading