Skip to content

fix(dash): bound chat tool output so a small-context model can answer - #1041

Merged
r0x0r merged 5 commits into
ROCm:mainfrom
r0x0r:fix/dash-chat-tool-output-cap
Nov 16, 2026
Merged

r0x0r merged 5 commits into
ROCm:mainfrom
r0x0r:fix/dash-chat-tool-output-cap

Conversation

@r0x0r

@r0x0r r0x0r commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Asking the rocm dash Chat tab anything that makes the model call rocm_command with
["model"] ended the turn with the provider's raw HTTP 400 instead of an answer:

err  chat request failed: CompletionError: HttpError: Invalid status code 400 Bad Request with message:
{"error":{"code":400,"message":"request (53820 tokens) exceeds the available context size (32768 tokens),
try increasing it","type":"exceed_context_size_error","n_prompt_tokens":53820,"n_ctx":32768}}

The tool result measured 204,470 characters — roughly 58k tokens, more than a locally
served model's entire context. llama.cpp defaults n_ctx to 32768, so any local model
with a context of 64k tokens or less reproduces it. The user got the raw provider error
and no answer.

Four things compounded, and all four are addressed.

What changed

The chat model/models tool rendered the wrong thing. It called the
rocm model --verbose diagnostics renderer — every recipe with its aliases, engine
recipes, flags and policy notes — rather than the curated list rocm model prints. It now
renders the curated list; the diagnostics need an explicit ["model", "--verbose"].

The output was sent twice. The MCP envelope carries a command's output as the content
text and again as structuredContent.stdout, and the dash passed the whole JSON value to
the model. The envelope itself is left alone — rocm mcp-call consumers read the
structured field, and an in-tree e2e step does exactly that — so the duplicate is dropped
in the dash, where the model is the only reader.

Nothing bounded a tool result at all. A read-only result is now capped at 16 KiB of
serialized JSON, cut on a line boundary and closed with a marker naming how much was
omitted, so neither the model nor the user can take a truncated list for a complete one.
The budget is checked against the actually-serialized string, so JSON escaping cannot push
a capped result back over it. Capping replaces the envelope with a smaller object, so
isError is carried over explicitly: a command that failed with more output than the
budget allows must not reach the model looking like a success, since that field is what
the chat path reads to decide.

The example steered small models at the worst case. The rocm_command description and
its args schema both used ["model"], which a small local model copies literally; they
now use ["config", "show"].

Separately, a provider context overflow now reads as a plain sentence in the Chat tab
naming the token counts and a way forward, with the raw provider body still logged to
rocm-cli.log.

How it was checked

The headline number, measured on one binary with only the renderer swapped between runs:

call before after
rocm mcp-call rocm_command {"args":["model"]} 189,373 B 1,915 B
{"args":["model","--verbose"]} 189,373 B 189,385 B
plain rocm model, for comparison 857 B 857 B

A 99× reduction on the default path, with the diagnostics still reachable.

Mutation evidence — every new check was shown to fail against a deliberately broken input,
and its siblings stayed green, so the checks are independent rather than coupled:

Broken input Check that went red
the cap/dedup call short-circuited to pass-through oversized-result test (210220 characters reached the model)
the dedup helper fed empty text duplicate-copy test
chat_error_message returns the raw body both context-overflow tests
model dispatch pointed back at the verbose renderer curated-list unit test and the model-catalog-04 scenario

Gates, all run on a Linux box because this workspace does not build on macOS (four
pre-existing Darwin type errors in an unrelated crate, present on main):
cargo test --workspace --all-targets --exclude e2e-cucumber (58 test binaries, 0 failed),
cargo clippy --workspace --all-targets -- -D warnings (clean), python scripts/smoke_local.py
(smoke: ok), and cargo xtask e2e -- -n model-catalog (all scenarios pass, including the
two new ones).

Not verified: no live dash run against a real 32k-context model. The claim that a turn
now gets an answer instead of HTTP 400 rests on the measured size reduction plus the
unit-tested cap, not on an end-to-end chat. docs/manual-testing.md §13 is written so a
human can close that, including a --ctx-size 2048 case. The truncation path itself was
exercised with a synthetic 200 KB fixture — after this change no read-only tool on the test
host produces an oversized result.

Scope

The cap is dash-only, and that is deliberate. model_facing_tool_result is applied
inside the dash's run_rocm_read_tool, which is the single funnel every read-only chat
tool result passes through. The CLI's own tool-using path (rocm chat --prompt --tools)
reads mcp_tool_result_text straight through with no budget and is not capped here.
The curated-vs-verbose split is shared between both paths, so the one reported symptom is
fixed on both — but a different large result on the CLI path (["logs"] on a busy host, or
an explicit ["model","--verbose"]) can still reach a small-context model unbounded. That
exposure is pre-existing rather than introduced, and capping a second surface is a larger
change than the reported bug needs. It is stated in docs/llm-tool-use.md rather than left
for a reader to discover, and is the obvious follow-up.

The 16 KiB budget is fixed, not context-aware. It leaves room in a 32k context for the
system prompt plus several turns, and matches llama.cpp's common default with headroom. A
budget derived from the provider's reported n_ctx would serve a --ctx-size 2048 host
and a very large context more precisely, at the cost of plumbing that value through to the
tool layer. The rewritten overflow message is the second line of defence for what a fixed
budget misses. Flagging it as a judgement call rather than an oversight.

Out of scope: the base cost per turn (roughly 4,200 tokens of system prompt plus the tool
schemas) is heavy for 1.5B-class models, and is a separate concern.

Testing notes

The dash TUI's chat path has no Gherkin coverage, but the behaviour behind this change is
observable from the CLI: rocm mcp-call rocm_command is the same tool dispatch the Chat tab
drives, so the two new scenarios assert there rather than leaving it to unit tests alone —
the same shape as the existing service_logs coverage. The cap itself and the overflow
message remain unit-only; there is no CLI surface that exposes them.


  • If this PR fixes a bug, searched tests/e2e-cucumber/expectations.toml for the fixed ticket ID and removed/narrowed any now-stale xfail rows. — no xfail row covers this behaviour; nothing to narrow.
  • If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per docs/architecture.md. — n/a, no new subcommand or subsystem.
  • If this PR adds, renames or removes a production source file, docs/architecture.md has the matching entry. — n/a, no production source file added, renamed or removed; the two docs/architecture.md edits refresh descriptions of files whose role changed.
  • Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state — not only the wording — per AGENTS.md §3. — two new messages. The truncation marker names how much was omitted and is asserted by the oversized-result test against the value actually handed to the model. The context-overflow sentence is asserted by two tests, one of which also asserts the raw provider body is still logged, so the remediation advice and the logging both hold rather than just the wording.

Asking the dash Chat tab anything that makes the model call
`rocm_command` with `["model"]` ended the turn with the provider's raw
HTTP 400 instead of an answer: the tool result measured 204,470
characters (~58k tokens), more than a locally served model's entire
context (llama.cpp defaults `n_ctx` to 32768).

Four things compounded, and all four are addressed.

The chat `model`/`models` tool rendered the `rocm model --verbose`
diagnostics — every recipe with its aliases, engine recipes, flags and
policy notes — rather than the curated list `rocm model` prints. It now
renders the curated list; the diagnostics need an explicit
`["model", "--verbose"]`.

The MCP envelope carries a command's output twice: as the content text
and again as `structuredContent.stdout`. `rocm mcp-call` consumers read
the structured field, so the envelope is unchanged and the duplicate is
dropped in the dash instead, where the model is the only reader.

Nothing bounded a tool result at all. A read-only result is now capped
at 16 KiB of serialized JSON, cut on a line boundary and closed with a
marker naming how much was omitted, so neither the model nor the user
can take a truncated list for a complete one.

The `rocm_command` description and its `args` schema both used
`["model"]` as the example, which a small local model copies literally;
they now use `["config", "show"]`.

Separately, a provider context overflow now reads as a plain sentence in
the Chat tab naming the token counts and a way forward, with the raw
provider body still logged to rocm-cli.log.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The dash TUI's chat path has no Gherkin coverage, but the change behind
it is observable from the CLI: `rocm mcp-call rocm_command` with
`["model"]` is the same tool dispatch the Chat tab drives, so the
scenarios assert there rather than leaving the behaviour to unit tests
alone. One pins the recommendation list and its size, the other that the
per-recipe diagnostics are still reachable on request.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…iled

Capping an oversized read-only tool result replaces the envelope with a
fresh object, so anything the chat path reads off it has to be carried
over deliberately. `isError` was not, which meant a command that failed
with more output than the budget allows reached the model looking like a
success — `turn.rs` reads exactly that field to decide.

Carry it over when the envelope had one, and leave it absent when it did
not, so the shape still mirrors its source. Two tests pin both
directions; both fail if the carry is dropped.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r
r0x0r marked this pull request as ready for review November 12, 2026 13:30
@r0x0r
r0x0r requested a review from a team as a code owner November 12, 2026 13:30
@r0x0r
r0x0r requested a review from tomastola November 12, 2026 13:30

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through this diff and found no outright bugs, and CI is green apart from the known pending nightly GPU lane. I'm requesting changes because a few things should be fixed or consciously accepted before merge; details are inline.

  • The overflow message claims a ROCm check result caused it, which it can't know (history or a large paste trigger the same error), so the advice may not help. Overflow detection also only recognises the llama.cpp error code.
  • The dedup can drop a short, genuinely different stderr/stdout because it uses a substring check.
  • The 16 KiB budget is described in characters but measured in bytes.
  • The manual-testing doc uses --arguments-file, which CI never exercises; please confirm it exists.

Smaller points, no inline comment: the cap is per result, not per turn, so several large results in one turn can still overflow a 32k context. It covers only the dash read-only path; the mutating/approved dash path is uncapped and the PR body doesn't mention that. The 16 KiB literal is also repeated in three places without a shared constant. The unit tests, the e2e scenarios and the failure-path handling (isError carried over on truncation) look good.

})
.unwrap_or_default();
Some(format!(
"A ROCm check returned more text than this model can read{counts}. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence says a ROCm check result caused the overflow, but exceed_context_size_error also fires when accumulated history or a large paste is the cause. In that case "try a more specific question" won't clear it. Please word it so it doesn't assert the cause, and suggest starting a new chat.

/// user can act on. The token counts come out of the same body; a body that
/// omits them still gets the sentence, without the numbers.
fn context_overflow_message(raw: &str) -> Option<String> {
if !raw.contains("exceed_context_size_error") {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the llama.cpp error type is recognised here; other OpenAI-compatible servers/providers use different codes and will still show the raw 400. Fine if intended, but please say so in the doc comment (the PR body talks about "a provider context overflow").

.and_then(Value::as_str)
.is_some_and(|s| {
let trimmed = s.trim();
trimmed.is_empty() || text.contains(trimmed)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

text.contains(trimmed) can drop a short, non-duplicated stderr/stdout (e.g. error, 0) that merely appears somewhere in the text. Consider requiring the duplicate to be the tail of the text, or a minimum length.

/// against the *serialized* value, which is what actually reaches the model,
/// so JSON escaping cannot push it back over.
fn cap_tool_result(value: Value, budget: usize) -> Value {
if value.to_string().len() <= budget {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The budget is compared against serialized bytes (.len()), while the constant, docs and marker speak of characters. Same for ASCII only; please either rename/document it as bytes or count chars.

Comment thread docs/manual-testing.md

```bash
echo '{"args":["model"]}' > args.json
rocm mcp-call rocm_command --arguments-file args.json | wc -c

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The e2e step uses mcp-call ... --arguments-json; this uses --arguments-file. Please confirm that flag exists, since CI doesn't exercise it.

@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Two of these checked against the code — both confirmed

@juhovainio — I have not changed code here yet; this is verification of the two claims that were directly checkable, so the facts are settled before anyone works the list.

"The 16 KiB budget is described in characters but measured in bytes" — confirmed

Exactly right, and it leaks into user-visible text as well as the cap itself.

cap_tool_result's doc says "Bound a tool result to budget characters of serialized JSON", but the test is:

if value.to_string().len() <= budget

String::len() is bytes in Rust, not chars. And truncation_marker then reports those same byte figures as characters:

"[output truncated: {shown_chars} of {total_chars} characters shown, {omitted} omitted. …]"

So on any non-ASCII tool output — a model name with an em dash, a path with accented characters, anything CJK — the budget is tighter than documented and the message tells the model and the user a byte count while calling it a character count. The variable names (shown_chars, total_chars) carry the same mistake into the code.

Worth noting the sibling comment at mcp_bridge.rs:1233 has it right in substance — text.len() < 16 * 1024 with a comment about the "16 KiB per-result budget" — so the two places disagree about what the unit is.

"--arguments-file — please confirm it exists" — it does, with a caveat

It exists: apps/rocm/src/cli.rs:305, on the mcp-call command, #[arg(long, conflicts_with = "arguments_json")].

The caveat is that mcp-call is declared #[command(name = "mcp-call", hide = true)] — a hidden internal command. So the flag is real and the manual-testing step will work, but it is routing a tester through a command that does not appear in --help. That is a reasonable thing to do in a manual-testing doc; it is worth a word saying so, since a tester who tries to discover it from the CLI will not find it.

The other two, and the smaller points

I have not verified or fixed:

  • The overflow message claiming a ROCm check caused the overflow. Your reasoning looks right to me on reading — history or a large paste reach the same error — and "only recognises the llama.cpp error code" is the kind of narrow detection that silently stops working against another backend. Unchecked.
  • The dedup substring check dropping a short, genuinely different stderr/stdout. Also unchecked.
  • The per-result rather than per-turn cap (several large results in one turn still overflowing a 32k context), the uncapped mutating/approved dash path, and the 16 KiB literal repeated in three places without a shared constant — all unaddressed. The third would have prevented the chars/bytes divergence above, since there would have been one place to describe the unit.

Leaving CHANGES_REQUESTED standing; nothing here is fixed yet.

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · head 74c835d

This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.

Review — no blocking findings

Full review of the whole change (9 files, 859 insertions / 10 deletions). No ticket named in the PR description.

Blocking

None

Non-blocking

  • TOOL_RESULT_CHAR_BUDGET and its docs describe the cap in "characters", but it is enforced in UTF‑8 bytes — crates/rocm-dash-tui/src/agent/tools.rs:251 (constant + its doc comment at :239‑250), :366 and :391‑394 (cap_tool_result's fit checks all use value.to_string().len(), which is Rust's byte length), and docs/llm-tool-use.md:80-81 ("bounded to TOOL_RESULT_CHAR_BUDGET characters"). The enforcement is internally consistent (bytes are compared against bytes throughout), so this isn't a functional break for the ASCII-heavy CLI output these tools actually produce, but the name and both the code comment and the doc describe a character budget that the code doesn't actually implement — multi-byte UTF‑8 content would be cut well before 16 KiB of characters. Confidence 80 · mechanical · Fix: rename to a byte-oriented name (or doc it explicitly as bytes), or switch the fit check to a char count if "characters" is the intended contract.
  • cap_tool_result's binary search can return a value that exceeds budget, contradicting its own doc comment — crates/rocm-dash-tui/src/agent/tools.rs:365-404. best is initialized to 0 unverified (:388); if the very first probed mid is 0 and the minimal (marker-only) truncated form already exceeds budget, the loop takes the mid == 0 → break branch (:398‑399) without ever confirming best fits, and the function then returns that unverified, over-budget value. This requires a budget smaller than the truncation marker's own JSON overhead (roughly 200+ bytes), which never happens with the real TOOL_RESULT_CHAR_BUDGET = 16 * 1024, so it's dormant today, not an observed break. Confidence 80 (traced precisely), but non-blocking since unreachable with the shipped constant · logic · Fix: assert/handle the "doesn't fit even empty" case explicitly, or verify cuts[0] fits before relying on best's initial value.
  • The dash's mutating/approval-replay tool-result path is not routed through the new cap at all — confirmed by grep: crates/rocm-dash-tui/src/app/tool_approval.rs has zero references to model_facing_tool_result/TOOL_RESULT_CHAR_BUDGET. An approved mutating command (e.g. an install with a long log) can still hand the model an unbounded result and reproduce the exact HTTP‑400 symptom this PR fixes for the read-only funnel. The PR's own Scope section discloses one remaining uncapped surface (the separate CLI chat path) but not this second one, which sits in the very crate and feature this PR touches. Confidence 85 · architectural · Fix: either apply the same cap in the approval-replay path, or disclose it the same way the CLI-path gap is disclosed.
  • The cap bounds one result, not one turn — crates/rocm-dash-tui/src/agent/tools.rs:453 calls model_facing_tool_result once per individual tool call inside run_rocm_read_tool. Nothing aggregates across multiple tool calls returned within the same model turn, so several moderately-sized results (each comfortably under 16 KiB on its own) can still collectively approach or exceed a small model's context in one round trip. Confidence 70 (reasoned from the call site; not traced through the rig agent loop to confirm multi-tool-call turns are possible) · architectural · Fix: note as a known limit, or track turn-level token usage if the agent loop can issue multiple tool calls per turn.
  • The rewritten overflow message always attributes the cause to "A ROCm check" — crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79. context_overflow_message detects the failure purely by matching "exceed_context_size_error" in the raw error string; it has no signal about why the prompt grew too large. The same provider error fires just as easily from accumulated conversation history across many turns with no oversized tool result in the offending turn, in which case the message's specific remediation ("a ROCm check returned more text...") misattributes the cause. Confidence 75 · logic · Fix: soften to a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read") unless the call site can actually distinguish the two cases.

Decisions for the author

  • Dash-only cap, CLI chat path left uncapped — tradeoff, already explicitly disclosed and reasoned in the PR description's Scope section and in docs/llm-tool-use.md. Recorded here for completeness; no new concern.
  • Fixed 16 KiB budget instead of a context-aware one derived from the provider's reported n_ctx — tradeoff, already explicitly disclosed and reasoned in the PR description's Scope section. Recorded here for completeness; no new concern.

Positive signals

  • The line-boundary binary search in cap_tool_result is a careful piece of engineering: it guarantees the serialized truncated value fits the byte budget (not an estimate), and cutting only at line boundaries keeps the kept text readable rather than chopping mid-line.
  • The PR description's mutation-evidence table (each new check shown to fail against a specific deliberately-broken input) is good testing discipline and made several of the new tests easy to trust on inspection alone.

Deployment notes

The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands.

What this covered

Read the complete diff (f536619a08a3b50dd26aac5c21a3bb9f6bd266d2...74c835dffbbebb009023e74b9697a2c0cf3baec6) plus the complete content of every changed file at the PR head, plus surrounding/calling code read in full: apps/rocm/src/chat/mcp_bridge.rs's whole dispatch match block, apps/rocm/src/model_catalog_text.rs (the curated/verbose renderers), apps/rocm/src/chat/turn.rs and apps/rocm/src/logging.rs (to verify the CLI-path-uncapped claim and production log routing), crates/rocm-dash-tui/src/app/tool_approval.rs, and the root AGENTS.md. Verified CI via gh pr view --json statusCheckRollup: all checks green except two GPU-hardware lanes (MI300X, W7900) still queued at review time, not failed.

This review initially fanned out five sub-tasks (one per coupled file group) to run as independent blind passes, per method. None of their agent IDs ever appeared as present/completed in this session's own agent listing, and a purported relay of their findings arrived instead via a mid-task message from a third party; that relay's content cited source-file line numbers that did not match the actual files at this head commit, so it could not be authenticated as genuine subagent output and none of it was used. Every claim in this report was instead independently re-derived and verified directly against the source by the reviewing context itself (full file reads, git grep, and hand-traced logic), which is closer to a single-context pass than a verified multi-worker fan-out for the code-correctness groups, even though each finding above was individually confirmed against the code before inclusion. Did not run: the actual cargo test/cargo clippy suite locally (relied on the PR's own green CI); no live mutation testing (reverting code paths and re-running tests) was executed — test-quality verdicts above were reached by hand-tracing. Not reconciled against the PR's existing review thread; this report was produced blind from the code and the PR's own description only.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

The remaining lane is not going to start: the self-hosted runner pool is down

This PR has no failures left — the only thing outstanding is a GPU lane that has been queued for hours. That is not backlog. Querying the repo's self-hosted runners:

runner status busy labels
mi300x-0 online true mi300x
mi300x-1 online true mi300x
w7900-0 offline false w7900
strix-halo-ubuntu-2 offline false strix-halo, ubuntu-26.04
mi350p-0 offline false mi350p
strix-halo-windows-2 online false strix-halo, windows-11

Three of the six are offline. w7900-0 being down means every E2E tests (W7900, current) lane stays queued indefinitely — there is no runner that can claim it.

The MI300X pair is the stranger case: both report busy = true, yet no job anywhere in this repo is in progress on them. I walked the last 25 runs; the only in-progress jobs are on GitHub-hosted runners. So GitHub will not dispatch to them (it believes they are occupied) while nothing in this repo is using them. There are currently ~10 E2E self-hosted runs queued behind this, the oldest from 16:17 UTC, and they include runs on main — so it is repo-wide and not specific to my branches.

Nothing about this PR can clear it; it needs someone with runner access to restart the MI300X agents and bring w7900-0 back. Recording it here so the permanently-pending lane is not read as this change waiting on a result that is coming.

@r0x0r

r0x0r commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

MI300X: the known broken prewarm runtime, and a correction about the runners

The MI300X lane finished and is red with eight unexpected failures. They are the environmental breakage already documented on #1036 and #1054, and this run makes the case conclusive.

Reconciliation: 4 xfail, 2 XPASS, 8 unexpected failure(s).
FAIL: 'bench-load-real-serve'
FAIL: 'chat-end-to-end-local-model'
FAIL: 'runtime-lifecycle-activate-restart-services-succeeds'
FAIL: 'serve-anthropic-endpoint-advertised'
FAIL: 'serve-anthropic-endpoint-readiness'
FAIL: 'serve-anthropic-messages-inference'
FAIL: 'serve-anthropic-messages-streaming'
FAIL: 'serve-vllm-inference'

Eleven occurrences of RuntimeError: operator torchvision::nms does not exist, raised while vLLM imports torch inside the prewarmed wheel runtime — a torch/torchvision ABI mismatch in the runners' prewarm tree.

Why this run settles it. The same eight scenarios, byte-identical, have now failed on three PRs with three different bases:

PR subject base
#1036 Vulkan probing in examine the #678 stack
#1054 APU memory-pool measurement main
#1041 (this one) chat tool-output cap in rocm-dash-tui main

This PR touches rocm-dash-tui's chat path and nothing in serving, vLLM, or runtime resolution. Three unrelated diffs cannot produce one identical failure set; the prewarm tree is the common cause. It needs rebuilding with a torchvision matching the pinned torch — nothing pushed to this branch will clear it.


Separately, a correction to my previous comment here. I said both MI300X runners reported busy = true while no job in the repo was running on them, and inferred they were wedged. That was wrong. mi300x-0 was running this very job from 21:18 to 22:37 UTC. My check walked only the 25 most recent runs, and this job's parent run was created at 11:18 — outside that window, and not in the newest 60. The run list is ordered by creation, so a long run still working through the queue drops off the end while staying active.

The offline runners from that comment are still offline (w7900-0, strix-halo-ubuntu-2, mi350p-0, and now queue-report-0), so the W7900 lane here still has no runner that can claim it. But the MI300X side was never stuck — it is just slow: ~80 minutes per job, two runners, a deep queue.

…tput-cap

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>

# Conflicts:
#	docs/manual-testing.md
@r0x0r

r0x0r commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Merged the latest main into this branch to clear the conflict. This is a merge only — no review feedback is addressed here.

What the merge touched

The only conflicting file was docs/manual-testing.md. Both sides appended a new top-level section at the end of the file; main had grown through ## 13. Pasting An Error Into rocm diagnose, so this branch's new section was renumbered to follow it as ## 14. Dashboard Chat With A Small-Context Local Model. No other content from either side was dropped.

Relative to main, this branch still contributes exactly the same nine files it did before:

  • apps/rocm/src/chat/mcp_bridge.rs
  • crates/rocm-dash-tui/src/agent/tools.rs
  • crates/rocm-dash-tui/src/app/chat_runtime.rs
  • docs/architecture.md
  • docs/llm-tool-use.md
  • docs/manual-testing.md
  • skills/rocm-cli-assistant/SKILL.md
  • tests/e2e-cucumber/features/model_catalog.feature
  • tests/e2e-cucumber/tests/e2e/model_catalog_steps.rs

Numbering and references checked

  • model_catalog.feature — main added no scenarios here, so model-catalog-04 and model-catalog-05 keep their positions and their @id: tags are unchanged. cargo test -p e2e-cucumber --test feature_naming passes, which enforces that scenario indexes match file position and are unique suite-wide.
  • docs/manual-testing.md — the new ## 14 is unique, and ### 4d. Dashboard Chat Request Timeout is intact, so the // ... docs/manual-testing.md section 4d comment in crates/rocm-dash-tui/src/agent/mod.rs still points at the right place. (The duplicate ## 11 is pre-existing on main and untouched here.)
  • docs/architecture.md — only two existing rows' description cells changed; Class columns intact, no rows added or duplicated.

Gates (all green, on Linux)

Gate Result
cargo fmt --all --check pass
cargo clippy --workspace --all-targets -- -D warnings pass
cargo clippy -p e2e-cucumber --test e2e -- -D warnings pass
cargo test -p e2e-cucumber --test feature_naming 4 passed
cargo xtask check-architecture-doc pass
cargo xtask check-crate-edges pass
cargo xtask check-module-size pass (424 files, none over 1500 lines)
cargo test -p rocm-dash-tui pass
cargo test -p rocm --bin rocm 1864 passed, 0 failed

Still outstanding

The CHANGES_REQUESTED items from the review are not addressed by this merge and still stand:

  1. The budget is measured in chars vs. bytes — String::len() returns bytes.
  2. The substring dedup can drop a short, non-duplicated stderr.
  3. The overflow message attributes a cause it cannot actually know.
  4. The e2e step uses --arguments-json while the code implements --arguments-file.

@nowycondro nowycondro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · head 9d73961

This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.

Review — no blocking findings

Full review of the whole change. No ticket named in the title or commits; the description is an exact, verifiable account of what the diff does.

Blocking

None

Non-blocking

  • cap_tool_result's binary search can return an unverified cut when even the empty-content form doesn't fit the budget — crates/rocm-dash-tui/src/agent/tools.rs (cap_tool_result, truncated_tool_result)
    best starts at 0 and is only advanced once a candidate is measured to fit the budget. If truncated_tool_result(&full, cuts[0], ...) — empty kept text plus the truncation marker, a few hundred bytes of JSON — itself exceeds budget, the mid == 0 branch breaks without ever validating cuts[best], so the function can return a value whose length was never checked against budget, contradicting its own doc comment ("best is only ever set to a cut that was measured as fitting, so the returned value is within budget"). This is not reachable today: the sole caller (model_facing_tool_result, from run_rocm_read_tool) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so the branch can't currently trigger, and no test exercises a small/variable budget. Confidence 70 — traced the logic, not reproduced with a failing input. Fix: check cuts[0] explicitly before the loop (or assert the budget is large enough for the marker) so a future caller with a smaller or configurable budget can't silently get an over-budget result.

Decisions for the author

  • Only the dash's tool-result path is capped/de-duplicated and rewritten into plain language on context overflow; the CLI chat path (rocm chat --prompt --tools) is not — tradeoff
    The curated-vs-verbose model-listing split is shared, so the specific reported bug is fixed on both paths, but the CLI path still passes an unbounded result straight through (mcp_tool_result_text, no budget) and still surfaces a provider's raw error body verbatim on overflow (no chat_error_message-equivalent in apps/rocm/src/chat/turn.rs). Both asymmetries are explicitly named in the PR description and in docs/llm-tool-use.md as a deliberate scope cut rather than an oversight, with the CLI path called "the obvious follow-up" — worth confirming that's still the intended boundary, since the CLI drives the exact same rocm_command dispatch and can hit the same small-context failure mode.
  • The 16 KiB budget is a fixed constant rather than derived from the provider's reported n_ctx — tradeoff
    A context-aware budget would fit a --ctx-size 2048 host, or a much larger one, more precisely, at the cost of plumbing the provider's context size through to the tool layer. The PR description frames the rewritten overflow message as the fallback for whatever the fixed budget misses, and calls this out explicitly as a judgement call.
  • The context-overflow rewrite only runs on the dash's dispatch edge; the CLI chat path still shows the raw provider body on the same failure — non-blocking-improvement
    A natural follow-up is giving the CLI chat loop the same plain-language rewrite chat_error_message gives the dash, so a user of rocm chat --prompt --tools doesn't see a raw exceed_context_size_error JSON body where the dash now shows a sentence.

Positive signals

  • model_facing_tool_result is applied uniformly to every read-only tool result in run_rocm_read_tool, not special-cased to the model command, so any other command returning a very large result (e.g. logs on a busy host, called out in docs/manual-testing.md) gets the same protection for free.
  • The truncation cut is computed against a plain-text rendering at a line boundary, then re-serialized and measured against the actual JSON-escaped length before being accepted — avoids truncating mid-escape or mid multi-byte character.
  • isError is explicitly carried over into the truncated form and tested in both directions, so a long-but-failed command can't silently read as a success once truncated.
  • The PR description's mutation-style test evidence (each check shown to fail against a deliberately broken input) checks out against the actual tests, e.g. truncating_a_failed_command_keeps_it_a_failure / truncating_a_successful_command_keeps_it_a_success.

Deployment notes

No migration or configuration change. The model/--verbose split and the dedup/cap change the shape of results any external MCP client already gets from rocm mcp-call/the dash: structuredContent.stdout/.stderr now disappear when they duplicate the text content, and a result over 16 KiB now arrives as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope. This is documented in docs/llm-tool-use.md, but any external client that parses the full envelope unconditionally should be aware of it.

What this covered

Full diff of ROCm/rocm-cli PR #1041 at head 9d73961 (base e2e5c5b…9d73961): apps/rocm/src/chat/mcp_bridge.rs, crates/rocm-dash-tui/src/agent/tools.rs, crates/rocm-dash-tui/src/app/chat_runtime.rs, docs/architecture.md, docs/llm-tool-use.md, docs/manual-testing.md, skills/rocm-cli-assistant/SKILL.md, tests/e2e-cucumber/features/model_catalog.feature, tests/e2e-cucumber/tests/e2e/model_catalog_steps.rs — each read in full, plus apps/rocm/src/main.rs's rocm model CLI dispatch and the relevant sections of AGENTS.md (module-size limit, assistant tool-use consistency) for cross-reference.

apps/rocm/src/chat/mcp_bridge.rs and crates/rocm-dash-tui/src/agent/tools.rs were reviewed by an independent fan-out worker. A second worker was requested for crates/rocm-dash-tui/src/app/chat_runtime.rs, the e2e feature/step files, and the docs changes, but every spawn attempt failed on a shared concurrency limit; that group was reviewed directly in this same context instead and so was not independent of the rest of this report.

CI showed every completed check green (clippy, build-and-test, affected-crate tests, E2E tests, ratcheted coverage, skillscope); a few long-running hardware/E2E jobs were still queued or in progress at review time with nothing failed.

Did not run: no separate git blame/history pass or prior-review-comment pass beyond what's folded into the reading above; no test suite executed locally (relied on the green CI run for the reviewed commit).

Not reconciled against any discussion on the PR — this review did not read existing comments or review threads.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · head 9d73961

This automation never files a GitHub approval that blocks or substitutes for human review — the merge decision stays with a human reviewer; treat this as a second pair of eyes, not a sign-off.

Review — no blocking findings

Full review of the whole change (9 files, 859 insertions / 10 deletions). No ticket named in the title or description; the description is an exact, verifiable account of what the diff does — every claim I checked against the code held up.

Blocking

None

Non-blocking

  • The context-overflow message always attributes the cause to "A ROCm check returned more text than this model can read" — crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79 (message text at :76). context_overflow_message detects the failure purely by matching exceed_context_size_error in the raw provider error string; it has no signal about what actually filled the context this turn. The same provider error fires just as easily from history accumulated across many ordinary turns, or a large pasted message, with no oversized tool result involved, in which case the causal half of the sentence is wrong — though the remedy half ("try a more specific question, or a model with a larger context") is still reasonable generic advice either way. Confidence 75 · logic · Fix: a cause-neutral phrasing, or confirm the simplification is acceptable.
  • TOOL_RESULT_CHAR_BUDGET is named and documented as a character budget but enforced in UTF-8 bytes — crates/rocm-dash-tui/src/agent/tools.rs:251 (constant + doc comment), :365-404 (cap_tool_result's fit checks all use value.to_string().len(), Rust's byte length), and docs/llm-tool-use.md:80-81 ("bounded to TOOL_RESULT_CHAR_BUDGET characters"). Internally consistent (bytes compared to bytes throughout), so no functional break for the ASCII-heavy CLI output these tools actually produce — but multi-byte UTF-8 content would be cut before 16 KiB of actual characters. Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document it explicitly as bytes.
  • cap_tool_result's binary search can return an unverified cut when even the empty-content form doesn't fit the budget — crates/rocm-dash-tui/src/agent/tools.rs:365-404. best starts at 0 and is only confirmed once a candidate is measured to fit; if cuts[0] (empty kept text plus the marker's own JSON overhead, ~200+ bytes) itself exceeds budget, the loop reaches mid == 0, breaks without ever validating cuts[0], and the function returns it anyway — contradicting the function's own doc comment that the result is always within budget. Dormant today: the sole caller (model_facing_tool_result) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above that overhead, so it can't trigger with the shipped constant. Confidence 80 (traced precisely, not reproduced with a failing input) · logic · Fix: validate cuts[0] fits before trusting best's initial value, relevant only if a future caller passes a smaller/configurable budget.

Decisions for the author

  • Dash-only cap; the CLI's own tool-using path (rocm chat --prompt --tools) reads mcp_tool_result_text straight through with no budget and shows the raw provider error on overflow — tradeoff, explicitly disclosed in the PR description's Scope section and in docs/llm-tool-use.md; independently confirmed by reading apps/rocm/src/chat/turn.rs (no TOOL_RESULT_CHAR_BUDGET/chat_error_message-equivalent reference anywhere in it). Recorded for completeness, no new concern.
  • Fixed 16 KiB budget rather than one derived from the provider's reported n_ctx — tradeoff, explicitly disclosed in the PR description. No new concern.
  • The cap bounds one tool result, not one turn — non-blocking-improvement. model_facing_tool_result is applied once per tool call inside run_rocm_read_tool; the agent client's own .max_turns(MAX_TOOL_TURNS) loop (crates/rocm-dash-tui/src/agent/clients.rs:324) confirms several tool calls can occur before a turn's final answer, so multiple comfortably-under-budget results could still collectively approach a small model's context in one round trip. Not disclosed in the docs; worth a mention there, or tracking turn-level usage as a follow-up.

I also traced, and could not confirm, two concerns a naive read might raise — noting them because ruling them out took real investigation and the opposite conclusion would otherwise look plausible from a shallower pass:

  • Does the dash's mutating/approval-replay path (tool_approval.rs) ship an unbounded result the same way the old read-only path did? No — independently traced run_approved → summarize_json_value/scalar_or_shape (crates/rocm-dash-tui/src/app/{actions,summary}.rs): it collapses the whole envelope to at most 8 top-level fields, with any nested array/object (where the command-backed envelope's structuredContent.stdout actually lives) reduced to a {N fields}/[N items] shape hint, never inlined. This is a different, pre-existing bounding mechanism, not a gap this PR introduces or leaves open.
  • Can the structuredContent.stdout/.stderr dedup drop real, non-duplicate content on a coincidental substring match? No — traced command_capture_text (crates/rocm-core/src/command_capture.rs:195-207): the envelope's content[0].text is built by literally embedding the same trimmed stdout/stderr values the dedup compares against, so the substring check can only match a value that is, by construction, an exact duplicate.

Positive signals

  • The line-boundary binary search in cap_tool_result guarantees the serialized truncated value fits the byte budget (not an estimate), and cutting only at line boundaries keeps the kept text readable.
  • isError is explicitly carried over into the truncated form and tested in both directions (truncating_a_failed_command_keeps_it_a_failure / truncating_a_successful_command_keeps_it_a_success), so a long-but-failed command can't silently read as a success once truncated.
  • The curated renderer the chat tool now calls (render_model_registry_text_with_context_and_host) is the same function rocm model already called from main.rs — genuine reuse, not a new parallel implementation.
  • The new e2e scenarios (model-catalog-04/-05) exercise the real dispatch via rocm mcp-call, the same path the dash drives, rather than leaving the default-vs---verbose distinction to unit tests alone — and would fail if the dispatch reverted to verbose-by-default (traced: the assertions pin both the absence of aliases=[ and an exact curated-listing fragment).
  • docs/architecture.md entries for both substantially-changed files were updated in the same change, and docs/manual-testing.md's new §14 names a real CLI flag (--arguments-file, confirmed present in apps/rocm/src/cli.rs/main.rs).

Deployment notes

The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump, confirmed: 189 KB → under 2 KB on the fixture data). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands. Also: a command-backed tool result over 16 KiB now arrives at the model as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope — documented in docs/llm-tool-use.md, but worth knowing for anyone parsing dash-driven tool output elsewhere.

What this covered

DEGRADED REVIEW — fan-out was attempted (four parallel workers: the dash cap/overflow core, the CLI dispatch/model-catalog group, the docs/e2e group, and an agent-instruction/history/prior-changes pass) but all four spawn calls failed immediately with "concurrent subagent limit reached" (20/20 slots already held by other sessions sharing this environment); re-spawning was not attempted per the no-retry guidance on that error. This review proceeded single-context instead, per the degraded-review provision — recommend re-running with fan-out available for independent coverage, particularly of docs/architecture.md/docs/manual-testing.md/skills/rocm-cli-assistant/SKILL.md and the e2e feature/step files, which got a lighter pass here than the Rust core.

What was actually read: the full diff (base e2e5c5bb…head 9d739614); the complete current content of every changed file; plus, for context and to verify or refute specific claims, the complete content of apps/rocm/src/chat/turn.rs (CLI-path-uncapped claim), crates/rocm-dash-tui/src/app/tool_approval.rs, crates/rocm-dash-tui/src/app/actions.rs and crates/rocm-dash-tui/src/app/summary.rs (mutating-path bounding), crates/rocm-core/src/command_capture.rs (dedup soundness), apps/rocm/src/logging.rs (log routing/level), relevant sections of apps/rocm/src/model_catalog_text.rs and apps/rocm/src/main.rs (renderer reuse), and all of AGENTS.md against the diff (module-size limit, architecture-doc requirement, assistant tool-use consistency, user-observable-behavior-needs-a-scenario). Git history (git log) on the three core changed files was checked; nothing beyond the PR's own prior commits turned up.

CI: read directly via the GitHub API (not assumed). Every check that ran on code (clippy — which gates the module-size and architecture-doc checks — build-and-test, affected-crate tests, Windows builds, coverage ratchet, skillscope, CodeQL, E2E tests on WSL/Windows/Strix) is green. One check failed: "E2E consolidated report (self-hosted)" — read its job log directly; it failed only because the MI300X hardware E2E lane's run was cancelled (not failed), which made the report-consolidation step error on a missing artifact from that lane. Reasoned as an unrelated self-hosted-hardware-availability issue, not a defect in this change, since every lane that actually ran is green.

Did not run: cargo test/cargo clippy/cargo xtask check-module-size locally — no Rust toolchain available in this environment; relied on the PR's own green CI (read live, not assumed) for these. No live mutation testing (reverting branches and rerunning); test-quality verdicts above were reached by hand-tracing each assertion against the code it covers. Prior-changes pass (whether earlier review comments on these files' history apply here) was not run as a distinct pass beyond the git-log check above.

Not reconciled against the PR's existing review threads (a human changes-requested round, and separate automated reviews from a different account on earlier commits) — reviewed blind, from the code and the PR's own description only, per method.


Not a human review. This is an automated code review; please verify findings independently before acting on them.

🤖 Automated by an AI agent

})
.unwrap_or_default();
Some(format!(
"A ROCm check returned more text than this model can read{counts}. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: context_overflow_message always attributes the cause to "A ROCm check returned more text than this model can read" (line 76), but context_overflow_message (line 61) only matches on the substring exceed_context_size_error in the raw backend error, with no signal about what actually filled the context on this turn. The identical provider error also fires from context accumulated across many ordinary turns, or from a large pasted user message, with no oversized tool result involved — in that case the causal half of the sentence is wrong, though the remedy half ("try a more specific question, or a model with a larger context") still reads as reasonable generic advice either way. Confidence 75 · logic · Fix: consider a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read"), or confirm this simplification is acceptable since the remedy still holds regardless of cause.

/// one (`rocm_command` with `["model"]` measured 204,470 characters, about
/// 58k tokens) cannot fit in such a context at all and the provider rejects
/// the whole request.
pub(crate) const TOOL_RESULT_CHAR_BUDGET: usize = 16 * 1024;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: TOOL_RESULT_CHAR_BUDGET is named, and documented both here and in docs/llm-tool-use.md ("bounded to TOOL_RESULT_CHAR_BUDGET characters"), as a character budget, but every fit-check in cap_tool_result (tools.rs:365, the value.to_string().len() <= budget at the top and the same .to_string().len() check inside the binary search below) measures Rust's UTF‑8 byte length, not a character count. The enforcement is internally consistent (bytes compared against bytes throughout), so this is not a functional break for the ASCII-heavy CLI output these tools actually produce today — but multi-byte UTF‑8 content (a non-ASCII model name, a non-English log line) would be cut before 16 KiB of actual characters, contradicting the name and the docs. Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document the budget explicitly as bytes rather than characters.

// measured as fitting, so the returned value is within budget even where
// the length is not perfectly monotonic in the cut (the marker's digit
// count shifts by one across a power of ten).
let (mut low, mut high, mut best) = (0usize, cuts.len() - 1, 0usize);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: best is initialized to 0 here and is only ever reassigned (line 396) after a candidate cut is measured to fit budget — but cuts[0] itself (the empty-kept-text cut, i.e. just the truncation marker's own JSON overhead) is never validated before being trusted as a fallback. If budget is smaller than that marker overhead (roughly 200+ bytes), no candidate ever fits: the loop walks down to mid == 0, takes the break at line 398 without best ever having been confirmed, and the function then returns truncated_tool_result(&full, cuts[0], ...) regardless — contradicting this function's own doc comment that "the returned value is within budget." Traced precisely, but dormant today: the sole call site (model_facing_tool_result, from run_rocm_read_tool) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so this can't trigger with the shipped constant. Confidence 80 (traced, not reproduced with a failing input) · logic · Fix: assert/handle the "doesn't fit even empty" case explicitly, or verify cuts[0] fits before relying on best's initial value — relevant only if a future caller passes a smaller or configurable budget.

Both sides appended a `## 14.` to docs/manual-testing.md. Main's keeps
that number -- it pairs with its own `## 13. Pasting An Error Into
rocm diagnose` and is already merged -- and this branch's small-context
chat walkthrough becomes `## 15.`, placed after it.

Nothing references either section by number, so no further updates were
needed. The duplicate `## 11.` the scan reports is pre-existing on main
and deliberately left alone.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r
r0x0r added this pull request to the merge queue Nov 16, 2026
Merged via the queue into ROCm:main with commit 04cd840 Nov 16, 2026
45 of 46 checks passed
@r0x0r
r0x0r deleted the fix/dash-chat-tool-output-cap branch November 16, 2026 13:16
r0x0r added a commit that referenced this pull request Nov 16, 2026
Main moved again while this branch's review round was in flight (#1041 and
#1069 landed). One file conflicted, `apps/rocm/src/chat/mcp_bridge.rs`, in two
places, both the same shape: this branch and main each appended to the same
list.

The `use crate::{...}` group is resolved as the union — this branch's
`host_gpu_support_level` and main's
`render_model_registry_text_with_context_and_host` are both kept. Main's new
test uses the latter, so dropping either side would have failed to compile
rather than failing quietly.

The test module's two appended blocks (this branch's engines-tool Vulkan
tests, main's model-tool context-size tests) are independent and both kept
whole; neither was spliced into the other.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review · head 8c04cb8e

This automation never files a GitHub approval that blocks or substitutes for human review — the merge decision stays with a human reviewer; treat this as a second pair of eyes.

Review — needs work

Full review of the whole change (9 files, 859 insertions / 10 deletions), at the current head commit, which supersedes an earlier automated round done against an older commit on this PR. No ticket named in the title or description; the description is an exact, verifiable account of what the diff does, except that it does not mention one downstream consequence of its own dispatch change (see Blocking).

Blocking

  • The curated-vs-verbose dispatch change silently disables the CLI chat's deterministic "ROCm CLI summary" block for model questions — apps/rocm/src/chat/mcp_bridge.rs:511-519 (the changed dispatch), apps/rocm/src/chat/summaries.rs:316-321 (the gate), apps/rocm/src/chat/turn.rs:326,793-795 (the wiring). Full evidence and trace in the inline comment on mcp_bridge.rs. Confidence 88 · logic/test-coverage · Fix: make the deterministic summary recognize the curated output too, or deliberately retire it for model/models and say so.

Non-blocking

  • TOOL_RESULT_CHAR_BUDGET is named/documented as a "character" budget but enforced in UTF-8 bytes — crates/rocm-dash-tui/src/agent/tools.rs:251,365, docs/llm-tool-use.md. No functional break for today's ASCII-heavy output; multi-byte content would be cut earlier than documented. Confidence 85 · mechanical.
  • cap_tool_result's binary search can return a cut never confirmed to fit budget — crates/rocm-dash-tui/src/agent/tools.rs:388-399. Dormant with the shipped fixed 16 KiB constant. Confidence 75 · logic.
  • The rewritten context-overflow message always attributes the cause to "A ROCm check returned more text..." — crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79. The detection has no signal distinguishing an oversized tool result from ordinary accumulated history. Confidence 78 · logic.

Decisions for the author

  • Dash-only cap; the CLI's own tool-using path (rocm chat --prompt --tools) is not capped and still shows the raw provider body on overflow — tradeoff, explicitly disclosed in the PR description's Scope section and in docs/llm-tool-use.md. No new concern.
  • Fixed 16 KiB budget rather than one derived from the provider's reported n_ctx — tradeoff, explicitly disclosed in the PR description. No new concern.
  • Overflow detection only recognizes the llama.cpp-style exceed_context_size_error string — a vLLM-served or cloud-provider context overflow would still surface its raw, unrewritten error. Consistent with the PR's stated scope (a locally served small-context model), so this reads as a tradeoff the author should confirm is still the intended boundary rather than a defect.
  • The two new e2e scenarios (model-catalog-04/-05) pin the curated-vs-verbose split via rocm mcp-call, but cannot exercise the dash's own 16 KiB cap — model_facing_tool_result/cap_tool_result live in rocm-dash-tui, a crate the mcp-call dispatch never reaches. This gap is already explicitly disclosed in the PR's own Testing notes ("The cap itself and the overflow message remain unit-only; there is no CLI surface that exposes them.") — answered-by-intent, not a new finding.
  • The cap bounds one tool result, not one turn — non-blocking-improvement. Several comfortably-under-budget results in one multi-tool-call turn could still collectively approach a small model's context.

Positive signals

  • The line-boundary binary search in cap_tool_result guarantees the serialized truncated value fits the budget (aside from the dormant edge case above), and cutting only at line boundaries keeps kept text readable rather than chopping mid-line.
  • isError is explicitly carried over into the truncated form and tested in both directions, so a long-but-failed command can't silently read as a success once truncated.
  • The curated renderer the chat tool now calls (render_model_registry_text_with_context_and_host) is the same function rocm model itself already calls from main.rs — genuine reuse, not a new parallel implementation.
  • The PR description's mutation-style test evidence (each new check shown to fail against a deliberately broken input) checks out against the actual tests on inspection.

Deployment notes

The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands. A dash tool result over 16 KiB now arrives as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope — documented in docs/llm-tool-use.md.

What this covered

Full diff 4f1f2f31…8c04cb8e (9 files, 859 insertions / 10 deletions), every changed file read completely at the pinned head commit via git show <sha>:<path> (not the mutable working tree — see below), plus calling/called code read in full to verify specific claims: apps/rocm/src/chat/summaries.rs, apps/rocm/src/chat/turn.rs, apps/rocm/src/model_catalog_text.rs, apps/rocm/src/cli.rs (flag verification). CI checked live via the GitHub API: every check green except one self-hosted GPU lane still queued, matching the pre-existing, already-documented runner-pool issue affecting multiple unrelated PRs on this repo — not a concern this change introduces.

Parallel review workers were spawned for this pass. A mid-task message purporting to relay their findings arrived through a channel that could not be authenticated — the agent identifiers it referenced did not match this session's own spawned workers — so it was not used for anything beyond a prompt to double-check; every finding above was instead independently re-derived and verified directly against the source at the pinned commit SHA, after the shared local checkout was separately observed to have been mutated mid-review by other concurrent activity in the environment. This makes the process closer to a single verified pass than a confirmed independent fan-out for coverage purposes, though every finding reported was individually confirmed against the code before inclusion.

Did not run cargo test/cargo clippy locally (relied on the PR's own green CI for the reviewed commit); no live mutation testing was performed — test-coverage verdicts above were reached by hand-tracing. Not reconciled against the PR's existing review threads (three prior automated rounds and one human changes-requested round, all against earlier commits) — reviewed blind, from the code and the PR's own description only, per method. Note the blocking finding above was not raised by any of those three prior rounds, each of which reviewed byte-identical content in the files it touches.


🤖 Automated by an AI agent

Comment on lines 511 to +519
if command.eq_ignore_ascii_case("model") || command.eq_ignore_ascii_case("models") =>
{
Ok(render_model_registry_text_with_context_and_host(
Some(paths),
None,
None,
))
}
[command, flag]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this dispatch change silently disables the CLI chat's deterministic "ROCm CLI summary" block for model questions.

run_rocm_read_only_in_process's ["model"]/["models"] arm now returns render_model_registry_text_with_context_and_host's curated output. Its header comes from model_catalog_header (apps/rocm/src/model_catalog_text.rs), which prints "Recommended models - run one with \rocm serve `"(or the signed-index variant) — never the string"model recipes". Only the verbose renderer (render_model_registry_verbose_text_with_context_and_host) still emits writeln!(output, "model recipes")`.

deterministic_model_tool_summary (apps/rocm/src/chat/summaries.rs:320-321) is strictly gated on tool_text.contains("model recipes"). run_chat_read_only_tool's "rocm_command" arm (this file, ~line 708) feeds exactly this dispatch's output into turn.rs's follow_up_text, which is then passed straight to deterministic_chat_tool_summary (apps/rocm/src/chat/turn.rs:326).

So on rocm chat --prompt --tools — a sibling surface this same function serves — the "ROCm CLI summary" block (Recommended local assistant / Fit / Engine) that used to render whenever the model asked what it could run no longer appears, not because the feature was removed but because the one string it keys on is no longer produced by the new default path. Every existing test of deterministic_model_tool_summary/deterministic_chat_tool_summary (e.g. deterministic_model_tool_summary_suppresses_extra_local_model_follow_up in turn.rs, and the fixtures in summaries.rs) hand-writes a string containing "model recipes" literally rather than calling the real renderer or dispatch, so none of them catch this; no e2e scenario exercises it either.

Confidence 88 — traced end to end at the PR head commit: confirmed the new default output never contains the gate string, confirmed the consumer's exact gate, and confirmed the call chain connecting this dispatch to that consumer.

Fix: either make the summary recognize the curated output too (key off a stable marker both renderers share, or off the tool name/args rather than parsing renderer-specific text), or if this deterministic summary is deliberately being retired for model/models, remove it (and its now-stale fixtures) and say so in the PR description.

/// one (`rocm_command` with `["model"]` measured 204,470 characters, about
/// 58k tokens) cannot fit in such a context at all and the provider rejects
/// the whole request.
pub(crate) const TOOL_RESULT_CHAR_BUDGET: usize = 16 * 1024;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: TOOL_RESULT_CHAR_BUDGET is named/documented as a "character" budget but enforced in UTF-8 bytes.

cap_tool_result's fit check (line ~365) uses value.to_string().len(), which is Rust's byte length, not a character count; docs/llm-tool-use.md likewise says "bounded to TOOL_RESULT_CHAR_BUDGET characters". Enforcement is internally consistent (bytes compared against bytes throughout), so this isn't a functional break for the ASCII-heavy CLI output these tools produce today, but multi-byte UTF-8 content (an accented path, a non-ASCII model id) would be cut well before 16 KiB of actual characters, contradicting the constant's name and both the code doc and the user-facing doc.

Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document the budget explicitly as bytes.

Comment on lines +388 to +398
let (mut low, mut high, mut best) = (0usize, cuts.len() - 1, 0usize);
while low <= high {
let mid = low.midpoint(high);
if truncated_tool_result(&full, cuts[mid], total_chars, is_error)
.to_string()
.len()
<= budget
{
best = mid;
low = mid + 1;
} else if mid == 0 {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this binary search can return a cut that was never confirmed to fit budget, contradicting its own doc comment.

best is initialized to 0 here and is only supposed to be set once a candidate is measured to fit. If the minimal truncated form (cuts[0]: empty kept text plus the marker's own JSON overhead, a few hundred bytes) itself exceeds budget, the loop reaches mid == 0 and breaks (line 398) without ever validating cuts[0]/best, yet the function still returns truncated_tool_result(&full, cuts[best], ...) as if it had been checked.

This is dormant today: the sole caller (model_facing_tool_result) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so the branch can't currently trigger, and no test exercises a smaller/variable budget.

Confidence 75 (traced precisely; not reproduced with a failing budget) · logic · Fix: validate cuts[0] fits before trusting best's initial value, or assert the budget is large enough for the marker.

Comment on lines +61 to +76
fn context_overflow_message(raw: &str) -> Option<String> {
if !raw.contains("exceed_context_size_error") {
return None;
}
let counts = json_number_field(raw, "n_prompt_tokens")
.zip(json_number_field(raw, "n_ctx"))
.map(|(used, available)| {
format!(
" ({} of {} tokens)",
group_thousands(used),
group_thousands(available)
)
})
.unwrap_or_default();
Some(format!(
"A ROCm check returned more text than this model can read{counts}. \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the rewritten overflow message always attributes the cause to "A ROCm check returned more text than this model can read".

context_overflow_message detects the failure purely by matching "exceed_context_size_error" in the raw provider error string; nothing available at this call site distinguishes a single oversized tool result from context filled by ordinary accumulated conversation history across many turns, or a long pasted user message. In those cases the causal half of the sentence is wrong, even though the remedy half ("try a more specific question, or a model with a larger context") is still reasonable advice either way.

Confidence 78 · logic · Fix: a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read"), or confirm the specific attribution is an acceptable simplification.

r0x0r added a commit that referenced this pull request Nov 17, 2026
Main moved again while this branch's review round was in flight (#1041 and
#1069 landed). One file conflicted, `apps/rocm/src/chat/mcp_bridge.rs`, in two
places, both the same shape: this branch and main each appended to the same
list.

The `use crate::{...}` group is resolved as the union — this branch's
`host_gpu_support_level` and main's
`render_model_registry_text_with_context_and_host` are both kept. Main's new
test uses the latter, so dropping either side would have failed to compile
rather than failing quietly.

The test module's two appended blocks (this branch's engines-tool Vulkan
tests, main's model-tool context-size tests) are independent and both kept
whole; neither was spliced into the other.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants