Repository navigation
fix(dash): bound chat tool output so a small-context model can answer - #1041
Conversation
Asking the dash Chat tab anything that makes the model call `rocm_command` with `["model"]` ended the turn with the provider's raw HTTP 400 instead of an answer: the tool result measured 204,470 characters (~58k tokens), more than a locally served model's entire context (llama.cpp defaults `n_ctx` to 32768). Four things compounded, and all four are addressed. The chat `model`/`models` tool rendered the `rocm model --verbose` diagnostics — every recipe with its aliases, engine recipes, flags and policy notes — rather than the curated list `rocm model` prints. It now renders the curated list; the diagnostics need an explicit `["model", "--verbose"]`. The MCP envelope carries a command's output twice: as the content text and again as `structuredContent.stdout`. `rocm mcp-call` consumers read the structured field, so the envelope is unchanged and the duplicate is dropped in the dash instead, where the model is the only reader. Nothing bounded a tool result at all. A read-only result is now capped at 16 KiB of serialized JSON, cut on a line boundary and closed with a marker naming how much was omitted, so neither the model nor the user can take a truncated list for a complete one. The `rocm_command` description and its `args` schema both used `["model"]` as the example, which a small local model copies literally; they now use `["config", "show"]`. Separately, a provider context overflow now reads as a plain sentence in the Chat tab naming the token counts and a way forward, with the raw provider body still logged to rocm-cli.log. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The dash TUI's chat path has no Gherkin coverage, but the change behind it is observable from the CLI: `rocm mcp-call rocm_command` with `["model"]` is the same tool dispatch the Chat tab drives, so the scenarios assert there rather than leaving the behaviour to unit tests alone. One pins the recommendation list and its size, the other that the per-recipe diagnostics are still reachable on request. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
…iled Capping an oversized read-only tool result replaces the envelope with a fresh object, so anything the chat path reads off it has to be carried over deliberately. `isError` was not, which meant a command that failed with more output than the budget allows reached the model looking like a success — `turn.rs` reads exactly that field to decide. Carry it over when the envelope had one, and leave it absent when it did not, so the shape still mirrors its source. Two tests pin both directions; both fail if the carry is dropped. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
I went through this diff and found no outright bugs, and CI is green apart from the known pending nightly GPU lane. I'm requesting changes because a few things should be fixed or consciously accepted before merge; details are inline.
- The overflow message claims a ROCm check result caused it, which it can't know (history or a large paste trigger the same error), so the advice may not help. Overflow detection also only recognises the llama.cpp error code.
- The dedup can drop a short, genuinely different stderr/stdout because it uses a substring check.
- The 16 KiB budget is described in characters but measured in bytes.
- The manual-testing doc uses
--arguments-file, which CI never exercises; please confirm it exists.
Smaller points, no inline comment: the cap is per result, not per turn, so several large results in one turn can still overflow a 32k context. It covers only the dash read-only path; the mutating/approved dash path is uncapped and the PR body doesn't mention that. The 16 KiB literal is also repeated in three places without a shared constant. The unit tests, the e2e scenarios and the failure-path handling (isError carried over on truncation) look good.
| }) | ||
| .unwrap_or_default(); | ||
| Some(format!( | ||
| "A ROCm check returned more text than this model can read{counts}. \ |
There was a problem hiding this comment.
This sentence says a ROCm check result caused the overflow, but exceed_context_size_error also fires when accumulated history or a large paste is the cause. In that case "try a more specific question" won't clear it. Please word it so it doesn't assert the cause, and suggest starting a new chat.
| /// user can act on. The token counts come out of the same body; a body that | ||
| /// omits them still gets the sentence, without the numbers. | ||
| fn context_overflow_message(raw: &str) -> Option<String> { | ||
| if !raw.contains("exceed_context_size_error") { |
There was a problem hiding this comment.
Only the llama.cpp error type is recognised here; other OpenAI-compatible servers/providers use different codes and will still show the raw 400. Fine if intended, but please say so in the doc comment (the PR body talks about "a provider context overflow").
| .and_then(Value::as_str) | ||
| .is_some_and(|s| { | ||
| let trimmed = s.trim(); | ||
| trimmed.is_empty() || text.contains(trimmed) |
There was a problem hiding this comment.
text.contains(trimmed) can drop a short, non-duplicated stderr/stdout (e.g. error, 0) that merely appears somewhere in the text. Consider requiring the duplicate to be the tail of the text, or a minimum length.
| /// against the *serialized* value, which is what actually reaches the model, | ||
| /// so JSON escaping cannot push it back over. | ||
| fn cap_tool_result(value: Value, budget: usize) -> Value { | ||
| if value.to_string().len() <= budget { |
There was a problem hiding this comment.
The budget is compared against serialized bytes (.len()), while the constant, docs and marker speak of characters. Same for ASCII only; please either rename/document it as bytes or count chars.
|
|
||
| ```bash | ||
| echo '{"args":["model"]}' > args.json | ||
| rocm mcp-call rocm_command --arguments-file args.json | wc -c |
There was a problem hiding this comment.
The e2e step uses mcp-call ... --arguments-json; this uses --arguments-file. Please confirm that flag exists, since CI doesn't exercise it.
Two of these checked against the code — both confirmed@juhovainio — I have not changed code here yet; this is verification of the two claims that were directly checkable, so the facts are settled before anyone works the list. "The 16 KiB budget is described in characters but measured in bytes" — confirmedExactly right, and it leaks into user-visible text as well as the cap itself.
if value.to_string().len() <= budget
"[output truncated: {shown_chars} of {total_chars} characters shown, {omitted} omitted. …]"So on any non-ASCII tool output — a model name with an em dash, a path with accented characters, anything CJK — the budget is tighter than documented and the message tells the model and the user a byte count while calling it a character count. The variable names ( Worth noting the sibling comment at "
|
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head 74c835d
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change (9 files, 859 insertions / 10 deletions). No ticket named in the PR description.
Blocking
None
Non-blocking
TOOL_RESULT_CHAR_BUDGETand its docs describe the cap in "characters", but it is enforced in UTF‑8 bytes —crates/rocm-dash-tui/src/agent/tools.rs:251(constant + its doc comment at :239‑250),:366and:391‑394(cap_tool_result's fit checks all usevalue.to_string().len(), which is Rust's byte length), anddocs/llm-tool-use.md:80-81("bounded toTOOL_RESULT_CHAR_BUDGETcharacters"). The enforcement is internally consistent (bytes are compared against bytes throughout), so this isn't a functional break for the ASCII-heavy CLI output these tools actually produce, but the name and both the code comment and the doc describe a character budget that the code doesn't actually implement — multi-byte UTF‑8 content would be cut well before 16 KiB of characters. Confidence 80 · mechanical · Fix: rename to a byte-oriented name (or doc it explicitly as bytes), or switch the fit check to a char count if "characters" is the intended contract.cap_tool_result's binary search can return a value that exceedsbudget, contradicting its own doc comment —crates/rocm-dash-tui/src/agent/tools.rs:365-404.bestis initialized to0unverified (:388); if the very first probedmidis0and the minimal (marker-only) truncated form already exceedsbudget, the loop takes themid == 0 → breakbranch (:398‑399) without ever confirmingbestfits, and the function then returns that unverified, over-budget value. This requires abudgetsmaller than the truncation marker's own JSON overhead (roughly 200+ bytes), which never happens with the realTOOL_RESULT_CHAR_BUDGET = 16 * 1024, so it's dormant today, not an observed break. Confidence 80 (traced precisely), but non-blocking since unreachable with the shipped constant · logic · Fix: assert/handle the "doesn't fit even empty" case explicitly, or verifycuts[0]fits before relying onbest's initial value.- The dash's mutating/approval-replay tool-result path is not routed through the new cap at all — confirmed by grep:
crates/rocm-dash-tui/src/app/tool_approval.rshas zero references tomodel_facing_tool_result/TOOL_RESULT_CHAR_BUDGET. An approved mutating command (e.g. an install with a long log) can still hand the model an unbounded result and reproduce the exact HTTP‑400 symptom this PR fixes for the read-only funnel. The PR's own Scope section discloses one remaining uncapped surface (the separate CLI chat path) but not this second one, which sits in the very crate and feature this PR touches. Confidence 85 · architectural · Fix: either apply the same cap in the approval-replay path, or disclose it the same way the CLI-path gap is disclosed. - The cap bounds one result, not one turn —
crates/rocm-dash-tui/src/agent/tools.rs:453callsmodel_facing_tool_resultonce per individual tool call insiderun_rocm_read_tool. Nothing aggregates across multiple tool calls returned within the same model turn, so several moderately-sized results (each comfortably under 16 KiB on its own) can still collectively approach or exceed a small model's context in one round trip. Confidence 70 (reasoned from the call site; not traced through therigagent loop to confirm multi-tool-call turns are possible) · architectural · Fix: note as a known limit, or track turn-level token usage if the agent loop can issue multiple tool calls per turn. - The rewritten overflow message always attributes the cause to "A ROCm check" —
crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79.context_overflow_messagedetects the failure purely by matching"exceed_context_size_error"in the raw error string; it has no signal about why the prompt grew too large. The same provider error fires just as easily from accumulated conversation history across many turns with no oversized tool result in the offending turn, in which case the message's specific remediation ("a ROCm check returned more text...") misattributes the cause. Confidence 75 · logic · Fix: soften to a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read") unless the call site can actually distinguish the two cases.
Decisions for the author
- Dash-only cap, CLI chat path left uncapped — tradeoff, already explicitly disclosed and reasoned in the PR description's Scope section and in
docs/llm-tool-use.md. Recorded here for completeness; no new concern. - Fixed 16 KiB budget instead of a context-aware one derived from the provider's reported
n_ctx— tradeoff, already explicitly disclosed and reasoned in the PR description's Scope section. Recorded here for completeness; no new concern.
Positive signals
- The line-boundary binary search in
cap_tool_resultis a careful piece of engineering: it guarantees the serialized truncated value fits the byte budget (not an estimate), and cutting only at line boundaries keeps the kept text readable rather than chopping mid-line. - The PR description's mutation-evidence table (each new check shown to fail against a specific deliberately-broken input) is good testing discipline and made several of the new tests easy to trust on inspection alone.
Deployment notes
The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands.
What this covered
Read the complete diff (f536619a08a3b50dd26aac5c21a3bb9f6bd266d2...74c835dffbbebb009023e74b9697a2c0cf3baec6) plus the complete content of every changed file at the PR head, plus surrounding/calling code read in full: apps/rocm/src/chat/mcp_bridge.rs's whole dispatch match block, apps/rocm/src/model_catalog_text.rs (the curated/verbose renderers), apps/rocm/src/chat/turn.rs and apps/rocm/src/logging.rs (to verify the CLI-path-uncapped claim and production log routing), crates/rocm-dash-tui/src/app/tool_approval.rs, and the root AGENTS.md. Verified CI via gh pr view --json statusCheckRollup: all checks green except two GPU-hardware lanes (MI300X, W7900) still queued at review time, not failed.
This review initially fanned out five sub-tasks (one per coupled file group) to run as independent blind passes, per method. None of their agent IDs ever appeared as present/completed in this session's own agent listing, and a purported relay of their findings arrived instead via a mid-task message from a third party; that relay's content cited source-file line numbers that did not match the actual files at this head commit, so it could not be authenticated as genuine subagent output and none of it was used. Every claim in this report was instead independently re-derived and verified directly against the source by the reviewing context itself (full file reads, git grep, and hand-traced logic), which is closer to a single-context pass than a verified multi-worker fan-out for the code-correctness groups, even though each finding above was individually confirmed against the code before inclusion. Did not run: the actual cargo test/cargo clippy suite locally (relied on the PR's own green CI); no live mutation testing (reverting code paths and re-running tests) was executed — test-quality verdicts above were reached by hand-tracing. Not reconciled against the PR's existing review thread; this report was produced blind from the code and the PR's own description only.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
The remaining lane is not going to start: the self-hosted runner pool is downThis PR has no failures left — the only thing outstanding is a GPU lane that has been
Three of the six are offline. The MI300X pair is the stranger case: both report Nothing about this PR can clear it; it needs someone with runner access to restart the MI300X agents and bring |
MI300X: the known broken prewarm runtime, and a correction about the runnersThe MI300X lane finished and is red with eight unexpected failures. They are the environmental breakage already documented on #1036 and #1054, and this run makes the case conclusive. Eleven occurrences of Why this run settles it. The same eight scenarios, byte-identical, have now failed on three PRs with three different bases:
This PR touches Separately, a correction to my previous comment here. I said both MI300X runners reported The offline runners from that comment are still offline ( |
…tput-cap Signed-off-by: Roman Sirokov <roman.sirokov@amd.com> # Conflicts: # docs/manual-testing.md
|
Merged the latest What the merge touchedThe only conflicting file was Relative to
Numbering and references checked
Gates (all green, on Linux)
Still outstandingThe CHANGES_REQUESTED items from the review are not addressed by this merge and still stand:
|
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head 9d73961
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change. No ticket named in the title or commits; the description is an exact, verifiable account of what the diff does.
Blocking
None
Non-blocking
cap_tool_result's binary search can return an unverified cut when even the empty-content form doesn't fit the budget —crates/rocm-dash-tui/src/agent/tools.rs(cap_tool_result,truncated_tool_result)
beststarts at0and is only advanced once a candidate is measured to fit the budget. Iftruncated_tool_result(&full, cuts[0], ...)— empty kept text plus the truncation marker, a few hundred bytes of JSON — itself exceedsbudget, themid == 0branchbreaks without ever validatingcuts[best], so the function can return a value whose length was never checked againstbudget, contradicting its own doc comment ("bestis only ever set to a cut that was measured as fitting, so the returned value is within budget"). This is not reachable today: the sole caller (model_facing_tool_result, fromrun_rocm_read_tool) always passes the fixedTOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so the branch can't currently trigger, and no test exercises a small/variable budget. Confidence 70 — traced the logic, not reproduced with a failing input. Fix: checkcuts[0]explicitly before the loop (or assert the budget is large enough for the marker) so a future caller with a smaller or configurable budget can't silently get an over-budget result.
Decisions for the author
- Only the dash's tool-result path is capped/de-duplicated and rewritten into plain language on context overflow; the CLI chat path (
rocm chat --prompt --tools) is not — tradeoff
The curated-vs-verbose model-listing split is shared, so the specific reported bug is fixed on both paths, but the CLI path still passes an unbounded result straight through (mcp_tool_result_text, no budget) and still surfaces a provider's raw error body verbatim on overflow (nochat_error_message-equivalent inapps/rocm/src/chat/turn.rs). Both asymmetries are explicitly named in the PR description and indocs/llm-tool-use.mdas a deliberate scope cut rather than an oversight, with the CLI path called "the obvious follow-up" — worth confirming that's still the intended boundary, since the CLI drives the exact samerocm_commanddispatch and can hit the same small-context failure mode. - The 16 KiB budget is a fixed constant rather than derived from the provider's reported
n_ctx— tradeoff
A context-aware budget would fit a--ctx-size 2048host, or a much larger one, more precisely, at the cost of plumbing the provider's context size through to the tool layer. The PR description frames the rewritten overflow message as the fallback for whatever the fixed budget misses, and calls this out explicitly as a judgement call. - The context-overflow rewrite only runs on the dash's dispatch edge; the CLI chat path still shows the raw provider body on the same failure — non-blocking-improvement
A natural follow-up is giving the CLI chat loop the same plain-language rewritechat_error_messagegives the dash, so a user ofrocm chat --prompt --toolsdoesn't see a rawexceed_context_size_errorJSON body where the dash now shows a sentence.
Positive signals
model_facing_tool_resultis applied uniformly to every read-only tool result inrun_rocm_read_tool, not special-cased to themodelcommand, so any other command returning a very large result (e.g.logson a busy host, called out indocs/manual-testing.md) gets the same protection for free.- The truncation cut is computed against a plain-text rendering at a line boundary, then re-serialized and measured against the actual JSON-escaped length before being accepted — avoids truncating mid-escape or mid multi-byte character.
isErroris explicitly carried over into the truncated form and tested in both directions, so a long-but-failed command can't silently read as a success once truncated.- The PR description's mutation-style test evidence (each check shown to fail against a deliberately broken input) checks out against the actual tests, e.g.
truncating_a_failed_command_keeps_it_a_failure/truncating_a_successful_command_keeps_it_a_success.
Deployment notes
No migration or configuration change. The model/--verbose split and the dedup/cap change the shape of results any external MCP client already gets from rocm mcp-call/the dash: structuredContent.stdout/.stderr now disappear when they duplicate the text content, and a result over 16 KiB now arrives as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope. This is documented in docs/llm-tool-use.md, but any external client that parses the full envelope unconditionally should be aware of it.
What this covered
Full diff of ROCm/rocm-cli PR #1041 at head 9d73961 (base e2e5c5b…9d73961): apps/rocm/src/chat/mcp_bridge.rs, crates/rocm-dash-tui/src/agent/tools.rs, crates/rocm-dash-tui/src/app/chat_runtime.rs, docs/architecture.md, docs/llm-tool-use.md, docs/manual-testing.md, skills/rocm-cli-assistant/SKILL.md, tests/e2e-cucumber/features/model_catalog.feature, tests/e2e-cucumber/tests/e2e/model_catalog_steps.rs — each read in full, plus apps/rocm/src/main.rs's rocm model CLI dispatch and the relevant sections of AGENTS.md (module-size limit, assistant tool-use consistency) for cross-reference.
apps/rocm/src/chat/mcp_bridge.rs and crates/rocm-dash-tui/src/agent/tools.rs were reviewed by an independent fan-out worker. A second worker was requested for crates/rocm-dash-tui/src/app/chat_runtime.rs, the e2e feature/step files, and the docs changes, but every spawn attempt failed on a shared concurrency limit; that group was reviewed directly in this same context instead and so was not independent of the rest of this report.
CI showed every completed check green (clippy, build-and-test, affected-crate tests, E2E tests, ratcheted coverage, skillscope); a few long-running hardware/E2E jobs were still queued or in progress at review time with nothing failed.
Did not run: no separate git blame/history pass or prior-review-comment pass beyond what's folded into the reading above; no test suite executed locally (relied on the green CI run for the reviewed commit).
Not reconciled against any discussion on the PR — this review did not read existing comments or review threads.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
juhovainio
left a comment
There was a problem hiding this comment.
🤖 Automated review · head 9d73961
This automation never files a GitHub approval that blocks or substitutes for human review — the merge decision stays with a human reviewer; treat this as a second pair of eyes, not a sign-off.
Review — no blocking findings
Full review of the whole change (9 files, 859 insertions / 10 deletions). No ticket named in the title or description; the description is an exact, verifiable account of what the diff does — every claim I checked against the code held up.
Blocking
None
Non-blocking
- The context-overflow message always attributes the cause to "A ROCm check returned more text than this model can read" —
crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79(message text at :76).context_overflow_messagedetects the failure purely by matchingexceed_context_size_errorin the raw provider error string; it has no signal about what actually filled the context this turn. The same provider error fires just as easily from history accumulated across many ordinary turns, or a large pasted message, with no oversized tool result involved, in which case the causal half of the sentence is wrong — though the remedy half ("try a more specific question, or a model with a larger context") is still reasonable generic advice either way. Confidence 75 · logic · Fix: a cause-neutral phrasing, or confirm the simplification is acceptable. TOOL_RESULT_CHAR_BUDGETis named and documented as a character budget but enforced in UTF-8 bytes —crates/rocm-dash-tui/src/agent/tools.rs:251(constant + doc comment),:365-404(cap_tool_result's fit checks all usevalue.to_string().len(), Rust's byte length), anddocs/llm-tool-use.md:80-81("bounded toTOOL_RESULT_CHAR_BUDGETcharacters"). Internally consistent (bytes compared to bytes throughout), so no functional break for the ASCII-heavy CLI output these tools actually produce — but multi-byte UTF-8 content would be cut before 16 KiB of actual characters. Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document it explicitly as bytes.cap_tool_result's binary search can return an unverified cut when even the empty-content form doesn't fit the budget —crates/rocm-dash-tui/src/agent/tools.rs:365-404.beststarts at0and is only confirmed once a candidate is measured to fit; ifcuts[0](empty kept text plus the marker's own JSON overhead, ~200+ bytes) itself exceedsbudget, the loop reachesmid == 0,breaks without ever validatingcuts[0], and the function returns it anyway — contradicting the function's own doc comment that the result is always within budget. Dormant today: the sole caller (model_facing_tool_result) always passes the fixedTOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above that overhead, so it can't trigger with the shipped constant. Confidence 80 (traced precisely, not reproduced with a failing input) · logic · Fix: validatecuts[0]fits before trustingbest's initial value, relevant only if a future caller passes a smaller/configurable budget.
Decisions for the author
- Dash-only cap; the CLI's own tool-using path (
rocm chat --prompt --tools) readsmcp_tool_result_textstraight through with no budget and shows the raw provider error on overflow — tradeoff, explicitly disclosed in the PR description's Scope section and indocs/llm-tool-use.md; independently confirmed by readingapps/rocm/src/chat/turn.rs(noTOOL_RESULT_CHAR_BUDGET/chat_error_message-equivalent reference anywhere in it). Recorded for completeness, no new concern. - Fixed 16 KiB budget rather than one derived from the provider's reported
n_ctx— tradeoff, explicitly disclosed in the PR description. No new concern. - The cap bounds one tool result, not one turn — non-blocking-improvement.
model_facing_tool_resultis applied once per tool call insiderun_rocm_read_tool; the agent client's own.max_turns(MAX_TOOL_TURNS)loop (crates/rocm-dash-tui/src/agent/clients.rs:324) confirms several tool calls can occur before a turn's final answer, so multiple comfortably-under-budget results could still collectively approach a small model's context in one round trip. Not disclosed in the docs; worth a mention there, or tracking turn-level usage as a follow-up.
I also traced, and could not confirm, two concerns a naive read might raise — noting them because ruling them out took real investigation and the opposite conclusion would otherwise look plausible from a shallower pass:
- Does the dash's mutating/approval-replay path (
tool_approval.rs) ship an unbounded result the same way the old read-only path did? No — independently tracedrun_approved→summarize_json_value/scalar_or_shape(crates/rocm-dash-tui/src/app/{actions,summary}.rs): it collapses the whole envelope to at most 8 top-level fields, with any nested array/object (where the command-backed envelope'sstructuredContent.stdoutactually lives) reduced to a{N fields}/[N items]shape hint, never inlined. This is a different, pre-existing bounding mechanism, not a gap this PR introduces or leaves open. - Can the
structuredContent.stdout/.stderrdedup drop real, non-duplicate content on a coincidental substring match? No — tracedcommand_capture_text(crates/rocm-core/src/command_capture.rs:195-207): the envelope'scontent[0].textis built by literally embedding the same trimmedstdout/stderrvalues the dedup compares against, so the substring check can only match a value that is, by construction, an exact duplicate.
Positive signals
- The line-boundary binary search in
cap_tool_resultguarantees the serialized truncated value fits the byte budget (not an estimate), and cutting only at line boundaries keeps the kept text readable. isErroris explicitly carried over into the truncated form and tested in both directions (truncating_a_failed_command_keeps_it_a_failure/truncating_a_successful_command_keeps_it_a_success), so a long-but-failed command can't silently read as a success once truncated.- The curated renderer the chat tool now calls (
render_model_registry_text_with_context_and_host) is the same functionrocm modelalready called frommain.rs— genuine reuse, not a new parallel implementation. - The new e2e scenarios (
model-catalog-04/-05) exercise the real dispatch viarocm mcp-call, the same path the dash drives, rather than leaving the default-vs---verbosedistinction to unit tests alone — and would fail if the dispatch reverted to verbose-by-default (traced: the assertions pin both the absence ofaliases=[and an exact curated-listing fragment). docs/architecture.mdentries for both substantially-changed files were updated in the same change, anddocs/manual-testing.md's new §14 names a real CLI flag (--arguments-file, confirmed present inapps/rocm/src/cli.rs/main.rs).
Deployment notes
The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump, confirmed: 189 KB → under 2 KB on the fixture data). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands. Also: a command-backed tool result over 16 KiB now arrives at the model as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope — documented in docs/llm-tool-use.md, but worth knowing for anyone parsing dash-driven tool output elsewhere.
What this covered
DEGRADED REVIEW — fan-out was attempted (four parallel workers: the dash cap/overflow core, the CLI dispatch/model-catalog group, the docs/e2e group, and an agent-instruction/history/prior-changes pass) but all four spawn calls failed immediately with "concurrent subagent limit reached" (20/20 slots already held by other sessions sharing this environment); re-spawning was not attempted per the no-retry guidance on that error. This review proceeded single-context instead, per the degraded-review provision — recommend re-running with fan-out available for independent coverage, particularly of docs/architecture.md/docs/manual-testing.md/skills/rocm-cli-assistant/SKILL.md and the e2e feature/step files, which got a lighter pass here than the Rust core.
What was actually read: the full diff (base e2e5c5bb…head 9d739614); the complete current content of every changed file; plus, for context and to verify or refute specific claims, the complete content of apps/rocm/src/chat/turn.rs (CLI-path-uncapped claim), crates/rocm-dash-tui/src/app/tool_approval.rs, crates/rocm-dash-tui/src/app/actions.rs and crates/rocm-dash-tui/src/app/summary.rs (mutating-path bounding), crates/rocm-core/src/command_capture.rs (dedup soundness), apps/rocm/src/logging.rs (log routing/level), relevant sections of apps/rocm/src/model_catalog_text.rs and apps/rocm/src/main.rs (renderer reuse), and all of AGENTS.md against the diff (module-size limit, architecture-doc requirement, assistant tool-use consistency, user-observable-behavior-needs-a-scenario). Git history (git log) on the three core changed files was checked; nothing beyond the PR's own prior commits turned up.
CI: read directly via the GitHub API (not assumed). Every check that ran on code (clippy — which gates the module-size and architecture-doc checks — build-and-test, affected-crate tests, Windows builds, coverage ratchet, skillscope, CodeQL, E2E tests on WSL/Windows/Strix) is green. One check failed: "E2E consolidated report (self-hosted)" — read its job log directly; it failed only because the MI300X hardware E2E lane's run was cancelled (not failed), which made the report-consolidation step error on a missing artifact from that lane. Reasoned as an unrelated self-hosted-hardware-availability issue, not a defect in this change, since every lane that actually ran is green.
Did not run: cargo test/cargo clippy/cargo xtask check-module-size locally — no Rust toolchain available in this environment; relied on the PR's own green CI (read live, not assumed) for these. No live mutation testing (reverting branches and rerunning); test-quality verdicts above were reached by hand-tracing each assertion against the code it covers. Prior-changes pass (whether earlier review comments on these files' history apply here) was not run as a distinct pass beyond the git-log check above.
Not reconciled against the PR's existing review threads (a human changes-requested round, and separate automated reviews from a different account on earlier commits) — reviewed blind, from the code and the PR's own description only, per method.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
🤖 Automated by an AI agent
| }) | ||
| .unwrap_or_default(); | ||
| Some(format!( | ||
| "A ROCm check returned more text than this model can read{counts}. \ |
There was a problem hiding this comment.
Non-blocking: context_overflow_message always attributes the cause to "A ROCm check returned more text than this model can read" (line 76), but context_overflow_message (line 61) only matches on the substring exceed_context_size_error in the raw backend error, with no signal about what actually filled the context on this turn. The identical provider error also fires from context accumulated across many ordinary turns, or from a large pasted user message, with no oversized tool result involved — in that case the causal half of the sentence is wrong, though the remedy half ("try a more specific question, or a model with a larger context") still reads as reasonable generic advice either way. Confidence 75 · logic · Fix: consider a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read"), or confirm this simplification is acceptable since the remedy still holds regardless of cause.
| /// one (`rocm_command` with `["model"]` measured 204,470 characters, about | ||
| /// 58k tokens) cannot fit in such a context at all and the provider rejects | ||
| /// the whole request. | ||
| pub(crate) const TOOL_RESULT_CHAR_BUDGET: usize = 16 * 1024; |
There was a problem hiding this comment.
Non-blocking: TOOL_RESULT_CHAR_BUDGET is named, and documented both here and in docs/llm-tool-use.md ("bounded to TOOL_RESULT_CHAR_BUDGET characters"), as a character budget, but every fit-check in cap_tool_result (tools.rs:365, the value.to_string().len() <= budget at the top and the same .to_string().len() check inside the binary search below) measures Rust's UTF‑8 byte length, not a character count. The enforcement is internally consistent (bytes compared against bytes throughout), so this is not a functional break for the ASCII-heavy CLI output these tools actually produce today — but multi-byte UTF‑8 content (a non-ASCII model name, a non-English log line) would be cut before 16 KiB of actual characters, contradicting the name and the docs. Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document the budget explicitly as bytes rather than characters.
| // measured as fitting, so the returned value is within budget even where | ||
| // the length is not perfectly monotonic in the cut (the marker's digit | ||
| // count shifts by one across a power of ten). | ||
| let (mut low, mut high, mut best) = (0usize, cuts.len() - 1, 0usize); |
There was a problem hiding this comment.
Non-blocking: best is initialized to 0 here and is only ever reassigned (line 396) after a candidate cut is measured to fit budget — but cuts[0] itself (the empty-kept-text cut, i.e. just the truncation marker's own JSON overhead) is never validated before being trusted as a fallback. If budget is smaller than that marker overhead (roughly 200+ bytes), no candidate ever fits: the loop walks down to mid == 0, takes the break at line 398 without best ever having been confirmed, and the function then returns truncated_tool_result(&full, cuts[0], ...) regardless — contradicting this function's own doc comment that "the returned value is within budget." Traced precisely, but dormant today: the sole call site (model_facing_tool_result, from run_rocm_read_tool) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so this can't trigger with the shipped constant. Confidence 80 (traced, not reproduced with a failing input) · logic · Fix: assert/handle the "doesn't fit even empty" case explicitly, or verify cuts[0] fits before relying on best's initial value — relevant only if a future caller passes a smaller or configurable budget.
Both sides appended a `## 14.` to docs/manual-testing.md. Main's keeps that number -- it pairs with its own `## 13. Pasting An Error Into rocm diagnose` and is already merged -- and this branch's small-context chat walkthrough becomes `## 15.`, placed after it. Nothing references either section by number, so no further updates were needed. The duplicate `## 11.` the scan reports is pre-existing on main and deliberately left alone. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Main moved again while this branch's review round was in flight (#1041 and #1069 landed). One file conflicted, `apps/rocm/src/chat/mcp_bridge.rs`, in two places, both the same shape: this branch and main each appended to the same list. The `use crate::{...}` group is resolved as the union — this branch's `host_gpu_support_level` and main's `render_model_registry_text_with_context_and_host` are both kept. Main's new test uses the latter, so dropping either side would have failed to compile rather than failing quietly. The test module's two appended blocks (this branch's engines-tool Vulkan tests, main's model-tool context-size tests) are independent and both kept whole; neither was spliced into the other. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
🤖 Automated review · head 8c04cb8e
This automation never files a GitHub approval that blocks or substitutes for human review — the merge decision stays with a human reviewer; treat this as a second pair of eyes.
Review — needs work
Full review of the whole change (9 files, 859 insertions / 10 deletions), at the current head commit, which supersedes an earlier automated round done against an older commit on this PR. No ticket named in the title or description; the description is an exact, verifiable account of what the diff does, except that it does not mention one downstream consequence of its own dispatch change (see Blocking).
Blocking
- The curated-vs-verbose dispatch change silently disables the CLI chat's deterministic "ROCm CLI summary" block for model questions —
apps/rocm/src/chat/mcp_bridge.rs:511-519(the changed dispatch),apps/rocm/src/chat/summaries.rs:316-321(the gate),apps/rocm/src/chat/turn.rs:326,793-795(the wiring). Full evidence and trace in the inline comment onmcp_bridge.rs. Confidence 88 · logic/test-coverage · Fix: make the deterministic summary recognize the curated output too, or deliberately retire it formodel/modelsand say so.
Non-blocking
TOOL_RESULT_CHAR_BUDGETis named/documented as a "character" budget but enforced in UTF-8 bytes —crates/rocm-dash-tui/src/agent/tools.rs:251,365,docs/llm-tool-use.md. No functional break for today's ASCII-heavy output; multi-byte content would be cut earlier than documented. Confidence 85 · mechanical.cap_tool_result's binary search can return a cut never confirmed to fitbudget—crates/rocm-dash-tui/src/agent/tools.rs:388-399. Dormant with the shipped fixed 16 KiB constant. Confidence 75 · logic.- The rewritten context-overflow message always attributes the cause to "A ROCm check returned more text..." —
crates/rocm-dash-tui/src/app/chat_runtime.rs:61-79. The detection has no signal distinguishing an oversized tool result from ordinary accumulated history. Confidence 78 · logic.
Decisions for the author
- Dash-only cap; the CLI's own tool-using path (
rocm chat --prompt --tools) is not capped and still shows the raw provider body on overflow — tradeoff, explicitly disclosed in the PR description's Scope section and indocs/llm-tool-use.md. No new concern. - Fixed 16 KiB budget rather than one derived from the provider's reported
n_ctx— tradeoff, explicitly disclosed in the PR description. No new concern. - Overflow detection only recognizes the llama.cpp-style
exceed_context_size_errorstring — a vLLM-served or cloud-provider context overflow would still surface its raw, unrewritten error. Consistent with the PR's stated scope (a locally served small-context model), so this reads as a tradeoff the author should confirm is still the intended boundary rather than a defect. - The two new e2e scenarios (
model-catalog-04/-05) pin the curated-vs-verbose split viarocm mcp-call, but cannot exercise the dash's own 16 KiB cap —model_facing_tool_result/cap_tool_resultlive inrocm-dash-tui, a crate themcp-calldispatch never reaches. This gap is already explicitly disclosed in the PR's own Testing notes ("The cap itself and the overflow message remain unit-only; there is no CLI surface that exposes them.") — answered-by-intent, not a new finding. - The cap bounds one tool result, not one turn — non-blocking-improvement. Several comfortably-under-budget results in one multi-tool-call turn could still collectively approach a small model's context.
Positive signals
- The line-boundary binary search in
cap_tool_resultguarantees the serialized truncated value fits the budget (aside from the dormant edge case above), and cutting only at line boundaries keeps kept text readable rather than chopping mid-line. isErroris explicitly carried over into the truncated form and tested in both directions, so a long-but-failed command can't silently read as a success once truncated.- The curated renderer the chat tool now calls (
render_model_registry_text_with_context_and_host) is the same functionrocm modelitself already calls frommain.rs— genuine reuse, not a new parallel implementation. - The PR description's mutation-style test evidence (each new check shown to fail against a deliberately broken input) checks out against the actual tests on inspection.
Deployment notes
The rocm_command MCP tool's default output for ["model"]/["models"] changes shape (curated list instead of the verbose per-recipe dump). Any existing external MCP client that depended on the old verbose-by-default behavior needs to pass ["model", "--verbose"] explicitly after this lands. A dash tool result over 16 KiB now arrives as a different ({"truncated": true, "tool_output": ...}) shape rather than the original envelope — documented in docs/llm-tool-use.md.
What this covered
Full diff 4f1f2f31…8c04cb8e (9 files, 859 insertions / 10 deletions), every changed file read completely at the pinned head commit via git show <sha>:<path> (not the mutable working tree — see below), plus calling/called code read in full to verify specific claims: apps/rocm/src/chat/summaries.rs, apps/rocm/src/chat/turn.rs, apps/rocm/src/model_catalog_text.rs, apps/rocm/src/cli.rs (flag verification). CI checked live via the GitHub API: every check green except one self-hosted GPU lane still queued, matching the pre-existing, already-documented runner-pool issue affecting multiple unrelated PRs on this repo — not a concern this change introduces.
Parallel review workers were spawned for this pass. A mid-task message purporting to relay their findings arrived through a channel that could not be authenticated — the agent identifiers it referenced did not match this session's own spawned workers — so it was not used for anything beyond a prompt to double-check; every finding above was instead independently re-derived and verified directly against the source at the pinned commit SHA, after the shared local checkout was separately observed to have been mutated mid-review by other concurrent activity in the environment. This makes the process closer to a single verified pass than a confirmed independent fan-out for coverage purposes, though every finding reported was individually confirmed against the code before inclusion.
Did not run cargo test/cargo clippy locally (relied on the PR's own green CI for the reviewed commit); no live mutation testing was performed — test-coverage verdicts above were reached by hand-tracing. Not reconciled against the PR's existing review threads (three prior automated rounds and one human changes-requested round, all against earlier commits) — reviewed blind, from the code and the PR's own description only, per method. Note the blocking finding above was not raised by any of those three prior rounds, each of which reviewed byte-identical content in the files it touches.
🤖 Automated by an AI agent
| if command.eq_ignore_ascii_case("model") || command.eq_ignore_ascii_case("models") => | ||
| { | ||
| Ok(render_model_registry_text_with_context_and_host( | ||
| Some(paths), | ||
| None, | ||
| None, | ||
| )) | ||
| } | ||
| [command, flag] |
There was a problem hiding this comment.
Blocking: this dispatch change silently disables the CLI chat's deterministic "ROCm CLI summary" block for model questions.
run_rocm_read_only_in_process's ["model"]/["models"] arm now returns render_model_registry_text_with_context_and_host's curated output. Its header comes from model_catalog_header (apps/rocm/src/model_catalog_text.rs), which prints "Recommended models - run one with \rocm serve `"(or the signed-index variant) — never the string"model recipes". Only the verbose renderer (render_model_registry_verbose_text_with_context_and_host) still emits writeln!(output, "model recipes")`.
deterministic_model_tool_summary (apps/rocm/src/chat/summaries.rs:320-321) is strictly gated on tool_text.contains("model recipes"). run_chat_read_only_tool's "rocm_command" arm (this file, ~line 708) feeds exactly this dispatch's output into turn.rs's follow_up_text, which is then passed straight to deterministic_chat_tool_summary (apps/rocm/src/chat/turn.rs:326).
So on rocm chat --prompt --tools — a sibling surface this same function serves — the "ROCm CLI summary" block (Recommended local assistant / Fit / Engine) that used to render whenever the model asked what it could run no longer appears, not because the feature was removed but because the one string it keys on is no longer produced by the new default path. Every existing test of deterministic_model_tool_summary/deterministic_chat_tool_summary (e.g. deterministic_model_tool_summary_suppresses_extra_local_model_follow_up in turn.rs, and the fixtures in summaries.rs) hand-writes a string containing "model recipes" literally rather than calling the real renderer or dispatch, so none of them catch this; no e2e scenario exercises it either.
Confidence 88 — traced end to end at the PR head commit: confirmed the new default output never contains the gate string, confirmed the consumer's exact gate, and confirmed the call chain connecting this dispatch to that consumer.
Fix: either make the summary recognize the curated output too (key off a stable marker both renderers share, or off the tool name/args rather than parsing renderer-specific text), or if this deterministic summary is deliberately being retired for model/models, remove it (and its now-stale fixtures) and say so in the PR description.
| /// one (`rocm_command` with `["model"]` measured 204,470 characters, about | ||
| /// 58k tokens) cannot fit in such a context at all and the provider rejects | ||
| /// the whole request. | ||
| pub(crate) const TOOL_RESULT_CHAR_BUDGET: usize = 16 * 1024; |
There was a problem hiding this comment.
Non-blocking: TOOL_RESULT_CHAR_BUDGET is named/documented as a "character" budget but enforced in UTF-8 bytes.
cap_tool_result's fit check (line ~365) uses value.to_string().len(), which is Rust's byte length, not a character count; docs/llm-tool-use.md likewise says "bounded to TOOL_RESULT_CHAR_BUDGET characters". Enforcement is internally consistent (bytes compared against bytes throughout), so this isn't a functional break for the ASCII-heavy CLI output these tools produce today, but multi-byte UTF-8 content (an accented path, a non-ASCII model id) would be cut well before 16 KiB of actual characters, contradicting the constant's name and both the code doc and the user-facing doc.
Confidence 85 · mechanical · Fix: rename to a byte-oriented name, or document the budget explicitly as bytes.
| let (mut low, mut high, mut best) = (0usize, cuts.len() - 1, 0usize); | ||
| while low <= high { | ||
| let mid = low.midpoint(high); | ||
| if truncated_tool_result(&full, cuts[mid], total_chars, is_error) | ||
| .to_string() | ||
| .len() | ||
| <= budget | ||
| { | ||
| best = mid; | ||
| low = mid + 1; | ||
| } else if mid == 0 { |
There was a problem hiding this comment.
Non-blocking: this binary search can return a cut that was never confirmed to fit budget, contradicting its own doc comment.
best is initialized to 0 here and is only supposed to be set once a candidate is measured to fit. If the minimal truncated form (cuts[0]: empty kept text plus the marker's own JSON overhead, a few hundred bytes) itself exceeds budget, the loop reaches mid == 0 and breaks (line 398) without ever validating cuts[0]/best, yet the function still returns truncated_tool_result(&full, cuts[best], ...) as if it had been checked.
This is dormant today: the sole caller (model_facing_tool_result) always passes the fixed TOOL_RESULT_CHAR_BUDGET = 16 * 1024, far above the marker's overhead, so the branch can't currently trigger, and no test exercises a smaller/variable budget.
Confidence 75 (traced precisely; not reproduced with a failing budget) · logic · Fix: validate cuts[0] fits before trusting best's initial value, or assert the budget is large enough for the marker.
| fn context_overflow_message(raw: &str) -> Option<String> { | ||
| if !raw.contains("exceed_context_size_error") { | ||
| return None; | ||
| } | ||
| let counts = json_number_field(raw, "n_prompt_tokens") | ||
| .zip(json_number_field(raw, "n_ctx")) | ||
| .map(|(used, available)| { | ||
| format!( | ||
| " ({} of {} tokens)", | ||
| group_thousands(used), | ||
| group_thousands(available) | ||
| ) | ||
| }) | ||
| .unwrap_or_default(); | ||
| Some(format!( | ||
| "A ROCm check returned more text than this model can read{counts}. \ |
There was a problem hiding this comment.
Non-blocking: the rewritten overflow message always attributes the cause to "A ROCm check returned more text than this model can read".
context_overflow_message detects the failure purely by matching "exceed_context_size_error" in the raw provider error string; nothing available at this call site distinguishes a single oversized tool result from context filled by ordinary accumulated conversation history across many turns, or a long pasted user message. In those cases the causal half of the sentence is wrong, even though the remedy half ("try a more specific question, or a model with a larger context") is still reasonable advice either way.
Confidence 78 · logic · Fix: a cause-neutral phrasing (e.g. "this conversation grew too large for the model to read"), or confirm the specific attribution is an acceptable simplification.
Main moved again while this branch's review round was in flight (#1041 and #1069 landed). One file conflicted, `apps/rocm/src/chat/mcp_bridge.rs`, in two places, both the same shape: this branch and main each appended to the same list. The `use crate::{...}` group is resolved as the union — this branch's `host_gpu_support_level` and main's `render_model_registry_text_with_context_and_host` are both kept. Main's new test uses the latter, so dropping either side would have failed to compile rather than failing quietly. The test module's two appended blocks (this branch's engines-tool Vulkan tests, main's model-tool context-size tests) are independent and both kept whole; neither was spliced into the other. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Problem
Asking the
rocm dashChat tab anything that makes the model callrocm_commandwith["model"]ended the turn with the provider's raw HTTP 400 instead of an answer:The tool result measured 204,470 characters — roughly 58k tokens, more than a locally
served model's entire context. llama.cpp defaults
n_ctxto 32768, so any local modelwith a context of 64k tokens or less reproduces it. The user got the raw provider error
and no answer.
Four things compounded, and all four are addressed.
What changed
The chat
model/modelstool rendered the wrong thing. It called therocm model --verbosediagnostics renderer — every recipe with its aliases, enginerecipes, flags and policy notes — rather than the curated list
rocm modelprints. It nowrenders the curated list; the diagnostics need an explicit
["model", "--verbose"].The output was sent twice. The MCP envelope carries a command's output as the content
text and again as
structuredContent.stdout, and the dash passed the whole JSON value tothe model. The envelope itself is left alone —
rocm mcp-callconsumers read thestructured field, and an in-tree e2e step does exactly that — so the duplicate is dropped
in the dash, where the model is the only reader.
Nothing bounded a tool result at all. A read-only result is now capped at 16 KiB of
serialized JSON, cut on a line boundary and closed with a marker naming how much was
omitted, so neither the model nor the user can take a truncated list for a complete one.
The budget is checked against the actually-serialized string, so JSON escaping cannot push
a capped result back over it. Capping replaces the envelope with a smaller object, so
isErroris carried over explicitly: a command that failed with more output than thebudget allows must not reach the model looking like a success, since that field is what
the chat path reads to decide.
The example steered small models at the worst case. The
rocm_commanddescription andits
argsschema both used["model"], which a small local model copies literally; theynow use
["config", "show"].Separately, a provider context overflow now reads as a plain sentence in the Chat tab
naming the token counts and a way forward, with the raw provider body still logged to
rocm-cli.log.How it was checked
The headline number, measured on one binary with only the renderer swapped between runs:
rocm mcp-call rocm_command {"args":["model"]}{"args":["model","--verbose"]}rocm model, for comparisonA 99× reduction on the default path, with the diagnostics still reachable.
Mutation evidence — every new check was shown to fail against a deliberately broken input,
and its siblings stayed green, so the checks are independent rather than coupled:
210220characters reached the model)chat_error_messagereturns the raw bodymodeldispatch pointed back at the verbose renderermodel-catalog-04scenarioGates, all run on a Linux box because this workspace does not build on macOS (four
pre-existing Darwin type errors in an unrelated crate, present on
main):cargo test --workspace --all-targets --exclude e2e-cucumber(58 test binaries, 0 failed),cargo clippy --workspace --all-targets -- -D warnings(clean),python scripts/smoke_local.py(
smoke: ok), andcargo xtask e2e -- -n model-catalog(all scenarios pass, including thetwo new ones).
Not verified: no live dash run against a real 32k-context model. The claim that a turn
now gets an answer instead of HTTP 400 rests on the measured size reduction plus the
unit-tested cap, not on an end-to-end chat.
docs/manual-testing.md§13 is written so ahuman can close that, including a
--ctx-size 2048case. The truncation path itself wasexercised with a synthetic 200 KB fixture — after this change no read-only tool on the test
host produces an oversized result.
Scope
The cap is dash-only, and that is deliberate.
model_facing_tool_resultis appliedinside the dash's
run_rocm_read_tool, which is the single funnel every read-only chattool result passes through. The CLI's own tool-using path (
rocm chat --prompt --tools)reads
mcp_tool_result_textstraight through with no budget and is not capped here.The curated-vs-verbose split is shared between both paths, so the one reported symptom is
fixed on both — but a different large result on the CLI path (
["logs"]on a busy host, oran explicit
["model","--verbose"]) can still reach a small-context model unbounded. Thatexposure is pre-existing rather than introduced, and capping a second surface is a larger
change than the reported bug needs. It is stated in
docs/llm-tool-use.mdrather than leftfor a reader to discover, and is the obvious follow-up.
The 16 KiB budget is fixed, not context-aware. It leaves room in a 32k context for the
system prompt plus several turns, and matches llama.cpp's common default with headroom. A
budget derived from the provider's reported
n_ctxwould serve a--ctx-size 2048hostand a very large context more precisely, at the cost of plumbing that value through to the
tool layer. The rewritten overflow message is the second line of defence for what a fixed
budget misses. Flagging it as a judgement call rather than an oversight.
Out of scope: the base cost per turn (roughly 4,200 tokens of system prompt plus the tool
schemas) is heavy for 1.5B-class models, and is a separate concern.
Testing notes
The dash TUI's chat path has no Gherkin coverage, but the behaviour behind this change is
observable from the CLI:
rocm mcp-call rocm_commandis the same tool dispatch the Chat tabdrives, so the two new scenarios assert there rather than leaving it to unit tests alone —
the same shape as the existing
service_logscoverage. The cap itself and the overflowmessage remain unit-only; there is no CLI surface that exposes them.
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. — no xfail row covers this behaviour; nothing to narrow.docs/architecture.md. — n/a, no new subcommand or subsystem.docs/architecture.mdhas the matching entry. — n/a, no production source file added, renamed or removed; the twodocs/architecture.mdedits refresh descriptions of files whose role changed.