Repository navigation
fix(logs): read every log tail through one bounded, lossy routine and stop reporting a log with content as empty - #961
Conversation
Three modules read the last lines of a log their own way: the ComfyUI log view (whole file, sixteen-bit decode, ANSI strip, lossy text), the services text module (streamed, fails on invalid text) and the rocmd MCP service_logs tool (whole file into one string, fails on invalid text), while the seek-bounded routine in rocm-core serves only the engines. Add characterisation tests that pass on the unmodified code and document the differences before they are removed: a large file (the whole-file readers return a line far outside any tail window; the shared routine does not), a sixteen-bit little-endian log with ANSI colour codes, invalid eight-bit text, a missing file, a limit of zero, fewer lines than the limit, a final line without a trailing newline, and CRLF endings. Test-only; no behaviour change. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The ComfyUI log view, the services text module and the rocmd MCP service_logs tool each read a log tail their own way, while the seek-bounded rocm_core::tail_lines served only the engines. Converge all of them on it. rocm_core::log_tail gains tail_lines_decoded, taking a LogDecode: PLAIN (UTF-8, what the engines and the service logs use) or CONSOLE (UTF-16 little-endian detected by byte order mark or zero-byte pattern, ANSI sequences stripped; what the ComfyUI log view used). The UTF-16 path keeps the seek-boundary handling: the window starts on a character boundary and a partial first line is dropped. strip_ansi_sequences is exported for the assistant-prose note in tool_policy. A shared LOG_VIEW_MAX_TAIL_READ budget (4 MiB) is what the three log views pass. The private readers are deleted; each caller keeps a thin adapter that picks a LogDecode and its own error label. User-visible differences: - A log larger than 4 MiB is read from its last 4 MiB, not whole (all three). - Invalid UTF-8 no longer fails the view: `rocm services logs` and `rocm logs` showed one "<failed to read ...>" placeholder line and the MCP service_logs tool returned an error; both now show the readable text with U+FFFD for the bad bytes. - Service logs and the MCP tool still read UTF-8 and keep ANSI sequences; a UTF-16 service log is now shown as lossy text rather than as an error. - ComfyUI logs: ANSI stripping is per line, so an unterminated escape sequence ends at its line instead of swallowing the lines after it. - Error wording: a missing log is "failed to read <label> <path>" (cause: "failed to open <path>") in services text, was "failed to open <label> <path>"; ComfyUI and MCP name the open step in the cause. - A limit of 0 returns no lines without opening the file in all three (the MCP reader used to return the whole file; ComfyUI and MCP used to fail on a missing file first). Callers clamp the limit, so only direct calls see it. The characterisation tests from the previous commit are updated to the new behaviour, and new tests cover the bound (a counting reader shows a read touches no more than its budget), every budget from nothing to the whole file for UTF-8 and UTF-16, and each caller through the shared routine. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
`rocm logs --search` reads the CLI's action logs through the shared log-tail routine. Add scenario logs-04: an action log with one byte that is not valid text still yields every matching line. Before the log views were converged on the shared routine, that byte made the whole file collapse into a single "failed to read" placeholder line and the search found nothing. The planting step gains an option for the stray byte; the existing steps and the match-count assertion are reused. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
… trailing escape fragment Two ways the shared tail read disagreed with what it advertises, both found in review. The UTF-8 branch seeked from the size taken by the earlier stat and then read to the current end of the file, so a log whose writer kept appending was consumed past the byte budget by however much it had grown. The sixteen-bit branch already capped its read with take(size - window_start); the UTF-8 branch now does the same. The new test hands the routine a source that grows on its first read and asserts the lines stop where the snapshot ended and the bytes touched stay within the window; it is red with the cap removed. A console log that ends in a colour reset after its last newline ended in an unterminated fragment made only of an escape sequence. The routine counted it as a line, so with a limit of one it evicted the real last line and returned an empty string, where the ComfyUI reader it replaced (strip ANSI from the whole text, then split) returned the line. A final unterminated fragment that strips to nothing is no longer counted when ANSI stripping is on; a terminated blank line, including one made only of an escape sequence, is still a line, and plain decoding strips nothing so it keeps the fragment. The tests cover UTF-8 and UTF-16 with and without a byte order mark, and compare the result with a copy of the old reader on inputs where the two agree. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…utput The log views read at most the last 4 MiB of a log and drop a partial first line. When the newest line is itself longer than that bound the window holds only a fragment of it, so the read returned no lines and the views printed "The saved log is empty." (rocm comfyui logs) and "Recent output: no output yet" (rocm services logs) for a file that is not empty. The byte bound stays. rocm-core gains tail_log_decoded, which returns the same lines in a LogTail that also records how many bytes at the start of the file the budget left unread, and LogTail::newest_line_exceeds_the_read_bound tells an empty tail of a log with content from an empty log (or a limit of zero). The two views use it: an empty file still says it is empty, and a log whose newest line does not fit says so and names the bound, derived from LOG_VIEW_MAX_TAIL_READ so the number cannot drift from it. The unit tests and four new scenarios (logs-05 to logs-08, one pair per view) assert each message together with the log on disk that it describes: the size and the absence of a newline for the long line, zero bytes for the empty file. The rocmd MCP service_logs tool and the chat and launch-failure readers print no claim about emptiness, so they are unchanged. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…s the read bound in the service_logs tool The tool read a log's tail without saying what the byte budget left unread, so an empty log and a log whose newest line is longer than the 4 MiB a log view reads both came back as "Read the last 80 line(s)" with an empty tail. The CLI log views already separate the two; the tool is the same class. It now answers through one shared sentence, rocm_core::service_log_tool_summary (no output yet, newest line too long to show, or how many lines were read), and its structured reply carries newest_line_exceeds_read_bound. Both copies of the tool use it: the daemon's MCP server and the CLI's own tool bridge (rocm mcp-call). The notice clause moves from the CLI into rocm-core so the daemon, which cannot depend on the CLI, words it the same way, derived from the read budget. The line count in the summary is now the number of lines returned, not the number asked for. A missing log file is still an error. Scenarios logs-09 and logs-10 run rocm mcp-call service_logs against an over-long line and an empty log and check the reply against the file on disk; the daemon's copy is covered by a rocmd unit test, since the harness drives only the rocm binary. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…hort clippy's too_long_first_doc_paragraph rejects the two doc comments added with the service_logs summary; split each into a one-sentence summary and the rest. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…utine The log views now read a log tail through one bounded, lossy routine and the logs scenarios cover it, but docs/testing.md only described the service_logs tool. Say which scenario covers which view (logs-08 to logs-12) and where the routine's own tests run, next to the existing service_logs paragraph. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟡 Changes recommended
The lines-only adapter discards oversized-line metadata, causing misleading empty-log or no-activity messages instead of an omitted-output placeholder.
1 open issue
What changed in this PR
This PR consolidates log-tail reading in rocm-core so the CLI, ComfyUI view, and daemon share bounded, lossy decoding without introducing a dependency between the two applications.
Changes:
- Replaces separate readers with shared tail routines and a 4 MiB log-view budget.
- Distinguishes empty logs from oversized lines and standardizes
service_logssummaries. - Adds decoding, read-bound, and output regression tests, plus seven end-to-end scenarios.
| File | Description |
|---|---|
| tests/e2e-cucumber/tests/e2e/service_cleanup_steps.rs | Exposes record fixtures for log scenarios. |
| tests/e2e-cucumber/tests/e2e/logs_steps.rs | Adds log fixtures and CLI/tool output assertions. |
| tests/e2e-cucumber/features/logs.feature | Adds invalid-byte, oversized-line, and empty-log scenarios. |
| docs/testing.md | Documents log-tail behavior and regression coverage. |
| docs/architecture.md | Updates shared-reader and caller descriptions. |
| crates/rocm-core/src/log_tail.rs | Implements shared decoding, bounded reads, summaries, and tests. |
| crates/rocm-core/src/lib.rs | Exports shared log-tail APIs. |
| apps/rocmd/src/mcp.rs | Uses shared tails and summaries for service_logs. |
| apps/rocm/src/tool_policy.rs | Updates a comment naming the ANSI-stripping helper. |
| apps/rocm/src/services_text.rs | Adopts bounded reads and oversized-line notices. |
| apps/rocm/src/comfyui.rs | Replaces private decoding with the shared console reader. |
| apps/rocm/src/chat/mcp_bridge.rs | Aligns CLI tool replies with shared log summaries. |
Review effort: Balanced
…e logs browser The lines-only reader behind the logs browser dropped what the byte budget left unread, so a command, screen or lifecycle log whose only line is longer than the 4 MiB a log view reads came back as an empty list. `rocm logs` then printed "No logs found yet." and the recent activity summary said "no activity yet" although the files are on disk. The same reader feeds the managed-engine startup failure detail, which likewise left the log out. read_optional_tail_lines now returns one placeholder line, `<no output shown from <label>: the newest line is longer than the 4 MiB a log view reads>`, for such a log, the way it already returns one `<failed to read ...>` line for a read error; a missing or empty log is still an empty list. The recent activity summary and the lifecycle entries of the browser read the tail with its bound and say "none shown" with the same clause the service log view uses, instead of formatting a placeholder as a lifecycle record. Unit tests cover the reader, the browser for a command and a screen log, the lifecycle summary and the startup failure detail, and scenario logs-15 runs `rocm logs` against a command log whose only line exceeds the bound and checks the printed line together with the file on disk. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
r0x0r
left a comment
There was a problem hiding this comment.
I went through this one carefully and it holds up, so I'm approving it. Two nits left inline, neither of them a blocker.
The thing I most wanted to break was the extracted tail routine, so I ran it differentially against a whole-file reference: 268,286 combinations of content, budget and decoding, covering UTF-16 both with and without a BOM, CRLF, a lone 0xff, seeks landing mid-multibyte, and ANSI. Zero mismatches. The only way I could provoke a false newest_line_exceeds was with a budget of two bytes or less, which the 4 MiB callers can never reach.
The tests are the kind I like to see: each one asserts the message together with the on-disk state that message describes, per AGENTS.md section 3. I also walked the PR description against the diff claim for claim, and it matches.
Of the two inline notes, the truncation race on the probe read is the one I'd act on before merge. It isn't a regression, tail_lines has had it all along, but this PR widens its reach from the engine adapters to every log view and the service_logs tool. The other one is a naming collision that will cost somebody an afternoon later on.
CI is green apart from E2E tests (W7900, current), which is still queued; every other E2E lane passed.
| source | ||
| .seek(SeekFrom::Start(seek_pos - unit)) | ||
| .with_context(|| format!("failed to seek in {}", path.display()))?; | ||
| let mut probe = [0u8; 1]; | ||
| file.read_exact(&mut probe) | ||
| let mut probe = [0u8; 2]; | ||
| let probe = &mut probe[..unit as usize]; | ||
| source | ||
| .read_exact(probe) | ||
| .with_context(|| format!("failed to read from {}", path.display()))?; |
There was a problem hiding this comment.
A log truncated between the stat and this probe read now fails the whole view. read_exact on the probe byte returns UnexpectedEof if the file shrank below seek_pos after metadata.len() was taken. I ran the production routine against a shrunk source to confirm: Err("failed to read from t: failed to fill whole buffer").
The race is real rather than theoretical. service_launch.rs:891 does fs::File::create(&record.log_path) on every restart, and services_text.rs's own follow cursor already handles exactly this case ("Truncated or replaced (a restart reuses the path)").
It is pre-existing in tail_lines, but this PR extends its reach from the engine adapters to every log view and the service_logs tool, reached via apps/rocm/src/services_text.rs:1377, apps/rocmd/src/mcp.rs:850 and read_log_tail in apps/rocm/src/comfyui.rs:683.
Fix: treat a short probe or window read as "the file shrank" and fall back to window_start = 0.
This is the one I'd act on before merge, though to be clear it is not a regression relative to today's behaviour in the adapters.
There was a problem hiding this comment.
Confirmed and fixed in 8840d0e. The routine took the size once and then probed the byte before its window with read_exact, so a log truncated between the two failed the whole view with failed to fill whole buffer; the race is real because a restart recreates the service log. I reproduced it by running the production routine over a source that shrinks when a read reaches a chosen offset, before changing the code. There was a second shrink point beyond the one the comment names: a log that shrank after the probe made the window read come back short, which showed no lines at all rather than an error.
The read now treats an early end of the source at the probe, and a window (UTF-8 or UTF-16) shorter than the size it was given, as the log having shrunk. It measures the source again and starts over from that size, which reads a log that is now small enough from its start and reads one that is still larger than the budget from its new end. A log that keeps shrinking through three passes reads as empty rather than failing. A log that did not shrink takes the same path as before. The callers all reach this one function (the engine adapters through tail_lines, services_text.rs, the rocmd service_logs tool and the ComfyUI view), so none needed a change; their suites pass.
Tests in log_tail.rs use a pure ShrinkingSource, so they run on every platform: a_log_that_shrank_before_the_probe_is_measured_again_instead_of_failing, a_log_that_shrank_before_the_window_read_is_measured_again, a_log_that_shrank_but_is_still_larger_than_the_budget_is_read_from_its_new_end, a_sixteen_bit_log_that_shrank_is_measured_again_at_the_probe_and_the_window, a_whole_file_read_of_a_log_that_shrank_is_measured_again, a_log_that_keeps_shrinking_ends_in_an_empty_tail_not_an_error and a_log_that_did_not_shrink_reads_exactly_as_before. With the probe's early-end case disabled, or with the short-window checks disabled, three of them fail each time.
| - [`service_records.rs`](../apps/rocm/src/service_records.rs) — mechanical relocation, owns types: removing one managed-service record and every file it owns (`remove_managed_service_record`, `service_record_artifacts`) and the `rocm services prune` plan/render/apply steps (`build_service_prune_plan`, `render_service_prune_plan`, `apply_service_prune_plan`, `prune_managed_service_records`) with the age gate behind them (`DEFAULT_SERVICE_PRUNE_MIN_AGE_HOURS`, `service_prune_min_age_hours`). It reads record state through [`managed_service_status.rs`](../apps/rocm/src/managed_service_status.rs) and [`services_text.rs`](../apps/rocm/src/services_text.rs) and builds its advice text with [`service_hints.rs`](../apps/rocm/src/service_hints.rs); none of the three depends on it. `remove_managed_service_record`, `prune_managed_service_records`, `service_prune_min_age_hours` and `DEFAULT_SERVICE_PRUNE_MIN_AGE_HOURS` are `pub(crate)` (`services()` in [`services_cmd.rs`](../apps/rocm/src/services_cmd.rs) calls the first three; the `ServicesCommand::Prune` clap definition in [`cli.rs`](../apps/rocm/src/cli.rs) uses the constant), and `ServicePruneOutcome`, the return type of `prune_managed_service_records`, is `pub(crate)` with `pub(crate)` `text`, `removed_records`, `removed_files` and `failures` fields that `services()` reads. Every other item is private. The cluster's regression tests live in this module's own `mod tests`, reaching the shared `test_paths`, `plant_service_record` and `plant_service_record_without_its_key` fixtures through `crate::tests::` (the last two widened to `pub(crate)` for that). | ||
| - [`services_cmd.rs`](../apps/rocm/src/services_cmd.rs) — mechanical relocation, dispatch-adjacent, no owned types: `rocm services`' command handler (`services()`, which dispatches `list`, `logs`, `stop`, `restart`, `remove` and `prune`) and `run_approved_service_action`, the approval-gated stop/restart path. `ServicesCommand` lives in [`cli.rs`](../apps/rocm/src/cli.rs). Both functions are `pub(crate)`: `dispatch()` calls `services()` through a re-import, and [`advised_commands.rs`](../apps/rocm/src/advised_commands.rs)'s test reaches `run_approved_service_action` through a test-only re-import in `main.rs`. The module reads from [`services_text.rs`](../apps/rocm/src/services_text.rs), [`service_control.rs`](../apps/rocm/src/service_control.rs), [`service_records.rs`](../apps/rocm/src/service_records.rs), [`service_hints.rs`](../apps/rocm/src/service_hints.rs) and [`cli_audit.rs`](../apps/rocm/src/cli_audit.rs); none of them calls back into it. The cluster's one regression test lives in this module's own `mod tests`, reaching the shared `test_paths` fixture through `crate::tests::`. | ||
| - [`services_text.rs`](../apps/rocm/src/services_text.rs) — mechanical relocation, owns private types (Phase 6f, ROCMAI-209): the CLI/TUI logs browser (`render_logs_text`, `render_logs_browser_text`, `render_logs_browser_page_text`, `render_logs_browser_page_text_with_options`, `logs_browser_total_pages`, the `LogBrowserEntry` type, `collect_log_browser_entries`, `filter_log_browser_entries`, `log_browser_source_label`, `write_past_attempts_hint`), the `rocm services`/`rocm services logs` status surfaces (`render_services_text`, `render_services_json`, `render_service_logs_text`, `render_service_logs_text_with_options`, `render_service_action_result`, `read_managed_service_manifest`, `load_managed_service`), the `--follow` tail behind `rocm logs --service <id> --follow` and `rocm services logs <id> --follow` (`follow_service_logs`, `follow_service_logs_with`, the `LogTail`, `LogSegment` and `FollowFooter` types, `write_log_segments`, `newly_ready`; new code written in this module, not relocated), the internal `rocmd`-compatible status surface (`render_internal_status_text`), the `stop_server` answer shape and its renderer (`internal_stop_server_value`, `service_stop_unconfirmed`, `render_unconfirmed_service_stop`, `process_check_command`), plus `validate_service_id`, `list_log_files`, `read_optional_tail_lines`, and the CLI-lifecycle-log tail formatting helpers (`format_cli_lifecycle_tail_line`, `lifecycle_event_label`, `humanize_log_label`, `lifecycle_field`, `read_tail_lines`). The managed-service stop and restart implementation that used to live here is in [`service_control.rs`](../apps/rocm/src/service_control.rs). The `LogBrowserEntry` struct moved here from `main.rs` with the logs browser and stays private. `dispatch()` calls directly into this module at several points — `render_internal_status_text`, `render_service_logs_text`, `render_logs_browser_text`, and `render_logs_text` — all re-imported via `use crate::services_text::{...};`; [`services_cmd.rs`](../apps/rocm/src/services_cmd.rs)'s `services()` calls `render_services_text`, `render_services_json` and `render_service_logs_text` through its own imports. `render_service_action_result`, `service_stop_unconfirmed` and `validate_service_id` are also `pub(crate)` and read from `services_cmd.rs`'s `run_approved_service_action`; `validate_service_id` is also read from [`service_records.rs`](../apps/rocm/src/service_records.rs)'s `remove_managed_service_record`/`service_record_artifacts`, and `load_managed_service` from its `remove_managed_service_record`/`apply_service_prune_plan`. `load_managed_service` is `pub(crate)`, read from [`runtime_services.rs`](../apps/rocm/src/runtime_services.rs) (reconciling live servers against a runtime activation), from [`service_control.rs`](../apps/rocm/src/service_control.rs) and from [`managed_service_status.rs`](../apps/rocm/src/managed_service_status.rs)'s own test module; `read_managed_service_manifest` and `internal_stop_server_value` are `pub(crate)` for `service_control.rs`, and `process_check_command` for its tests; `read_tail_lines`, `render_logs_text`, `render_logs_browser_text`, `render_services_text`, `render_service_logs_text`, and `validate_service_id` are also `pub(crate)`, read from [`chat/mcp_bridge.rs`](../apps/rocm/src/chat/mcp_bridge.rs)'s read-only command runner; `read_optional_tail_lines` and `render_service_action_result` are also `pub(crate)`, read from [`service_launch.rs`](../apps/rocm/src/service_launch.rs)'s managed-spawn/attach path. Every other item in this list has no caller outside this module and stays private. The cluster's regression tests live in this module's own `mod tests`, reaching the shared `test_paths` fixture through `crate::tests::` the same way other mechanically relocated modules do; `assert_no_retry_line` stays here as a `pub(crate)` test helper because it checks the unconfirmed-stop rendering and `service_launch.rs`'s tests read it too. Two tests that call this module's `load_managed_service` only incidentally — their primary subject is `runtime_services::pin_service_record_to_runtime`/`restore_service_record_pin` and `runtime_services::restart_service_onto_runtime_with`'s outcome classification respectively — stay in `main.rs`'s shared `mod tests` instead. | ||
| - [`services_text.rs`](../apps/rocm/src/services_text.rs) — mechanical relocation, owns private types (Phase 6f, ROCMAI-209): the CLI/TUI logs browser (`render_logs_text`, `render_logs_browser_text`, `render_logs_browser_page_text`, `render_logs_browser_page_text_with_options`, `logs_browser_total_pages`, the `LogBrowserEntry` type, `collect_log_browser_entries`, `filter_log_browser_entries`, `log_browser_source_label`, `write_past_attempts_hint`), the `rocm services`/`rocm services logs` status surfaces (`render_services_text`, `render_services_json`, `render_service_logs_text`, `render_service_logs_text_with_options`, `render_service_action_result`, `read_managed_service_manifest`, `load_managed_service`), the `--follow` tail behind `rocm logs --service <id> --follow` and `rocm services logs <id> --follow` (`follow_service_logs`, `follow_service_logs_with`, the `LogTail`, `LogSegment` and `FollowFooter` types, `write_log_segments`, `newly_ready`; new code written in this module, not relocated), the internal `rocmd`-compatible status surface (`render_internal_status_text`), the `stop_server` answer shape and its renderer (`internal_stop_server_value`, `service_stop_unconfirmed`, `render_unconfirmed_service_stop`, `process_check_command`), plus `validate_service_id`, `list_log_files`, `read_optional_tail_lines`, and the CLI-lifecycle-log tail formatting helpers (`format_cli_lifecycle_tail_line`, `lifecycle_event_label`, `humanize_log_label`, `lifecycle_field`, `read_log_tail`). The managed-service stop and restart implementation that used to live here is in [`service_control.rs`](../apps/rocm/src/service_control.rs). The `LogBrowserEntry` struct moved here from `main.rs` with the logs browser and stays private. `dispatch()` calls directly into this module at several points — `render_internal_status_text`, `render_service_logs_text`, `render_logs_browser_text`, and `render_logs_text` — all re-imported via `use crate::services_text::{...};`; [`services_cmd.rs`](../apps/rocm/src/services_cmd.rs)'s `services()` calls `render_services_text`, `render_services_json` and `render_service_logs_text` through its own imports. `render_service_action_result`, `service_stop_unconfirmed` and `validate_service_id` are also `pub(crate)` and read from `services_cmd.rs`'s `run_approved_service_action`; `validate_service_id` is also read from [`service_records.rs`](../apps/rocm/src/service_records.rs)'s `remove_managed_service_record`/`service_record_artifacts`, and `load_managed_service` from its `remove_managed_service_record`/`apply_service_prune_plan`. `load_managed_service` is `pub(crate)`, read from [`runtime_services.rs`](../apps/rocm/src/runtime_services.rs) (reconciling live servers against a runtime activation), from [`service_control.rs`](../apps/rocm/src/service_control.rs) and from [`managed_service_status.rs`](../apps/rocm/src/managed_service_status.rs)'s own test module; `read_managed_service_manifest` and `internal_stop_server_value` are `pub(crate)` for `service_control.rs`, and `process_check_command` for its tests; `read_log_tail`, `render_logs_text`, `render_logs_browser_text`, `render_services_text`, `render_service_logs_text`, and `validate_service_id` are also `pub(crate)`, read from [`chat/mcp_bridge.rs`](../apps/rocm/src/chat/mcp_bridge.rs)'s read-only command runner; `read_optional_tail_lines` and `render_service_action_result` are also `pub(crate)`, read from [`service_launch.rs`](../apps/rocm/src/service_launch.rs)'s managed-spawn/attach path. Every other item in this list has no caller outside this module and stays private. The cluster's regression tests live in this module's own `mod tests`, reaching the shared `test_paths` fixture through `crate::tests::` the same way other mechanically relocated modules do; `assert_no_retry_line` stays here as a `pub(crate)` test helper because it checks the unconfirmed-stop rendering and `service_launch.rs`'s tests read it too. Two tests that call this module's `load_managed_service` only incidentally — their primary subject is `runtime_services::pin_service_record_to_runtime`/`restore_service_record_pin` and `runtime_services::restart_service_onto_runtime_with`'s outcome classification respectively — stay in `main.rs`'s shared `mod tests` instead. |
There was a problem hiding this comment.
Anchored on this doc line because it's the half of the issue that lands in the diff; the code half is apps/rocm/src/services_text.rs:564.
Two different LogTail types now live in services_text.rs: the private follow cursor declared at services_text.rs:564, and rocm_core::LogTail, the bounded-read result, used throughout the same file. This entry still lists "the LogTail ... type" without distinguishing them. AGENTS.md section 6 names this exact hazard for the two "spinner" types.
Renaming the follow cursor, to FollowCursor say, would settle both.
There was a problem hiding this comment.
Agreed, and renamed in aaa6670. The private follow cursor in services_text.rs is now FollowCursor, with its constructor calls, tests and comments, and its doc comment, the module comment and the architecture entry say that it follows a growing file and is not rocm_core::LogTail, the result of a bounded tail read, which the entry now links. It is a rename only. I also swept the rest of this PR for same-named types: LogDecode and LogTail exist only in rocm-core, the new private Pass in log_tail.rs has no namesake in that crate, and the three read_log_tail adapters are functions that each wrap the one routine for their own caller, not types.
…he read The tail read took the file size once and then probed the byte before its window with read_exact. A service restart recreates its log, so a log truncated between the size and that probe made read_exact fail with "failed to fill whole buffer" and the whole view with it; a log that shrank after the probe read short and showed no lines. The read now treats an early end of the source, at the probe or in the window (UTF-8 and UTF-16), as the log having shrunk: it measures the source again and starts over from that size, which reads a log that is now small enough from its start. After a few passes a log that will not hold still reads as empty rather than failing. A log that did not shrink is read exactly as before. Tests run the production routine over a source that shrinks when a read reaches a chosen offset: before the probe, before the window, still larger than the budget, UTF-16 at both points, a whole-file read, and a source that never settles. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…sult services_text.rs declared a private LogTail, the cursor that reads what a growing log gained for --follow, and also uses rocm_core::LogTail, the result of a bounded tail read, so one file held two unrelated types of the same name and the architecture entry listed "the LogTail type" without saying which. Rename the private one to FollowCursor, with its constructor calls, tests and comments, say in its doc comment and in the architecture entry which is which, and link the shared one. A rename only; no behaviour changes. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…terator Clippy rejects collecting format! results into a String; write the lines with writeln! instead. Test-only. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head ab0695bb91a
Outcome: needs work — 3 blocking findings, all narrow and cheaply fixable; CI is fully green and the new bounded/lossy tail routine itself (crates/rocm-core/src/log_tail.rs) held up well under a dedicated review pass (boundary math, the shrink/retry race handling, and the trailing-escape-fragment logic all checked out against the code and its tests).
Blocking
-
MCP
service_logsstructured reply echoes the requested line limit, not the lines actually returned —apps/rocmd/src/mcp.rs:637,apps/rocm/src/chat/mcp_bridge.rs:221
Both call sites buildjson!({ ..., "lines": lines, ..., "tail": log_tail.lines.join("\n") })wherelinesis the clamped requested limit (arguments.get("lines")...clamp(1, 500)), whiletailand the prose summary (rocm_core::service_log_tool_summary) correctly uselog_tail.lines.len()— the actual count returned, per the PR's own stated fix. So requesting 50 lines on a 10-line log yields"lines": 50next to a 10-linetail: the prose and structured halves of the same reply disagree, and nothing in the test suite asserts the structured field's value.
Confidence 85 · logic · Fix:"lines": log_tail.lines.len()at both sites (or rename the field if echoing the request is deliberately wanted, and add a separate "returned" count). -
UTF-16 sniff doc comment misstates its own threshold by 2x —
crates/rocm-core/src/log_tail.rs:33-34(doc) vs:418(code)
The doc says UTF-16-without-BOM is detected "when at least a quarter of the odd-numbered bytes in its first 512 bytes are zero." The code checkszero_odd_bytes * 4 >= head.len(); since odd-indexed bytes are ~half ofhead.len(), this actually requires half of the odd bytes to be zero, not a quarter. Low functional risk (genuine mostly-ASCII UTF-16 text is close to 100% zero odd bytes either way), but it's a new, factually wrong claim about new public-API behavior.
Confidence 85 · mechanical · Fix: say "at least half" instead of "a quarter" (or loosen the code tozero_odd_bytes * 8 >= head.len()if a quarter was the intent). -
docs/manual-testing.mdnot updated for the new observable messages, despite the repo's own contributor guide requiring it in the same change —docs/manual-testing.md(untouched; see the existingrocm services logswalkthrough)
This PR introduces new user-visible strings (e.g. "Recent output: none shown; the newest line is longer than the 4 MiB a log view reads", the ComfyUI equivalent, and the MCP tool's three-way summary) and updatesdocs/testing.mdanddocs/architecture.mdaccordingly, but the manual-testing walkthrough forrocm services logsstill doesn't mention the new empty-vs-oversized-line distinction.
Confidence 80 · mechanical · Fix: add a short bullet to the manual-testing walkthrough covering the new distinction, or state explicitly in the PR why it's intentionally left to the e2e scenarios.
Non-blocking / decisions for the author
None beyond the above.
What this covered: full diff 3dc4a31a…ab0695bb (12 files, ~2.6k added lines) — the shared log_tail.rs routine, all three callers (services_text.rs, comfyui.rs, apps/rocmd/src/mcp.rs, chat/mcp_bridge.rs), the new e2e scenarios/steps, and docs — each read in full by an independent pass, plus history/comment/agent-instruction checks. CI on the PR is green across the board.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
…mber asked for The structured reply put the requested line limit in "lines" while "tail" and the summary used the lines actually read, so asking for 50 lines of a ten-line log said "lines": 50 beside a ten-line tail. "lines" is now the number of lines returned, and the number asked for (after clamping to 1 to 500) is reported as "lines_requested". Both copies of the tool change: the daemon's MCP server and the CLI's own tool bridge behind rocm mcp-call. Nothing in the tree reads the field. A unit test per copy asks for more lines than the log has, for fewer, and for an empty log, and checks both fields; scenario logs-16 runs rocm mcp-call service_logs for 50 lines of a three-line log and checks the reply against the file on disk. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
LogDecode documented the detection of a log without a byte order mark as a quarter of the odd-numbered bytes in its first 512 bytes being zero, but the check is zero_odd_bytes * 4 >= sampled length, which is half of the odd positions. The check is the rule the ComfyUI reader used before the views shared one routine, so the documentation changes rather than the rule. A test pins the boundary: 4 of 8 odd bytes zero is UTF-16 and 3 is not, likewise 128 and 127 within the 512 byte sample, and a source under 8 bytes is never UTF-16. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…om one they cannot show Add the observable messages to the local server records walkthrough: the none-shown line for a log whose newest line is longer than the 4 MiB a log view reads in rocm services logs, rocm logs and rocm comfyui logs, the lossy decoding of a stray byte, and the three summaries and the lines and lines_requested fields of the service_logs tool. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The walkthrough told the reader to replace a server's log file and then expected rocm logs to list it, but without --service that command reads the activity log and the files under logs/cli/ and logs/tui/ in the data folder, never a server's log. Split the check by view and give each the exact line it prints: the server log through rocm services logs or rocm logs --service, the rocm logs browser over a file in logs/cli/ (and the activity log), and rocm comfyui logs over a file under apps/comfyui/logs/. The service_logs tool bullet now says it reads the same server log. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…n Windows PR 966 re-exports managed_engine_startup_failure_detail from service_launch only off Windows, but the services_text log-tail tests call it through crate::service_launch and are compiled on Windows too, so the Windows clippy leg failed with E0425 in the merge queue. Re-export it for Windows test builds only: non-test Windows builds gain no unused import and Linux is unchanged. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Caution 3 potential secret finding(s) were committed in this pull request. Treat each credential as compromised and revoke or rotate it now, even if a later commit removed it. Deleting the line or force-pushing does not undo the leak: the commits stay reachable from this pull request and from every clone or fork, and automated scanners harvest secrets pushed to public repositories within minutes. If a finding is a test fixture or a false positive, allowlist it in the configuration of the scanner that reported it. Security scan found 3 critical or high finding(s) on lines changed by this pull request
View the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |


Closes ROCMAI-879.
Summary
Three places each read the last lines of a log their own way: the ComfyUI log view (whole file, UTF-16 decode, ANSI strip, lossy text), the
rocm services logs/rocm logstext inservices_text.rs(a streamed line reader that failed on any invalid text), and the rocmd MCPservice_logstool (whole file into one string, also failing on invalid text). Meanwhile the seek-bounded routine inrocm-coreserved only the lemonade and vLLM adapters. The three copies disagreed on what a log is, so a stray byte that is not valid text maderocm logsshow a single "failed to read" placeholder instead of the readable lines around it, and a log whose newest line is longer than the read budget was reported as empty or as having no output.This PR reads every one of those tails through one bounded, lossy routine in
rocm-core(tail_lines_decoded/tail_log_decoded, taking aLogDecode:PLAINfor UTF-8,CONSOLEfor ComfyUI's UTF-16 plus ANSI stripping). The private readers are deleted; each caller keeps a thin adapter that picks aLogDecode, passes the sharedLOG_VIEW_MAX_TAIL_READbudget (4 MiB) and adds its own error label.rocmdcannot depend onrocm, which is why the routine lives inrocm-core.Intended user-visible differences:
rocm services logs,rocm logsand theservice_logstool show the readable text with U+FFFD for the bad bytes instead of a placeholder line or an error. A UTF-16 service log is now shown as lossy text rather than as an error.rocm services logsprintsRecent output: none shown; the newest line is longer than the 4 MiB a log view reads, androcm comfyui logssays the same instead ofThe saved log is empty.. A genuinely empty log still saysno output yet/empty.service_logstool (the daemon's MCP server and the CLI's own tool bridge behindrocm mcp-call) now gives three distinct answers through one shared sentence,rocm_core::service_log_tool_summary: no log output yet, log output exists but none is shown because the newest line exceeds the read bound, or how many lines were read. Its structured reply carriesnewest_line_exceeds_read_bound. The line count is now the number of lines returned, not the number asked for: in the summary and in the structuredlinesfield (asking for 50 lines of a three-line log gives"lines": 3), with the number asked for, after clamping to 1 to 500, in a newlines_requestedfield. Nothing in the tree reads the field. A missing log file is still an error.failed to read <label> <path>with the open step in the cause (services text previously saidfailed to open <label> <path>).Split from #929 at the reviewer's request; the other parts are #960 (stop routine), #959 (engine IPC client) and the respawn guard, which follows once those two have merged.
Commits, in reading order
service_logstool's empty-versus-over-long answer and scenarios logs-13 and logs-14.docs/testing.mdparagraph for the scenarios.services_text.rsis renamedFollowCursor, so it no longer shares a name withrocm_core::LogTail; the architecture entry says which is which. A rename only.linesfield of theservice_logstool reports the lines returned, withlines_requestedbeside it, in both copies of the tool, with unit tests and scenario logs-16.docs/manual-testing.mddescribes the new messages and the tool's fields.Test plan
Ran locally (debug build, one target dir, targeted rather than the full workspace gate):
cargo test -p rocm-core --lib log_tail: 40 passed (including the shrinking-source tests and the UTF-16 detection boundary test); the wholerocm-corecrate passed at 1273 tests in the previous round, before that boundary test was added.cargo test -p rocmd: 204 passed, 2 ignored (includesservice_logs_tell_an_empty_log_from_one_whose_newest_line_exceeds_the_read_boundandservice_logs_report_the_lines_returned_and_the_lines_asked_for_apart).cargo test -p rocm --bin rocmwhole suite with--no-fail-fast: 1712 passed, 9 ignored, 0 failed (also the filteredservices_text,comfyui::andchat::groups).rocm/rocmdbuilt with--features rocm/e2e-test-hooksand-n 'logs-': 16 scenarios, 47 steps, all passed;feature_namingandcargo test -p xtask prepushpass.cargo fmt --all --check,cargo clippy --locked -p rocm-core -p rocmd -p rocm --all-targets -- -D warnings,cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings,cargo xtask manifest --check,cargo xtask check-crate-edges,cargo xtask check-architecture-doc,cargo xtask lychee: all clean.origin/main..HEADread through: the only hits are identifiers such asrun_internal_mcp_callalready in the tree.read_tail_lines_decodes_invalid_eight_bit_text_lossilyand the matchingrocm-coretests red; makingnewest_line_exceeds_the_read_boundignore the skipped-byte count turnsan_empty_tail_says_whether_the_log_is_empty_or_its_newest_line_is_too_long,the_service_log_summary_names_each_of_the_three_states_apart,service_logs_say_no_output_only_for_an_empty_log_and_name_the_bound_otherwiseand the rocmdservice_logstest red. Both mutations were reverted. Removing the shrink handling (the probe's early-end case, or the two short-window checks) turns three of the shrinking-source tests red each; the same tests fail withfailed to fill whole bufferor an empty tail before the fix. Reporting the requested count inlinesturns the unit test of each copy of theservice_logstool and scenario logs-16 red, and changing the UTF-16 detection threshold turns the boundary test red.Not run: the full workspace test gate,
scripts/smoke_local.py,cargo xtask e2ein release mode, the coverage floors, and a Windows-target clippy (no platform-gated code is touched). CI is the authoritative run for those.Scenarios
New scenarios in
tests/e2e-cucumber/features/logs.feature, all on the mock lane that runs on every PR (they read planted log files only, no GPU, no network): logs-08 (rocm logs --searchover a log with an invalid byte), logs-09 and logs-10 (rocm services logsfor an over-long line and an empty log), logs-11 and logs-12 (rocm comfyui logsfor the same pair), logs-13 and logs-14 (rocm mcp-call service_logsfor the same pair). logs-15 (rocm logsover a command log whose only line exceeds the bound, which prints the omitted-output line instead ofNo logs found yet.) and logs-16 (rocm mcp-call service_logsfor 50 lines of a three-line log:linesis 3 andlines_requestedis 50). The daemon's copy of theservice_logstool is covered by a rocmd unit test, because the harness drives only therocmbinary. Existing scenarios logs-01 to logs-07 are unchanged.