Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
f911a5c
test: pin what rocm and rocmd each do when a managed stop terminates …
jussielo-amd Nov 5, 2026
0cccfc5
refactor: stop managed services through one rocm-core routine shared …
jussielo-amd Nov 5, 2026
9a62ece
fix(services): clear the recorded processes after a confirmed rocm stop
jussielo-amd Nov 6, 2026
b867089
test(e2e): stop a real managed server and its child through rocm serv…
jussielo-amd Nov 6, 2026
fa60b69
test(e2e): write the service-stop wait in minutes for clippy
jussielo-amd Nov 6, 2026
de8ee04
fix(core): record a process creation time as the start identity on Wi…
jussielo-amd Nov 6, 2026
f01721f
fix(core): decide a stop's identity verdict in one place that a table…
jussielo-amd Nov 6, 2026
e1ded95
test(e2e): end the process a stop scenario's recorded process started…
jussielo-amd Nov 6, 2026
d82052d
test(e2e): number the recycled-PID stop scenario service-stop-08 so t…
jussielo-amd Nov 9, 2026
d965fc8
fix(core): keep a stop unconfirmed when a Windows process cannot be q…
jussielo-amd Nov 9, 2026
4ebcdf6
fix(core): keep an unverified stop's key through a liveness refresh, …
jussielo-amd Nov 9, 2026
ec3888e
fix(core): treat a taskkill that found no process as a stop when the …
jussielo-amd Nov 9, 2026
6747590
fix(services): verify a Windows launch child's token in the failed-la…
jussielo-amd Nov 9, 2026
441ecea
fix(core): report a root that exited under a failed tree kill as desc…
jussielo-amd Nov 9, 2026
ecb53f3
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 9, 2026
c5cc607
fix(core): confirm a root that exited alone only when another recorde…
jussielo-amd Nov 9, 2026
e979ea2
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 9, 2026
7118669
fix(core): refuse an unverifiable stop before the tree kill is skippe…
jussielo-amd Nov 9, 2026
ace27d5
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 9, 2026
79bc063
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 9, 2026
8ec8def
fix(rocm): re-apply the failed-launch cleanup refusal in the split la…
jussielo-amd Nov 9, 2026
438b353
docs: select only the scenarios that apply to Windows, and say what t…
jussielo-amd Nov 9, 2026
5f75aa1
fix(core): do not tree-kill a Windows PID that could not be held
jussielo-amd Nov 9, 2026
24f0993
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 9, 2026
0d664c9
fix(services): forget the recorded processes when a listing completes…
jussielo-amd Nov 10, 2026
86737e2
test(e2e): fail the Windows stop scenario when tasklist fails instead…
jussielo-amd Nov 10, 2026
44a4afd
fix(services): count a stop that a listing completed under it as the …
jussielo-amd Nov 10, 2026
ceeac3d
fix(services): clean up a failed launch's child through the shared te…
jussielo-amd Nov 10, 2026
4d24e0d
fix(services): do not complete a container engine's pending stop from…
jussielo-amd Nov 10, 2026
78d9c6b
docs(cli): shorten the services help text this change added, to stay …
jussielo-amd Nov 10, 2026
08c5f44
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 10, 2026
ef9b025
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 10, 2026
bd168e9
refactor(cli): move the clap value enums out of cli.rs into cli_args.rs
jussielo-amd Nov 10, 2026
2ca770d
fix(services): keep a container engine's unresolved stop protected, t…
jussielo-amd Nov 10, 2026
40fe3ae
Merge origin/main into split-managed-service-stop-routine
jussielo-amd Nov 10, 2026
751e6f5
Merge remote-tracking branch 'origin/main' into rf-929c-stop
jussielo-amd Nov 10, 2026
e04c9db
fix(services): keep the evidence of a stop's own marker when a listin…
jussielo-amd Nov 10, 2026
3fc1d5e
Merge remote-tracking branch 'origin/main' into rf-929c-stop
jussielo-amd Nov 10, 2026
b467281
test(rocmd): drive the listing's rewrite from the engine stop callbac…
jussielo-amd Nov 10, 2026
937e30b
fix(services): keep an older stop from claiming the empty record a re…
jussielo-amd Nov 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1347,7 +1347,7 @@ rocm services list [--all] [--json]
rocm services logs <service-id> [--follow]
rocm services stop <service-id> --yes
rocm services restart <service-id> --yes
rocm services remove <service-id> --yes
rocm services remove <service-id> --yes [--force]
rocm services prune [--older-than-hours <n> | --any-age] [--dry-run] [--yes]
```

Expand All @@ -1356,10 +1356,15 @@ gone. If it cannot confirm that, it prints `Local server stop not confirmed` on
stderr and exits with an error. It keeps the record, which still names the
processes, and any endpoint key, which the server may still be enforcing. It
names each process that, or a process under it, did not exit after it was
signaled; for a process it could not verify as the server's, it signals nothing
and names an operating-system command to check on it. It does not suggest
running the stop again: when only a process under the recorded one survived, a
second stop can report the server stopped while that process still runs.
signaled. On Windows it also names a process that did exit (or was already gone)
when taking the processes under it down did not complete, or found nothing and
no other recorded process is confirmed gone, because one of them may still be
running. For a process it could not verify as the server's (its recorded
start-time can no longer be read back, or, on Windows, `rocm` cannot query it at
all), it signals nothing and names an operating-system command to check on it.
It does not suggest running the stop again: when only a process under the
recorded one survived, a second stop can report the server stopped while that
process still runs.

`restart` starts a server again only once its stop has confirmed that every
recorded process is gone; if the stop cannot confirm that, it says so, starts
Expand All @@ -1368,15 +1373,25 @@ nothing, and leaves the stop request recorded.
After the stop confirms, `restart` takes the same launch lock `rocm serve` uses
and re-checks the port before it respawns: if another live managed service is
now recorded on that port, or (on the default `127.0.0.1` host only) the port is
bound by another process, it refuses and the service stays stopped (`rocm services list` shows it). The lock wait is
bounded (5 seconds), unlike `rocm serve`'s: if a launch still holds the lock,
bound by another process, it refuses and the service stays stopped
(`rocm services list` shows it). The lock wait is bounded (5 seconds), unlike
`rocm serve`'s: if a launch still holds the lock,
`restart` fails saying the server was stopped but not restarted and names
`rocm services restart <id> --yes` to run once that launch finishes.

`remove` deletes one record that is no longer running, together with its log,
its engine state file, and its endpoint key file; a running server is refused,
so stop it first. `prune` does the same in bulk, always leaves running servers
alone, and additionally clears leftover files whose record is already gone.
alone, and additionally clears leftover files whose record is already gone. A
server counts as running while any process it recorded may be running, which on
Windows includes one `rocm` cannot query (a live process the account may not
open): `remove` and `prune` refuse such a record until that process is gone.
`remove <service-id> --force --yes` is the way out for exactly that case: it
removes the record when a stop was requested, no recorded process is confirmed
running, and at least one could not be queried, says which one, and never
removes a record whose process `rocm` can see running. `prune` has no
`--force`, but when it leaves such a record alone it lists it with the exact
`remove --force --yes` command. `docs/manual-testing.md` walks through it.

Removal destroys both the log and the `restart` option for the records it
takes, so `prune` only considers records untouched for 24 hours. Age is
Expand Down
197 changes: 26 additions & 171 deletions apps/rocm/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,14 @@
//! the completion builder, and re-imports these types so every other module
//! keeps reaching them as `crate::Command`, `crate::Provider` and so on.

pub(crate) use crate::cli_args::{
DevicePolicyArg, FrameworkArg, InstallFormat, PermissionsModeArg, Provider, SandboxToolArg,
TelemetryModeArg, WatcherModeArg,
};
use crate::service_records::DEFAULT_SERVICE_PRUNE_MIN_AGE_HOURS;
use crate::{bootstrap, chat, comfyui, remote, storage};
use clap::{Parser, Subcommand, ValueEnum};
use rocm_core::{
DEFAULT_LOCAL_HOST, PERMISSIONS_MODE_ASK, PERMISSIONS_MODE_FULL_ACCESS, TELEMETRY_MODE_LOCAL,
TELEMETRY_MODE_OFF, WatcherMode,
};
use clap::{Parser, Subcommand};
use rocm_core::DEFAULT_LOCAL_HOST;
use std::path::PathBuf;

/// User-selectable serving engines, in the order shown in `--help` and shell
Expand Down Expand Up @@ -1157,9 +1158,10 @@ pub(crate) enum ServicesCommand {
/// confirmed gone. Otherwise it reports on stderr that the stop was not
/// confirmed and exits with an error, keeping the record, which still names
/// the processes, and any endpoint key, which the server may still be
/// enforcing. It names each process that did not exit, and for one it could
/// not verify it signals nothing and names an operating-system command to
/// check on it. It does not suggest running the stop again.
/// enforcing. It names each process that did not exit (on Windows, also one
/// whose child processes may still be running), and for one it could not
/// verify it signals nothing and names an operating-system command to check
/// on it. It does not suggest running the stop again.
Stop {
/// Service id from `rocm services list`.
service_id: String,
Expand All @@ -1181,21 +1183,32 @@ pub(crate) enum ServicesCommand {
},
/// Delete one local server record and its files.
///
/// Only works on a record that is not running: stop the server first. The
/// record's details file, log, engine state file, and endpoint key file are
/// all deleted, so its log can no longer be read and it can no longer be
/// Only works on a record that is not running: stop the server first (on
/// Windows a process `rocm` cannot query counts as running). The record's
/// details file, log, engine state file, and endpoint key file are all
/// deleted, so its log can no longer be read and it can no longer be
/// restarted.
Remove {
/// Service id from `rocm services list --all`.
service_id: String,
/// Required; remove refuses without it.
#[arg(long)]
yes: bool,
/// Remove a record whose recorded process `rocm` cannot query.
///
/// Only when a stop was requested, no recorded process is confirmed
/// running, and at least one could not be queried (on Windows, a live
/// process the account may not open). Deletes the record, log, engine
/// state and endpoint key, and says which process; if it is still
/// running it keeps running. Refused for a process confirmed running.
#[arg(long)]
force: bool,
},
/// Delete local server records that are no longer running.
///
/// Running servers are always left alone. Leftover files whose record is
/// already gone are cleaned up too.
/// Running servers are always left alone; a record held only by a process
/// `rocm` cannot query (Windows) is listed with the `remove --force` command.
/// Leftover files whose record is already gone are cleaned up too.
///
/// Waits for a managed launch already under way to publish its record
/// before reading the directory, and waits as long as that launch takes.
Expand Down Expand Up @@ -1333,164 +1346,6 @@ pub(crate) enum SetupCommand {
},
}

/// Which framework `rocm examine` should probe.
///
/// The CLI-facing mirror of [`rocm_core::FrameworkProbe`], which cannot derive
/// `ValueEnum` itself without pulling clap into `rocm-core`. Every variant of
/// that enum existed and was reachable in the library, but no caller ever passed
/// anything but the default, so the choice could not be made from the command
/// line at all.
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum FrameworkArg {
/// Try PyTorch first, then llama.cpp (the default).
Auto,
/// Probe PyTorch only.
Pytorch,
/// Probe llama.cpp only.
#[value(name = "llama-cpp", alias = "llamacpp")]
LlamaCpp,
/// Probe no framework at all - fastest, and enough for GPU and driver questions.
Skip,
}

impl From<FrameworkArg> for rocm_core::FrameworkProbe {
fn from(value: FrameworkArg) -> Self {
match value {
FrameworkArg::Auto => Self::Auto,
FrameworkArg::Pytorch => Self::PyTorch,
FrameworkArg::LlamaCpp => Self::LlamaCpp,
FrameworkArg::Skip => Self::Skip,
}
}
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum InstallFormat {
/// Python wheel packages (recommended; smaller, pip-installable).
Wheel,
/// Self-contained tarball archive of the ROCm SDK.
Tarball,
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum Provider {
/// A local OpenAI-compatible server started via `rocm serve` (no API key).
Local,
/// Anthropic Claude API (requires an API key).
Anthropic,
/// OpenAI API or OpenAI-compatible endpoint (requires an API key).
Openai,
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum WatcherModeArg {
/// Only report findings; never take action.
Observe,
/// Report findings and propose changes for you to approve.
Propose,
/// Apply changes automatically within a contained, reversible scope.
Contained,
}

impl From<WatcherModeArg> for WatcherMode {
fn from(value: WatcherModeArg) -> Self {
match value {
WatcherModeArg::Observe => Self::Observe,
WatcherModeArg::Propose => Self::Propose,
WatcherModeArg::Contained => Self::Contained,
}
}
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum TelemetryModeArg {
/// Collect GPU telemetry on this machine only; nothing leaves the device.
Local,
/// Disable telemetry collection entirely.
Off,
}

/// User-facing `rocm serve --device` policy.
///
/// Modeled as a clap `ValueEnum` so invalid input is rejected by clap's usage
/// validation (exit code 2, with the valid choices listed) rather than a generic
/// application error (exit code 1) — matching how every other enum-style CLI
/// argument behaves. The historical aliases (`auto`/`gpu` for `gpu_required`,
/// `cpu` for `cpu_only`) stay accepted for backward compatibility but are hidden
/// from the advertised list.
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
#[value(rename_all = "snake_case")]
pub(crate) enum DevicePolicyArg {
/// Require a ROCm GPU; fail if none is usable (the default).
#[value(alias = "auto", alias = "gpu")]
GpuRequired,
/// Accepted for compatibility; behaves identically to `gpu_required`
/// (rocm serve has no CPU fallback path).
GpuPreferred,
/// Accepted but always rejected: rocm serve has no CPU fallback path, so
/// this exits with an error. Hidden from `--help`, shell completions and the
/// invalid-value suggestion list so it is never advertised as usable, while
/// still parsing so the deliberate rejection message is preserved.
#[value(alias = "cpu", hide = true)]
CpuOnly,
}

impl DevicePolicyArg {
/// Canonical policy string understood by [`crate::gpu_policy::parse_device_policy`]. Routing the
/// parsed variant back through the string keeps `parse_device_policy` the
/// single choke point that enforces the `cpu_only` rejection across every
/// caller, rather than adding a second conversion path that could bypass it.
pub(crate) const fn as_policy_str(self) -> &'static str {
match self {
Self::GpuRequired => "gpu_required",
Self::GpuPreferred => "gpu_preferred",
Self::CpuOnly => "cpu_only",
}
}
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
#[value(rename_all = "snake_case")]
pub(crate) enum SandboxToolArg {
ListServers,
RestartServer,
StopServer,
}

impl SandboxToolArg {
pub(crate) const fn as_cli_value(self) -> &'static str {
match self {
Self::ListServers => "list_servers",
Self::RestartServer => "restart_server",
Self::StopServer => "stop_server",
}
}
}

impl TelemetryModeArg {
pub(crate) const fn as_str(self) -> &'static str {
match self {
Self::Local => TELEMETRY_MODE_LOCAL,
Self::Off => TELEMETRY_MODE_OFF,
}
}
}

#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum PermissionsModeArg {
FullAccess,
Ask,
}

impl PermissionsModeArg {
pub(crate) const fn as_str(self) -> &'static str {
match self {
Self::FullAccess => PERMISSIONS_MODE_FULL_ACCESS,
Self::Ask => PERMISSIONS_MODE_ASK,
}
}
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down
Loading
Loading