Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 13 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,17 @@ for Lemonade and vLLM.
| Platform | Prebuilt binary | Notes |
|---|---|---|
| Linux (x86_64) | Yes | Ubuntu 24.04 or newer; full support, including the live dashboard and both inference engines |
| Windows (x86_64) | Yes | CLI and Lemonade serving; live dashboard shows GPU/CPU/memory but not the serving-instance registry; no vLLM |
| Windows (x86_64) | Yes | CLI and Lemonade serving; live dashboard shows GPU/CPU/memory and running managed servers, without per-server throughput; no vLLM |
| WSL2 (x86_64) | Yes (Linux binary) | Ubuntu 24.04 or newer; full support, including the live dashboard |
| macOS | No | No official installer, release, CI, or QA coverage |
<!-- platform-support-table-end -->

For WSL2 setup and diagnosis, see [docs/wsl.md](docs/wsl.md).

Live dashboard telemetry works on Windows too, but shows only GPU/CPU/memory —
the serving-instance registry view needs the `rocmdashd` unix socket, so it's
Linux/WSL2 only (see [Interactive interfaces](#interactive-interfaces)). vLLM
Live dashboard telemetry works on Windows too: GPU/CPU/memory and the managed
servers `rocm services list` reports. Per-server throughput and Docker-discovered
instances need the `rocmdashd` unix socket, so they're Linux/WSL2 only (see
[Interactive interfaces](#interactive-interfaces)). vLLM
serving is Linux or WSL2 only (see [docs/vllm.md](docs/vllm.md)).

The minimum supported Linux release, native or under WSL2, is Ubuntu 24.04. On
Expand Down Expand Up @@ -247,9 +248,10 @@ five tabs (switch with `Tab`/`Shift+Tab` or number keys `1`–`5`):
| **Observe** | Live GPU utilization, instances, and benchmark telemetry |
| **Chat** | Assistant chat backed by a local server or configured provider |

Live mode works on Windows too, but shows only GPU/CPU/memory — the
serving-instance registry needs the Unix domain socket telemetry, so that view
is Linux/WSL2 only. Use `rocm dash --demo` for a synthetic session that runs
Live mode works on Windows too, showing GPU/CPU/memory and the managed servers
`rocm services list` reports. Per-server throughput and Docker-discovered
instances need the Unix domain socket telemetry, so they are Linux/WSL2 only.
Use `rocm dash --demo` for a synthetic session that runs
anywhere without a GPU or daemon.

On Linux and WSL, the embedded daemon also discovers loaded models from a
Expand Down Expand Up @@ -1603,9 +1605,10 @@ For `rocm dash`:
- `--demo` runs a deterministic synthetic session with no GPU or daemon needed.
It works on all platforms.
- `--replay <file>` replays a recorded NDJSON session.
- Live mode works on Windows too, but shows only GPU/CPU/memory — there's no
`rocmdashd` unix socket to connect to there, so the serving-instance
registry view stays empty (Linux and WSL2 get the full live view).
- Live mode works on Windows too, showing GPU/CPU/memory and the managed
servers `rocm services list` reports. There's no `rocmdashd` unix socket
there, so per-server throughput and Docker-discovered instances are missing
(Linux and WSL2 get the full live view).

### Bench

Expand Down
94 changes: 79 additions & 15 deletions apps/rocm/src/dash.rs
Original file line number Diff line number Diff line change
Expand Up @@ -370,8 +370,9 @@ pub fn resolved_args(
/// `rocm_dash_tui::client`'s `#[cfg(windows)]` arm does connect and stream
/// live GPU/CPU/memory every tick but — by design, there being no
/// `rocmdashd` unix socket to source one from (`maybe_spawn_embedded_daemon`
/// is unix-only) — never sends an instance snapshot, so
/// `AppState::instances` never gets a first one there.
/// is unix-only) — never sends an instance snapshot, so this probe is also
/// where `AppState::instances` comes from there: the dashboard polls it
/// (`ManagedRegistrySnapshot::instances`) once a second off the UI thread.
///
/// **Known divergence, same shape as [`services_past_attempts`]'s:** this
/// reads the record's raw on-disk `status`, not process liveness. A service
Expand All @@ -391,11 +392,31 @@ impl ManagedRegistryProbe for LiveManagedRegistryProbe {
let discovery = discover_managed_services(&records);
ManagedRegistrySnapshot {
has_live_service: discovery.svcs.iter().any(|svc| svc.status.is_serving()),
instances: managed_instances(&discovery),
managed_ids: discovery.seen,
}
}
}

/// The live managed services as dashboard instances with no telemetry yet —
/// the same mapping the daemon applies to a newly discovered service.
fn managed_instances(
discovery: &rocm_dash_daemon::registry::ManagedDiscovery,
) -> Vec<rocm_dash_core::metrics::Instance> {
discovery
.svcs
.iter()
.map(|svc| {
rocm_dash_core::traits::merge_instance(
svc,
&rocm_dash_core::traits::InstanceSample::default(),
0,
0,
)
})
.collect()
}

/// Managed-service records that are no longer running, read from the same
/// registry `rocm services` reads. A status-only read over already-loaded
/// records: no readiness probes, no daemon, no disk I/O of its own.
Expand Down Expand Up @@ -731,19 +752,7 @@ pub fn run_launcher(chat_mock: bool) -> Result<()> {
fn launcher_serving_instances(paths: &AppPaths) -> Vec<rocm_dash_core::metrics::Instance> {
use rocm_dash_daemon::registry::{discover_managed_services, load_service_records};
let records = load_service_records(&paths.services_dir());
discover_managed_services(&records)
.svcs
.into_iter()
.map(|svc| rocm_dash_core::metrics::Instance {
container_id: svc.container_id,
container_name: svc.container_name,
model_name: svc.model_name,
status: svc.status,
port: svc.port,
managed: svc.managed,
..Default::default()
})
.collect()
managed_instances(&discover_managed_services(&records))
}

/// Entry point for a focused launcher flow (Set up / Serve / Diagnose).
Expand Down Expand Up @@ -1384,6 +1393,61 @@ mod tests {
let _ = std::fs::remove_dir_all(&root);
}

/// Windows has no daemon, so the dashboard's instance list comes from the
/// probe's `instances` (see `AppState::push_host_gpu_snapshot`). A ready
/// Lemonade record must come through with its model and port; a stopped
/// record and one with no bound port must not.
#[test]
fn live_registry_probe_carries_the_live_managed_instances() {
let root = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join(".rocm-work")
.join("tests")
.join("dash")
.join(format!(
"probe-instances-{}-{}",
std::process::id(),
rocm_core::unix_time_millis()
));
let p = AppPaths {
config_dir: root.join("config"),
data_dir: root.join("data"),
cache_dir: root.join("cache"),
};
std::fs::create_dir_all(p.services_dir()).unwrap();
std::fs::write(
p.services_dir().join("ready.json"),
br#"{"service_id":"lemonade-qwen","engine":"lemonade","model_ref":"Qwen/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M","port":11436,"status":"ready","created_at_unix_ms":3}"#,
)
.unwrap();
std::fs::write(
p.services_dir().join("stopped.json"),
br#"{"service_id":"lemonade-old","engine":"lemonade","model_ref":"Qwen/Qwen2.5-1.5B-Instruct","port":11435,"status":"stopped","created_at_unix_ms":2}"#,
)
.unwrap();
std::fs::write(
p.services_dir().join("unbound.json"),
br#"{"service_id":"lemonade-unbound","engine":"lemonade","port":0,"status":"ready","created_at_unix_ms":1}"#,
)
.unwrap();

let args = resolved_args(&cfg(), &p, ActiveTab::Home);
let instances = probe_snapshot(&args).instances;
assert_eq!(
instances.len(),
1,
"only the live, bound record: {instances:?}"
);
let inst = &instances[0];
assert_eq!(inst.container_id, "lemonade-qwen");
assert_eq!(inst.model_name, "Qwen/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M");
assert_eq!(inst.port, Some(11436));
assert_eq!(inst.status, rocm_dash_core::metrics::InstanceStatus::Ready);

let _ = std::fs::remove_dir_all(&root);
}

/// Paired with `resolved_args_reports_no_live_managed_service_when_none_are_ready_or_running`:
/// `managed_ids` and `has_live_service` (`ManagedRegistrySnapshot`) are two
/// different questions over the same live read (liveness vs. reachability), so a record
Expand Down
Loading
Loading