Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
488 changes: 425 additions & 63 deletions crates/rocm-core/src/automation.rs

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions crates/rocm-core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ pub use automation::{
AutomationTriggerEvent, BuiltinWatcherSpec, EVENT_LOG_ROTATE_BYTES,
EVENT_LOG_ROTATED_GENERATIONS, REVIEWED_PROPOSALS_RETAINED, WatcherRuntimeSnapshot,
append_audit_event, append_automation_event, append_automation_proposal, builtin_watcher,
builtin_watchers, find_automation_proposal, generate_proposal_id, load_automation_proposals,
load_recent_audit_events, load_recent_automation_events, load_recent_automation_proposals,
update_automation_proposal_status,
builtin_watchers, find_automation_proposal, generate_proposal_id, jsonl_lock_path,
load_automation_proposals, load_recent_audit_events, load_recent_automation_events,
load_recent_automation_proposals, update_automation_proposal_status,
};
pub use binary_paths::{daemon_binary_path, engine_binary_path, sibling_binary_path};
pub use codex_bridge::{CodexBridgeEngine, CodexBridgeGpuSnapshot, CodexBridgeSnapshot};
Expand Down
13 changes: 11 additions & 2 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -1500,9 +1500,18 @@ rotation policy; two further tests go through the public
`append_automation_event`/`append_audit_event` and
`load_recent_automation_events`/`load_recent_audit_events` wrappers instead,
at the real size cap, so the wiring between them and the rotation policy
itself stays covered. The user-visible side is `automations-07` through
`automations-09` in
itself stays covered. The repair of a torn tail, which normally replaces the
store and falls back to cutting the debris in place when that replacement
fails, has its own tests there: each step injected to fail, the error naming
both causes when both fail, and a real fault on each platform (a directory
that takes no new files on Unix, a handle without delete sharing on Windows).
The user-visible side is `automations-07` through `automations-09`, plus
`automations-10` for the fallback, in
[automations.feature](../tests/e2e-cucumber/features/automations.feature).
`automations-10` makes the audit directory read-only, which does not bind root,
so it carries `@requires-dir-permissions` and resolves to a skip for a user the
permission bits do not bind (the self-hosted WSL2 guest runs as root) rather
than pass without testing the fallback.

Natural-language plan handoff tests:

Expand Down
1 change: 1 addition & 0 deletions tests/e2e-cucumber/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ Scenarios carry stable-id and capability tags:
| `@requires-multi-gpu` | Premise is a host with **more than one** AMD GPU present. Resolves to **skip** wherever the probed device count is not known to exceed one: a single-GPU host such as Strix Halo, and any host whose count could not be read (non-Linux, or WSL, which exposes neither a KFD topology nor an amdgpu DRM card). `@requires-gpu` cannot express this — it asks only whether *a* device is usable — so scenarios carry both. The count comes from the capability probe's `amd_gpu_count`, which mirrors the product's KFD+DRM rule rather than counting `amd-smi list` — the gate must ask the same "how many devices are present" question `--gpu` validation is built on, and `amd-smi` is only a best-effort fallback that may be absent or disagree. |
| `@requires-bare-metal` | Premise is a host running the in-tree amdgpu driver, so it does not hold under WSL2. Resolves to **skip** there. `@requires-os:linux` cannot express this: WSL2 reports an `os_family` of `linux`. |
| `@requires-wsl` | The inverse: premise **is** a WSL2 host. Resolves to **skip** on native Linux, native Windows, and everything else. |
| `@requires-dir-permissions` | Premise is a directory that refuses new files once its write bits are cleared. Resolves to **skip** for a user those bits do not bind: root, which is what the self-hosted WSL2 guest runs the suite as. The capability probe's `dir_permissions_bind` clears the write bits on a directory under the temp root and tries to create a file in it. `@requires-os:linux` cannot express this: root on Linux is still `linux`, and the scenario would pass without testing anything. Always **skip** on Windows, which has no such bits. |
| `@requires-case-sensitive-fs` | Premise is two files whose names differ only in letter case. Resolves to **skip** where the scenarios' temp root folds case. The capability probe's `case_sensitive_fs` writes two such names in its own temp dir, which follows `$TMPDIR` like every scenario's isolated root, and checks they stayed two files. `@requires-os:linux` cannot express this: `$TMPDIR` on Linux, WSL2 included, can sit on a casefolded directory or a mounted Windows drive. |
| `@requires-engine:<vllm\|lemonade>` | Pins the serve engine. Resolves to skip where that engine can't start (e.g. vLLM on a lemonade-only Strix host). |
| `@requires-os:<linux\|windows>` | Premise is OS-specific; skip on other OSes. |
Expand Down
26 changes: 25 additions & 1 deletion tests/e2e-cucumber/features/automations.feature
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Feature: Automation watchers
# per-entry approve/reject controls are gated on "pending"). This feature
# pins the enable/disable/mode-confirmation slice, the proposal-history
# listing's unit formatting, the bare listing's tolerance of a torn
# audit-log tail, and how the audit log and proposal history are kept
# audit-log tail, how the next write repairs one, and how the audit log
# and proposal history are kept
# bounded; the watcher/daemon trigger behaviour itself is covered
# elsewhere. Config-only (no GPU, no network), so it runs on the mock lane
# every PR.
Expand Down Expand Up @@ -127,3 +128,26 @@ Feature: Automation watchers
When the user selects a permissions mode
Then each rotated generation holds what the previous one held
And the oldest previous generation is gone

# A write that was cut off leaves a torn last line, and the next append
# repairs it by writing a clean copy of the audit log beside it and renaming
# that over the original. When the copy cannot be made — a full disk, or on
# Windows another program holding the log open without delete sharing — the
# repair used to give up: the debris stayed, the CLI warned that it failed to
# write the audit event on every command that records one, and the event was
# lost. The repair now falls back to cutting the debris in place. A directory
# that takes no new files is the same failure without filling a disk, so this
# drives it through `rocm config set-permissions`, a command the CLI audits.
# The fault is a directory permission, which does not bind root (the
# self-hosted WSL2 guest runs the suite as root), so `@requires-dir-permissions`
# skips it there rather than let it pass without testing anything. The unit
# tests in `automation.rs` cover the other triggers (an injected failure of
# each step, and on Windows a handle without delete sharing).
@id:automations-torn-audit-tail-repaired-when-log-cannot-be-replaced @requires-os:linux @requires-dir-permissions
Scenario: automations-10 - A command still records its audit event when the torn audit log cannot be replaced
Given the audit event log ends in a torn, unterminated line
And the audit log's directory takes no new files
When the user selects a permissions mode
Then the CLI confirms the permissions mode
And the CLI prints no warning about the audit log
And the audit log keeps its earlier record, drops the torn line and records the permissions change
89 changes: 89 additions & 0 deletions tests/e2e-cucumber/src/capability.rs
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,10 @@ pub struct HostCapability {
/// the temp root the scenarios' isolated roots are created in. Gates
/// `@requires-case-sensitive-fs`; see [`probe_case_sensitive_fs`].
pub case_sensitive_fs: bool,
/// Whether a directory made read-only refuses new files to the user
/// running the suite. Gates `@requires-dir-permissions`; see
/// [`probe_dir_permissions_bind`].
pub dir_permissions_bind: bool,
}

impl HostCapability {
Expand Down Expand Up @@ -411,6 +415,7 @@ fn probe_host_capability() -> HostCapability {
let platform_slug =
derive_platform_slug(has_amd_gpu, gfx_target.as_deref(), &os_family, is_wsl);
let case_sensitive_fs = probe_case_sensitive_fs(root);
let dir_permissions_bind = probe_dir_permissions_bind(root);

HostCapability {
os_family,
Expand All @@ -422,9 +427,47 @@ fn probe_host_capability() -> HostCapability {
effective_serve_engine,
platform_slug,
case_sensitive_fs,
dir_permissions_bind,
}
}

/// Whether a directory with its write bits cleared refuses to create a file for
/// the user running the suite.
///
/// Probed rather than inferred from the OS: root is not bound by directory
/// permissions, and the self-hosted WSL2 guest runs the suite as root even
/// though it is `os_family` `linux`. A scenario whose fault is "this directory
/// takes no new files" would then pass without exercising anything, so it is
/// skipped where this answers `false`.
///
/// The probe's directory is created under `dir`, the temp root the scenarios'
/// isolated roots follow, so it answers for the filesystem they write to, and
/// its permissions are restored before it returns so the temp root can be
/// removed. Any I/O error answers `false`: the premise has not been shown to
/// hold. Always `false` where there are no Unix permission bits to clear.
#[cfg(unix)]
fn probe_dir_permissions_bind(dir: &std::path::Path) -> bool {
use std::os::unix::fs::PermissionsExt;

let probe = || -> std::io::Result<bool> {
let probe_dir = dir.join("permission-probe");
std::fs::create_dir_all(&probe_dir)?;
std::fs::set_permissions(&probe_dir, std::fs::Permissions::from_mode(0o555))?;
let created = std::fs::File::create(probe_dir.join("probe-file"));
std::fs::set_permissions(&probe_dir, std::fs::Permissions::from_mode(0o755))?;
Ok(created.is_err())
};
probe().unwrap_or(false)
}

#[cfg(not(unix))]
// Platform stub. The Unix arm does I/O and cannot be `const`, so making only
// this arm `const` would give the two platforms different signatures.
#[allow(clippy::missing_const_for_fn)]
fn probe_dir_permissions_bind(_dir: &std::path::Path) -> bool {
false
}

/// Whether `dir`'s filesystem keeps two names that differ only in letter case
/// apart.
///
Expand Down Expand Up @@ -836,6 +879,7 @@ mod tests {
effective_serve_engine: "lemonade".to_owned(),
platform_slug: "strix-halo".to_owned(),
case_sensitive_fs: false,
dir_permissions_bind: true,
};
assert!(strix.engine_available("lemonade"));
// vLLM adapter is "built-in" but cannot start on Windows / non-dcgpu.
Expand All @@ -851,6 +895,7 @@ mod tests {
effective_serve_engine: "vllm".to_owned(),
platform_slug: "mi300x".to_owned(),
case_sensitive_fs: true,
dir_permissions_bind: true,
};
assert!(mi300x.engine_available("vllm"));
assert!(mi300x.engine_available("lemonade"));
Expand Down Expand Up @@ -1168,4 +1213,48 @@ Local model engines
std::fs::write(&not_a_dir, "").expect("write file");
assert!(!probe_case_sensitive_fs(&not_a_dir));
}

/// A probe that cannot even make its directory has not shown the premise
/// holds, so the gate must skip rather than let the scenario fail.
#[test]
fn permission_probe_answers_false_when_it_cannot_write() {
let tmp = tempfile::TempDir::with_prefix("capability-").expect("temp dir");
let not_a_dir = tmp.path().join("plain-file");
std::fs::write(&not_a_dir, "").expect("write file");
assert!(!probe_dir_permissions_bind(&not_a_dir));
}

/// The probe's directory is read-only while it probes; it must be given
/// back, or the temp root it lives in could not be removed.
#[cfg(unix)]
#[test]
fn permission_probe_leaves_its_directory_removable() {
use std::os::unix::fs::PermissionsExt;

let tmp = tempfile::TempDir::with_prefix("capability-").expect("temp dir");
probe_dir_permissions_bind(tmp.path());
let mode = std::fs::metadata(tmp.path().join("permission-probe"))
.expect("probe dir")
.permissions()
.mode();
assert_eq!(mode & 0o200, 0o200, "owner write bit was not restored");
}

/// The answer the gated scenario relies on: where the suite runs as an
/// ordinary user, a read-only directory refuses new files. Skipped as root,
/// for whom the probe is rightly `false`.
#[cfg(unix)]
#[test]
fn permission_probe_agrees_with_a_direct_check() {
use std::os::unix::fs::PermissionsExt;

let tmp = tempfile::TempDir::with_prefix("capability-").expect("temp dir");
let direct = tmp.path().join("direct");
std::fs::create_dir(&direct).expect("create dir");
std::fs::set_permissions(&direct, std::fs::Permissions::from_mode(0o555))
.expect("make read-only");
let refused = std::fs::File::create(direct.join("file")).is_err();
std::fs::set_permissions(&direct, std::fs::Permissions::from_mode(0o755)).expect("restore");
assert_eq!(probe_dir_permissions_bind(tmp.path()), refused);
}
}
Loading
Loading