H4 and H5 for hideOS: hardware watchdog, user managers, a shutdown that cannot hang

.github/workflows/ci.yml

@@ -75,7 +75,7 @@ jobs:
         run: python3 .github/scripts/check-release-notes.py
 
       - name: Host tests
-        run: cargo test -p oxinit-unit -p oxinit-graph -p oxinit-service -p oxinit-cgroup -p oxinit-user -p oxinit-ipc -p oxinit-log -p oxinit-timer
+        run: cargo test -p oxinit-unit -p oxinit-graph -p oxinit-service -p oxinit-cgroup -p oxinit-user -p oxinit-ipc -p oxinit-log -p oxinit-timer -p oxinit-paths -p oxinit-watchdog
 
   msrv:
     name: builds on the declared MSRV

CLAUDE.md

@@ -19,6 +19,8 @@ crates/oxinit-graph/   dependency resolution
 crates/oxinit-service/ service state machine and restart policy
 crates/oxinit-cgroup/  cgroup v2
 crates/oxinit-ipc/     control protocol types
+crates/oxinit-paths/   where the sockets and logs are, system and user
+crates/oxinit-watchdog/ hardware watchdog configuration and policy
 crates/xtask/          build and boot automation
 docs/                  specifications
 units/                 the test image's unit files
@@ -64,6 +66,11 @@ only exists for as long as its PID 1 does. Nowhere else, and never on a
 failure. If the loop cannot continue — in a container too — spawn `/bin/sh` on
 `/dev/console` and keep reaping.
 
+A user manager (`oxinit --user`, `user.rs`) is not PID 1, and is the one
+place this bends: it exits at the end of its shutdown, in the same
+`shutdown::exit`, and when its loop cannot continue, in `user::give_up`. Both
+are reachable only with `--user`. Do not add a third.
+
 **No async.** No `tokio`, no `async-std`, no `futures`, no `async fn`. One
 thread, one `epoll` loop. This is not negotiable for `crates/oxinit`; other
 crates may do what they like.

CONTRIBUTING.md

@@ -22,7 +22,8 @@ rustup target add aarch64-unknown-linux-musl
 MSRV is stable minus two releases. No nightly features are used.
 
 **Tools.** `qemu-system-x86_64`, `qemu-system-aarch64` and `cpio`. Docker or
-podman as well, for `cargo xtask container`, `test-distro` and `test-demo`.
+podman as well, for `cargo xtask container`, `test-distro`, `test-demo` and
+`test-watchdog`.
 
 ```bash
 # Debian/Ubuntu
@@ -123,6 +124,9 @@ cargo xtask test-boot --arch all            # M9: x86_64 and aarch64, in turn
 cargo xtask container                       # M6: runs it in Docker, same
 cargo xtask test-demo                       # M16: every command the README gives
 cargo xtask test-distro                     # M10: a real Alpine userspace
+cargo test -p oxinit-watchdog               # M17: when to feed, when to starve
+cargo xtask test-watchdog                   # M17: QEMU's i6300esb, fed and starved
+cargo test -p oxinit-paths                  # M18: where a user manager's things are
 ```
 
 The library crates have no Linux dependencies and run anywhere. Parser, graph,

Cargo.toml

@@ -9,10 +9,12 @@ members = [
     "crates/oxinit-graph",
     "crates/oxinit-ipc",
     "crates/oxinit-log",
+    "crates/oxinit-paths",
     "crates/oxinit-service",
     "crates/oxinit-timer",
     "crates/oxinit-unit",
     "crates/oxinit-user",
+    "crates/oxinit-watchdog",
     "crates/oxlogd",
     "crates/xtask",
 ]

README.md

@@ -271,6 +271,11 @@ immediate and says what was *asked for*, not what has finished: a stop ends
 when the unit's cgroup empties, and the one socket that has to stay responsive
 does not wait on the slowest thing on the machine.
 
+`oxctl --user` asks the same things of your own user manager — `oxinit
+--user`, the same supervisor run by a user for their own services, with units
+in `~/.config/oxinit/user-units` and sockets under `$XDG_RUNTIME_DIR`. Whatever
+starts your session starts it; oxinit does not manage logins.
+
 ## Logs
 
 A service declaring `output = "log"` writes into a pipe rather than onto the
@@ -325,7 +330,7 @@ getting it wrong strands the machine.
 
 ## Current state
 
-Milestones 0 through 14 are done, and the roadmap is closed. oxinit boots under QEMU and runs as a
+Milestones 0 through 18 are done, and the roadmap is closed. oxinit boots under QEMU and runs as a
 container's PID 1.
 
 | | |
@@ -347,12 +352,14 @@ container's PID 1.
 | **M14** | Calendar schedules on the wall clock; datagram sockets; the list closed. |
 | **M15** | The documents' invariants enforced by the build, not by discipline. |
 | **M16** | A demo, a released binary, and a README that starts at the beginning. |
+| **M17** | A hardware watchdog fed from the loop, with a boot deadline. |
+| **M18** | User managers: `oxinit --user`, `oxctl --user`, `oxlogd --user`. |
 
 One `epoll` loop multiplexes the signalfd, the timerfd, the notify socket, the
 control socket, every socket unit's listening descriptor and every service
 cgroup's `cgroup.events`. One thread. No async runtime.
 
-Nothing is scheduled after M16. [ROADMAP.md](ROADMAP.md) has the breakdown —
+Nothing is scheduled after M18. [ROADMAP.md](ROADMAP.md) has the breakdown —
 what each milestone was verified against, what was deferred out of it, and
 which remaining ideas are deliberately not being taken.
 

ROADMAP.md

@@ -1025,6 +1025,108 @@ Third time in this project that the failure was *editing text without looking
 at the output*, and the second time it reached somebody. It is worth naming as
 a pattern rather than as three separate slips.
 
+## M17 — A hardware watchdog
+
+**Done.**
+
+Asked for by hideOS, whose failure table has a row reading "PID 1 hangs —
+hardware watchdog reboots", and nothing behind it once the boot was over.
+Nothing on a machine recovers a hung PID 1, because nothing is above it; a
+timer in the chipset is.
+
+- [x] `/usr/lib/oxinit/watchdog.toml`, replaced by `/etc/oxinit/watchdog.toml`,
+      overridden key by key by `oxinit.watchdog.*` on the kernel command line.
+- [x] Fed from the loop body, not from a handler or a deadline alarm, so what
+      keeps the machine up is the loop coming round.
+- [x] Half the timeout the driver settled on; a device that is not there yet
+      looked for again.
+- [x] A boot deadline: `boot-unit` has to come up within `boot-sec`, or the
+      watchdog stops being fed and the hardware resets the machine.
+- [x] Disarmed with the magic close before a power off or a halt, left
+      running into a reboot. Never opened in a container.
+
+**Fed from the loop, and that is the design.** The tempting version is an
+alarm on the deadline heap, next to restart backoff and start timeouts. It
+would keep the machine alive through exactly the failures the watchdog is
+for: a heap that still fires while a handler elsewhere is wedged, or a panic
+that drops the alarm and resets a machine that had survived it. The write is
+in the loop body instead, after every handler, and `epoll_wait` gets a
+timeout for it — the only thing in oxinit that wakes without a descriptor.
+
+**The boot deadline is what makes it worth having on a machine that updates
+itself.** A boot manager that counts attempts — hideOS's does — can only
+count a hang if something resets it. Before this, hideOS armed the watchdog
+in its initramfs and stopped it from a unit at the end of boot; a hang
+anywhere before that unit was a reset, and a hang after it was a machine
+that sat there. Now one mechanism covers both: fed until the boot unit comes
+up or the deadline passes, and fed for ever after that.
+
+**The driver is a module, which the test had to face.** The netboot kernel
+`fetch` downloads has every watchdog driver as a module, and nothing in a
+hand-built initramfs can load one it does not carry. `test-watchdog` takes
+the kernel and its i6300esb module from the same `linux-virt` package, so
+they always match, and loads the module from a unit — which is how the
+device turning up after oxinit has started got a test rather than an
+assumption.
+
+Verified: `cargo xtask test-watchdog`. Fed for fourteen seconds on a four
+second timeout, then halted, and still running twelve seconds after the halt
+— under `-no-reboot`, where a reset would have ended QEMU. Then a boot unit
+that fails, a deadline cut from five minutes to eight seconds on the command
+line, and QEMU ending without a line of shutdown. 23 host tests in
+`oxinit-watchdog`, four more for `reached` in `oxinit-service`.
+
+## M18 — User managers
+
+**Done,** for what a service manager should do. What a session manager would
+do is listed below, open.
+
+Also asked for by hideOS, whose desktop starts PipeWire and WirePlumber from
+XDG autostart because oxinit had nowhere else to put a service that belongs
+to a user.
+
+- [x] `oxinit --user`: the same supervisor, run by a user, with its sockets
+      under `$XDG_RUNTIME_DIR/oxinit` and its units in
+      `/usr/lib/oxinit/user-units`, `/etc/oxinit/user-units` and
+      `~/.config/oxinit/user-units`.
+- [x] `user` and `tty` refused in a user unit; `%u` is the user.
+- [x] One per user: a second one finds the first answering and exits 0.
+- [x] A child subreaper, its own cgroup subtree when one is delegated, and
+      none — PID 1's degraded mode — when not.
+- [x] Every stop signal, and `SIGHUP`, stops its units and exits.
+- [x] `oxctl --user` and `oxlogd --user`.
+- [x] `oxinit-paths`: one definition of where every socket and log is, for
+      both scopes, which `oxinit-ipc` and `oxinit-log` now take theirs from.
+
+**The same binary, not a second program.** A user unit has to mean what the
+same file means to PID 1, and the way to guarantee that is to run it through
+the same code. What differs is a table in ARCHITECTURE, and every row in it
+is something the user manager *skips* — mounts, console, hostname, identity
+changes, `reboot(2)`, the watchdog — plus the one thing it does differently
+when its loop breaks: it exits, because it is not PID 1. That is the second
+`exit()` in the crate, and CLAUDE.md and ARCHITECTURE now say so.
+
+**Starting it is not oxinit's job.** "Manage logins or sessions" is a
+non-goal, and this does not change it: whatever starts a session starts the
+user manager, with the session's environment, and signals it at the end. The
+test image does it with a system unit running as `nobody`, which is also the
+answer for services that should outlive a login.
+
+Open, each a decision rather than missing code — see ARCHITECTURE:
+
+- Several sessions of one user share one manager, and the first to end stops
+  it. Counting sessions is a login manager's knowledge.
+- No cgroup delegation from PID 1, which would need a unit key.
+- No way to give running services a variable the session set after the
+  manager started.
+
+Verified: `cargo xtask test-boot`, 46 checks — a user manager for `nobody`
+started from a system unit, finding its cgroup not delegated and saying so;
+a user unit printing `oxinit-user: nobody is nobody`, which is `%u` and
+`id -un` agreeing; a `notify` user unit going ready on the user's own socket;
+and, at shutdown, the user manager stopping it and exiting before the machine
+syncs. The same units pass `test-distro` (48 checks) and `container` (10).
+
 ## Not doing, and why
 
 These were on the list. They are coming off it with a reason rather than

crates/oxctl/Cargo.toml

@@ -16,4 +16,5 @@ publish.workspace = true
 [dependencies]
 oxinit-ipc = { path = "../oxinit-ipc" }
 oxinit-log = { path = "../oxinit-log" }
+oxinit-paths = { path = "../oxinit-paths" }
 rustix = { workspace = true, features = ["net"] }