M14: the loose ends, and closing the roadmap

ARCHITECTURE.md

@@ -318,7 +318,9 @@ there is no framing layer, no length prefix, and no partial-message state to get
 wrong. `SOCK_STREAM` would require all three; `SOCK_DGRAM` would drop the
 connection semantics needed to reply.
 
-**Timerfd** — exactly one, paired with a `BinaryHeap<Deadline>`. Restart
+**Timerfd** — one per clock, each paired with a `BinaryHeap<Deadline>`. The
+monotonic one carries every relative deadline; the realtime one carries
+absolute calendar moments. See [Timers](#timers). Restart
 backoff, start timeouts, and watchdog intervals all become entries in the heap.
 The timerfd is armed for the earliest deadline. When it fires, every expired
 entry is popped and dispatched, then it is rearmed. One fd per timer would mean
@@ -827,11 +829,21 @@ not stop the schedule**: the timer is re-armed before the service is started,
 because a timer says *when*, and what to do about a failure is the service's
 `restart`.
 
-**Known limitation.** A calendar deadline is computed once, as a delay on the
-monotonic clock, so a clock step between arming and firing moves that firing.
-Every firing after it is recomputed against the corrected clock. The proper
-fix is a second timerfd on `CLOCK_REALTIME` with `TFD_TIMER_CANCEL_ON_SET`,
-which is a second clock in the heap and has not been earned yet.
+**Two clocks.** A backoff, a stop timeout and a start timeout are all "this
+long from now" and must not move when the clock is corrected: `CLOCK_MONOTONIC`,
+relative. A calendar schedule is "at 03:30" and must stay at 03:30 however the
+clock gets there: `CLOCK_REALTIME`, and **absolute**. Two timerfds, a heap
+each, both in the same epoll loop.
+
+Collapsing the second into the first — computing a delay once and arming a
+monotonic timer with it — is what made an NTP correction move a nightly job.
+The fix is to stop converting, not to recompute on a step: an absolute
+deadline needs no recomputation, because the kernel compares it against the
+same clock that changed. `TFD_TIMER_CANCEL_ON_SET` only makes the step
+visible; it is not what makes the deadline right.
+
+`Schedule` returns which clock it means — `Next::After(Duration)` or
+`Next::At(Unix)` — rather than a delay that has already lost the distinction.
 
 `oxinit-timer` holds the schedule types and the arithmetic, takes the current
 time as an argument, and makes no syscall — so "what does this do at 23:59:59"

README.md

@@ -109,7 +109,9 @@ on-calendar = "03:30"
 `on-boot` and `interval` cover the monotonic cases — first firing, then every
 firing after it, measured from the previous one. The calendar vocabulary is
 deliberately closed: `hourly`, `daily`, `weekly`, `HH:MM`, `HH:MM:SS`, in UTC.
-It is not a cron expression and will not become one. A firing that arrives
+It is not a cron expression and will not become one. Calendar deadlines are
+armed absolutely on a `CLOCK_REALTIME` timerfd, so "at 03:30" survives the
+clock being corrected under it. A firing that arrives
 while the service is still running is skipped rather than queued, and a failed
 run does not stop the schedule.
 
@@ -203,7 +205,7 @@ getting it wrong strands the machine.
 
 ## Current state
 
-Milestones 0 through 13 are done. oxinit boots under QEMU and runs as a
+Milestones 0 through 14 are done, and the roadmap is closed. oxinit boots under QEMU and runs as a
 container's PID 1.
 
 | | |
@@ -222,12 +224,14 @@ container's PID 1.
 | **M11** | `after`, `requires` and `conflicts` enforced; `start-sec` bounds activation. |
 | **M12** | CI: every suite above, on every push. |
 | **M13** | Coverage for seven behaviours the milestones claimed and nothing re-ran. |
+| **M14** | Calendar schedules on the wall clock; datagram sockets; the list closed. |
 
 One `epoll` loop multiplexes the signalfd, the timerfd, the notify socket, the
 control socket, every socket unit's listening descriptor and every service
 cgroup's `cgroup.events`. One thread. No async runtime.
 
-Nothing is scheduled after M13. [ROADMAP.md](ROADMAP.md) has the breakdown,
+Nothing is scheduled after M14, and [ROADMAP.md](ROADMAP.md) says which two
+remaining ideas are deliberately not being taken and why. [ROADMAP.md](ROADMAP.md) has the breakdown,
 including what each milestone was verified against and what was deferred out
 of it.
 

ROADMAP.md

@@ -792,21 +792,76 @@ watching it.
 Verified on both architectures, in the container, and against the real Alpine
 userspace: 38 checks per boot, 40 against the distribution image.
 
-Deferred out of M13: the backoff *reset* after a sustained `Active` period,
-which M2 also claims. Distinguishing "reset" from "never incremented" needs an
-assertion about a counter's history, and substring matching over a log cannot
-make one. It needs a different kind of test than this suite is.
-
-## Beyond M13
-
-Not planned, not designed, listed only so the questions have an answer:
-
-- Coverage for `type = "datagram"` and `type = "seqpacket"` sockets, which
-  needs a fixture that speaks them; `listen-probe` only does streams.
-- A `CLOCK_REALTIME` timerfd with `TFD_TIMER_CANCEL_ON_SET`, so a calendar
-  schedule survives the clock being stepped under it. Until then a wall-clock
-  firing is computed as a monotonic delay and an NTP correction moves it by
-  however much the clock moved.
-- Local time, which means a timezone database.
-- Replacing JSON with `postcard` on the control socket.
+Deferred out of M13: nothing. This entry originally said the backoff *reset*
+was uncovered and needed "a different kind of test than this suite is". That
+was wrong, and M14 found it: `oxinit-service` has had
+`backoff_resets_after_the_service_stays_up` and
+`a_brief_active_period_does_not_reset_backoff` all along, in exactly the right
+place. The reset is policy logic in a crate with no OS dependency, and a host
+test is what it should be tested by — the boot suite was never the thing
+missing.
+
+## M14 — The loose ends
+
+**Done.**
+
+Everything left on the list, and an honest account of the two things that are
+staying off it.
+
+- [x] Calendar schedules on a `CLOCK_REALTIME` timerfd, armed absolutely.
+- [x] `type = "datagram"` sockets, activated and asserted.
+- [x] The M13 note about the backoff reset, which was wrong.
+
+**Two clocks, because two kinds of deadline mean different things.** A backoff,
+a stop timeout and a start timeout are all "this long from now" and must not
+move when the clock is corrected — monotonic, relative. A calendar schedule is
+"at 03:30" and must stay at 03:30 however the clock gets there — realtime,
+*absolute*.
+
+M8 collapsed the second into the first: it computed "in fourteen hours" once
+and armed a monotonic timer with it, so an NTP correction moved a nightly job
+by however much the clock moved. The fix is not to recompute on a clock step —
+it is to stop converting. An absolute realtime deadline needs no
+recomputation, because the kernel compares it against the same clock that
+changed.
+
+`TFD_TIMER_CANCEL_ON_SET` is set as well and is *not* what makes that true. It
+only makes the step visible: the read fails with `ECANCELED` once, worth a line
+in the log and nothing else. Saying so matters, because the roadmap entry that
+asked for this named the flag as though the flag were the fix.
+
+`Schedule` now returns which clock it means — `Next::After(Duration)` or
+`Next::At(Unix)` — rather than a delay that has already lost the distinction.
+That is the whole of the type change, and it is what makes the routing
+impossible to get wrong.
+
+**A datagram socket is a different shape.** It is never `listen`ed on and never
+accepted, so oxinit starts the service on a *readable* descriptor rather than a
+pending connection, and the datagram that woke it is still queued for the
+service to read. `listen-probe` grew `--recv` and `--send` for it; the stream
+path could not have exercised any of that.
+
+`type = "seqpacket"` is still uncovered, and deliberately: it is `accept`ed
+like a stream, so it exercises the same oxinit path the `stream` units already
+do. A fixture for it would test the fixture.
+
+Verified: 41 checks per architecture, 43 against the distribution image, 148
+host tests.
+
+## Not doing, and why
+
+These were on the list. They are coming off it with a reason rather than
+sitting there forever.
+
+**`postcard` instead of JSON on the control socket.** ARCHITECTURE sets the
+precondition itself — "the likely replacement *once the message set stops
+changing*" — and it has not. `UnitStatus` gained a field in M8 and another in
+M14. JSON's stated benefit, that a message is readable with `socat` while the
+protocol is still moving, is worth more than bytes on a socket carrying a few
+messages a minute. The crate boundary is placed so this stays a one-file change
+whenever it is worth making.
+
+**Local time.** It means carrying a timezone database, which contradicts a
+position the project has already taken in writing, twice. Calendar schedules
+are UTC and say so. Reopening that is a decision, not a loose end.
 - A test suite that boots a real distribution userspace rather than a shell.

crates/listen-probe/src/main.rs

@@ -7,18 +7,26 @@
 //! Usage:
 //!
 //! ```text
-//! listen-probe --serve NAME     # the activated service
-//! listen-probe --connect PATH   # the client that triggers it
+//! listen-probe --serve NAME     # the activated service, over a stream
+//! listen-probe --recv NAME      # the activated service, over a datagram
+//! listen-probe --connect PATH   # the client that triggers a stream
+//! listen-probe --send PATH      # the client that triggers a datagram
 //! ```
 //!
+//! A datagram socket is never `listen`ed on and never accepts, so oxinit
+//! watches it and starts the service on a *readable* descriptor rather than on
+//! a pending connection — and the service reads the datagram that is already
+//! queued rather than accepting anything. Different enough to be worth its own
+//! pair of modes.
+//!
 //! `--serve NAME` looks its descriptor up by name in `LISTEN_FDNAMES` rather
 //! than assuming fd 3, which is what the variable is for: a service given
 //! descriptors by more than one socket unit has to be able to tell them apart.
 
 use std::io::{Read as _, Write as _};
 use std::net::Shutdown;
 use std::os::fd::FromRawFd as _;
-use std::os::unix::net::{UnixListener, UnixStream};
+use std::os::unix::net::{UnixDatagram, UnixListener, UnixStream};
 use std::process::ExitCode;
 
 /// Where the protocol says the first inherited descriptor is.
@@ -27,10 +35,16 @@ const FIRST_LISTEN_FD: i32 = 3;
 fn main() -> ExitCode {
     let args: Vec<String> = std::env::args().collect();
 
-    let result = match (flag(&args, "--serve"), flag(&args, "--connect")) {
-        (Some(name), _) => serve(name),
-        (None, Some(path)) => connect(path),
-        (None, None) => Err("usage: listen-probe --serve NAME | --connect PATH".to_owned()),
+    let result = if let Some(name) = flag(&args, "--serve") {
+        serve(name)
+    } else if let Some(name) = flag(&args, "--recv") {
+        recv(name)
+    } else if let Some(path) = flag(&args, "--connect") {
+        connect(path)
+    } else if let Some(path) = flag(&args, "--send") {
+        send(path)
+    } else {
+        Err("usage: listen-probe --serve|--recv NAME | --connect|--send PATH".to_owned())
     };
 
     match result {
@@ -44,6 +58,37 @@ fn main() -> ExitCode {
 
 /// Serve exactly one connection on the descriptor named `name`.
 fn serve(name: &str) -> Result<(), String> {
+    let index = inherited(name)?;
+
+    // SAFETY: the descriptor was inherited from oxinit at this number, is not
+    // owned by anything else in this process, and the protocol says it is a
+    // listening socket. This is a test fixture, not PID 1.
+    let listener = unsafe { UnixListener::from_raw_fd(FIRST_LISTEN_FD + index) };
+
+    let (mut stream, _) = listener.accept().map_err(|e| format!("accept: {e}"))?;
+
+    let mut request = [0u8; 64];
+    let read = stream
+        .read(&mut request)
+        .map_err(|e| format!("read: {e}"))?;
+    let request = String::from_utf8_lossy(request.get(..read).unwrap_or_default()).into_owned();
+
+    println!("listen-probe: serving a connection, request {request:?}");
+
+    stream
+        .write_all(format!("echo: {request}").as_bytes())
+        .map_err(|e| format!("write: {e}"))?;
+
+    Ok(())
+}
+
+/// Find the descriptor named `name`, checking the protocol's one hard rule on
+/// the way.
+///
+/// Returns the offset from [`FIRST_LISTEN_FD`], not the descriptor: looking it
+/// up by name is the whole point of `LISTEN_FDNAMES`, and a service handed
+/// descriptors by more than one socket unit cannot assume fd 3.
+fn inherited(name: &str) -> Result<i32, String> {
     let count: usize = var("LISTEN_FDS")?
         .parse()
         .map_err(|_| "LISTEN_FDS is not a number".to_owned())?;
@@ -66,31 +111,51 @@ fn serve(name: &str) -> Result<(), String> {
         .position(|candidate| candidate == name)
         .ok_or_else(|| format!("no descriptor named `{name}` in `{names}`"))?;
 
+    let index = i32::try_from(index).map_err(|_| "too many descriptors".to_owned())?;
+
     println!(
         "listen-probe: {count} descriptor(s) as `{names}`, LISTEN_PID={announced} is mine, \
          serving `{name}` on fd {}",
-        FIRST_LISTEN_FD + index as i32
+        FIRST_LISTEN_FD + index
     );
 
-    // SAFETY: the descriptor was inherited from oxinit at this number, is not
-    // owned by anything else in this process, and the protocol says it is a
-    // listening socket. This is a test fixture, not PID 1.
-    let listener = unsafe { UnixListener::from_raw_fd(FIRST_LISTEN_FD + index as i32) };
-
-    let (mut stream, _) = listener.accept().map_err(|e| format!("accept: {e}"))?;
+    Ok(index)
+}
 
-    let mut request = [0u8; 64];
-    let read = stream
-        .read(&mut request)
-        .map_err(|e| format!("read: {e}"))?;
-    let request = String::from_utf8_lossy(request.get(..read).unwrap_or_default()).into_owned();
+/// The datagram half of `serve`.
+///
+/// The datagram that started this service is already in the socket's queue —
+/// oxinit does not read it, exactly as it does not accept a connection. There
+/// is nothing to accept and no peer to reply to unless the sender bound an
+/// address, so this reports what it got and stops.
+fn recv(name: &str) -> Result<(), String> {
+    let index = inherited(name)?;
+
+    // SAFETY: same as `serve` — the descriptor was inherited at this number,
+    // nothing else in this process owns it, and this is a test fixture rather
+    // than PID 1.
+    let socket = unsafe { UnixDatagram::from_raw_fd(FIRST_LISTEN_FD + index) };
+
+    let mut buf = [0u8; 64];
+    let read = socket.recv(&mut buf).map_err(|e| format!("recv: {e}"))?;
+    let payload = String::from_utf8_lossy(buf.get(..read).unwrap_or_default()).into_owned();
+
+    println!("listen-probe: datagram on `{name}`, got {payload:?}");
+    Ok(())
+}
 
-    println!("listen-probe: serving a connection, request {request:?}");
+/// Send one datagram, which is what makes oxinit start the service.
+fn send(path: &str) -> Result<(), String> {
+    let socket = UnixDatagram::unbound().map_err(|e| format!("socket: {e}"))?;
 
-    stream
-        .write_all(format!("echo: {request}").as_bytes())
-        .map_err(|e| format!("write: {e}"))?;
+    socket
+        .send_to(b"knock", path)
+        .map_err(|e| format!("send to {path}: {e}"))?;
 
+    // Nothing to wait for. A datagram socket has no connection to hold open
+    // and this sender bound no address to be answered on, so the send is the
+    // whole of the client's side.
+    println!("listen-probe: sent a datagram to {path}");
     Ok(())
 }