From 183826d3a25592bb182732b87edf153384fbdf59 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:05:02 +0000 Subject: [PATCH 01/20] wip(docs): snapshot RFC-0005 and RFC-0006 drafts mid-review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Backup only — the review and revision workflows are still running. This commit will be replaced by the final, reviewed RFC commits before any PR is opened. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 454 +++++++++++++++++ docs/rfcs/0006-threads-and-scheduling.md | 482 ++++++++++++++++++ 2 files changed, 936 insertions(+) create mode 100644 docs/rfcs/0005-address-spaces-and-physical-memory.md create mode 100644 docs/rfcs/0006-threads-and-scheduling.md diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md new file mode 100644 index 0000000..8fea185 --- /dev/null +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -0,0 +1,454 @@ + + +# RFC-0005 — Address spaces and physical memory + +| Field | Value | +|-------|-------| +| Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | +| Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | +| Date | 2026-09-27 | +| Affects | Constitution §3 (kernel doctrine); RFC-0003 (one new right, three new objects; `capability/src/rights.rs`); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 (one proposed assumption) | +| Depends on | RFC-0003 (capability table), RFC-0004 (IPC); interlocks with RFC-0006 (threads and scheduling) and RFC-0007 (system calls and process bootstrap), proposed the same day | +| Discharges | O-7 (the memory and object half), O-8, O-13 (the mapping half); O-5 for every path by which the kernel touches user memory; O-3's destruction case extended to memory; answers RFC-0003 §14.3 and RFC-0004 §9.1 | + +> **Proposed verdicts.** +> +> 1. **Discovery:** RAM comes from firmware tables into one bounded `MemoryMap`; the image moves to 0x4020_0000, because QEMU gives an ELF at RAM base no DTB (§4). +> 2. **Frame allocator:** a bitmap over 4 KiB frames, colour-capable from day one; every frame zeroed before any process can see it (§4). +> 3. **Who pays (O-7):** typed fixed-capacity slot arrays are the object store; every frame and slot taken after boot is charged to a capability-named `Pool` (§5). +> 4. **Layout:** a higher-half kernel with the *same* layout on both architectures; 47-bit halves; a non-executable physmap; guarded kernel stacks (§6). +> 5. **Objects:** `Region` (eager, zeroed, at most 8 extents) and `AddressSpace` join RFC-0003's objects beside `Pool`; each mapping is a charged record (§7). +> 6. **Operations:** `map` / `unmap` / `protect` are mechanical, capability-checked, bounded and atomic; `protect` only narrows; the kernel never picks an address (§8). +> 7. **Rights:** `EXECUTE` joins RFC-0003's set as bit 5 — map-executable on a Region, mint-executable on a Pool, meaningless elsewhere (§8). +> 8. **W^X (O-13):** unrepresentable in the mapping type, enforced per Region across *all* address spaces, backed by `SCTLR_EL1.WXN` (§9). +> 9. **Sharing (RFC-0004 §9.1):** *share* is derive-and-transfer; *donate* is transfer made exclusive by `reissue`, a destruction-case operation (§10). +> 10. **User memory (O-5):** the kernel never dereferences a user virtual address, so no syscall takes a pointer argument, ever (§11). +> 11. **TLB:** ASIDs/PCIDs are kernel-private tags with generation rollover; maintenance is inner-shareable from the first line (§12). +> 12. **Early boot:** the stub enables the MMU with throw-away tables; Rust runs only at the high alias and installs the final W^X map (§13). + +## 1. The question + +**Where does memory come from, who pays for it, and how does a process reach exactly the memory it holds +capabilities for — never writable and executable at once, and without the kernel trusting an address a +process gave it?** Mechanism only: how much memory, which program may execute code, and where a heap goes +are userspace's. + +## 2. Which pillar + +**Pillar 2 and the doctrine that capabilities are the only authority, where authority becomes hardware.** A +`READ` capability that becomes a writable page is O-2 violated by the silicon. Pillar 1 lands here too: the +threat model files integrity *in memory* under A1, and W^X is how signed code stays what was signed. + +## 3. What the hardware gives — one shape, two encodings + +Both Tier-1 MMUs, as configured here, are one machine: **four levels of 512 eight-byte entries, 4 KiB pages, +nine index bits per level, and 47-bit user and kernel halves whose top level uses 256 entries each.** +AArch64 gets this from the 4 KiB granule with `TCR_EL1.T0SZ = T1SZ = 17` (a level-0 start); x86_64 from +IA-32e four-level paging without LA57. The differences are in the leaf encoding: + +| Property | AArch64 (VMSAv8-64, 4 KiB granule) | x86_64 (IA-32e, 4-level) | +|---|---|---| +| User / kernel split | two roots: `TTBR0_EL1` (low), `TTBR1_EL1` (high) | one root, `CR3`; PML4 entries 0–255 user, 256–511 kernel | +| Present | bits 1:0 = `0b11` (valid page) at level 3 | bit 0 (P) | +| Writable | `AP[2]` (bit 7) clear | bit 1 (R/W) set | +| User-accessible | `AP[1]` (bit 6) set | bit 2 (U/S) set | +| No-execute | `PXN` (bit 53) and `UXN` (bit 54), separately | `XD` (bit 63), only with `IA32_EFER.NXE` | +| Global | `nG` (bit 11) clear | `G` (bit 8) set, with `CR4.PGE` | +| Accessed | `AF` (bit 10), set by software on this CPU | bit 5 (A), set by hardware | +| Address-space tag | 16-bit ASID in `TTBR0_EL1[63:48]` | 12-bit PCID in `CR3[11:0]` | + +*(Bits checked against Linux's `arch/arm64/include/asm/pgtable-hwdef.h` — Arm's site was unreachable from +this session — and Intel SDM Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, +no hardware access-flag update and no PAN: `target/arm/tcg/cpu64.c` sets `ID_AA64MMFR0_EL1 = 0x1124` and +leaves `ID_AA64MMFR1_EL1` at 0.)* Hence **one table engine**, generic over a per-architecture `Descriptor` +encoder, host-tested on both against a fake frame store in `memory/`; only turning a frame number into +`&mut [u64; 512]` through the physmap (§6) is `unsafe`, in `kernel/src/mm/`. + +## 4. Physical memory: discovery and the frame allocator + +**Discovery.** QEMU documents two fixed facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every +other address "may vary across QEMU versions" and must be read from the DTB, which a bare-metal ELF finds +"at the start of RAM" with no register pointing at it (`docs/system/arm/virt.rst`). But `hw/arm/boot.c` +puts it there only if the image leaves room below itself, and today's image *is* the base of RAM. Checked +on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB anywhere in RAM** and `x0 = 0`; at +0x4020_0000 it finds one (magic `0xd00dfeed`) at 0x4000_0000. **So the image loads at 0x4020_0000**, 2 MiB +in and block-aligned; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT reader, written for the +purpose (the Borrow Ledger's boot-path row; Devicetree Specification v0.4), reads the cell sizes, `/memory`, +`/memreserve/` and `/reserved-memory`, bounds-checks every offset and fails closed. Threat model §10 names no +firmware assumption; this RFC proposes one — *firmware tables describe memory truthfully* — through +`docs/CHANGELOG.md`, as §10 already assumes the CPU. On x86_64 the UEFI stub (owned by no RFC yet) hands +over `GetMemoryMap`'s output. **Both produce one `MemoryMap`**: at most 64 sorted, disjoint `(PhysRange, +Kind)` entries — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that, +with image, boot stack, DTB and boot tables tagged before the allocator sees it. Nothing above `arch` names +FDT or UEFI. The DTB passes to the root task read-only (RFC-0007 (proposed) provides the handover). + +**The frame allocator is a bitmap**: one bit per 4 KiB frame, its storage taken from the first free frames +(16 KiB for 512 MiB). It allocates frames from a *colour set*, or runs of `n` at an alignment; a double +free fails closed. Colour is `pfn mod C`, `C = 1` until time protection is wanted *(Ge et al., EuroSys '19, +via research/0002 Part 6)*. **Every frame is zeroed before a process can observe it** — a recycled frame +is an O-8 breach that needs no bug. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing fights +colouring, until measurement asks; a **free list**, which cannot find `n` contiguous frames without a scan. + +## 5. Who pays: the object store and Pools (O-7) + +The kernel has no heap and must never acquire one a process can drain; every kernel object and page-table +frame is memory some process caused it to spend. **A — seL4 Untyped and retype** is the proven strong form, +but reusing an Untyped revokes everything retyped from it through the derivation tree RFC-0003 §4 refused; +it would decide RFC-0003a by the back door and make object layouts ABI. **Rejected.** **B — a kernel heap +with creation policy** *(Zircon)* limits *whether* a process creates objects, not *how much* memory they +take; exhaustion is global and unnamed. **Rejected.** **C — static per-type tables** *(Hubris)* are bounded +but cannot say *who* filled one — research/0002 Part 6's AWS Kinesis outage. **Rejected alone.** **D — C's +structure with counted attribution: verdict sought**, this paper's one deliberate merge *(the requester pays, +as in seL4; a counted quota, as in Fiasco.OC's factory limits and Genode's quota trading)*. + +**The object store is typed slot arrays.** Each object type has one fixed-capacity array in `.bss`; a slot +holds the object, a generation and a reference count, and **the slot index is the object's identity**. +`ObjectRef` (`capability/src/object.rs`) is `(type, index)` and `current_generation()` is a slot read — no +raw object pointer, slab or size class. A process's capability table lives in its slot +(`CapabilityTable` is fixed-capacity). A destroyed object frees its frames at once; its slot is reused +only when the count, stale capabilities' references included, reaches zero, and is retired, as RFC-0003 +requires, if its generation cannot advance. + +**A Pool is the accounting layer**: an RFC-0003 object holding balances — frames, and slots of each type. At +boot every free frame and slot becomes the balance of one root Pool handed to the root task (RFC-0007 +(proposed) provides the handover). Thereafter, for each resource `r` (frames, or one type's slots): + +```text +Σ balance(r) over all Pools == units of r free -- at every instant after boot +every unit of r in use == kept by the kernel at boot, or charged to exactly one Pool +``` + +- **Everything is charged**: Region frames; page-table frames, to the AddressSpace's Pool; one `Mapping` + slot per mapping; kernel stacks; every object slot. Each creation presents a Pool with `WRITE`. +- **Charge first, then take.** Conservation guarantees the allocator can then supply; the only later + failure, `TooFragmented` for a multi-frame extent, refunds before returning. +- **Failure is local and attributed.** `PoolExhausted` names the Pool and resource; no array can fill while + a Pool holds its slots, so no unnamed limit exists after boot. Nothing overcommits *(research/0002 Part 6, + "budgets are minima")*. Refund goes to the payer, whose index each object records — no tree. +- **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child with the parent's rights; + `pool_move(from, into, r, n)` moves `n` units of one resource; `pool_merge(from, into)` succeeds only + when `from` has no outstanding charges. Balance is conserved, never minted — O-2's shape for quantity. +- **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). + +This is not option B's heap: nothing grows after boot, and the only variable memory — frames — comes from +the bitmap in page units, charged. The kernel keeps its image, the map, the bitmap, the slot arrays, its +final tables and the boot stack. + +## 6. The kernel's virtual layout + +Both architectures get the **same layout**, the cheapest way to keep the HAL honest. The image sits in the +top 2 GiB because `x86_64-unknown-none` uses `code-model: kernel`, which requires it; AArch64 follows. + +| Range (both architectures) | Contents | Attributes | +|----------------------------|----------|------------| +| `0x0000_0000_0001_0000` – `0x0000_7FFF_FFFF_EFFF` | the current AddressSpace (§7) | per mapping; not-global; never kernel-executable | +| `0xFFFF_8000_0000_0000` – `+64 TiB` | physmap: every discovered RAM frame at `base + pa` | Normal write-back; RW; execute-never; global; largest blocks alignment allows | +| `0xFFFF_C000_0000_0000` – `+1 TiB` | MMIO window: kernel-owned devices only (console, interrupt controller) | Device-nGnRE / uncached; RW; execute-never | +| `0xFFFF_FFFF_0000_0000` – `0xFFFF_FFFF_7FFF_FFFF` | kernel stacks, an unmapped guard page below each | Normal; RW; execute-never; frames charged like any object | +| `0xFFFF_FFFF_8000_0000` – top | image at `+0x20_0000` | `.text` RX, `.rodata` R, `.data`/`.bss` RW execute-never | + +- **Kernel stacks have their own window** so each has a guard page, which the hole-free physmap cannot give. + RFC-0006 (proposed) decides how many — per core or per thread — and allocates here; the boot stack too. +- **The user floor is 64 KiB**, so a kernel NULL dereference never lands on user data *(Linux + `mmap_min_addr`)* — a fixed mechanism constant where Linux has a sysctl (§17). +- **The user ceiling leaves the last page below the hole unmapped**, closing Intel's `SYSRET` hazard + (CVE-2012-0217) for RFC-0007's x86_64 exit path by layout, as Linux does. The physmap covers RAM, never + MMIO, so no frame is aliased with conflicting attributes. +- **The kernel half never changes after boot.** AArch64 shares `TTBR1_EL1` for free; x86_64 copies PML4 + entries 256–511 into each address space, safe only because they are fixed *(seL4 x86)*. + +*Rejected:* an **identity-mapped kernel** (impossible under the kernel code model); **no physmap** (an +invalidation per IPC-buffer touch, for a side-channel gain threat model §7 puts out of scope). + +## 7. The objects: Region, AddressSpace and Mapping + +**Frame ownership is pKVM's state machine** (research/0002 Part 7): every frame has exactly one owner — the +kernel, or one `Region`. **A Region** is a fixed number of pages backed by **at most eight extents** of +contiguous frames, allocated eagerly, zeroed and charged; a request the bitmap cannot meet in eight fails +`TooFragmented`, and DMA-ready memory is one extent at an alignment. Rights are `READ`, `WRITE`, `EXECUTE`, +with RFC-0003's `DUPLICATE`, `TRANSFER`, `REVOKE`. Mappings and IPC-buffer bindings hold counted references, +so closing the last handle does not pull memory from under a live mapping *(Zircon: a mapping keeps its VMO +alive)*; the forceful paths are destruction and `reissue` (§10). + +**An AddressSpace** is a user root table, an ASID or PCID (§12), the Pool paying for its tables, and its +mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a process is RFC-0007's. +**A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, vaddr, pages, perms}`, +linked by slot index into its AddressSpace's list and its Region's. + +**Why a Region must remember its mappings.** RFC-0003's 2026-07-30 amendment makes an invariant +load-bearing: *a resolved capability is never observed or cached outside the kernel table* — why a +generation bump alone makes capabilities inert. **A page-table entry is exactly such a cache**, consulted by +hardware on every access and never re-checked. The MMU makes the exception unavoidable; the per-Region +mapping list is the compensation, and the reason a generation bump alone cannot revoke memory. + +**Device Regions** are minted at boot over `Mmio` ranges the kernel does not keep — never executable, zeroed +or charged; built in Phase 2. No operation maps a physical address given as a number: the catch-all right in +another costume. *Rejected:* **Zircon VMOs and VMARs** — lazy commit, copy-on-write, pagers and nesting are +policy-rich and put faults inside kernel paths; **seL4's capability per frame** — 256 handles per MiB; +**userspace-built page tables** (Xen PV) — research/0002 Part 7's bug farm; O-5 is a rights check, not a walk. + +## 8. Map, unmap, protect — and the rights they check + +Every operation takes handles (RFC-0003 §9); RFC-0007 (proposed) encodes them. + +| Operation | Needs | Refused when | +|-----------|-------|--------------| +| `region_create(pool, pages, extents, align)` | pool `WRITE`; pool `EXECUTE` to mint `EXECUTE` | `PoolExhausted`; `TooFragmented`; zero pages | +| `as_create(pool)` | pool `WRITE` | `PoolExhausted` (tags roll over, §12) | +| `map(as, region, first_page, vaddr, pages, perms)` | as `WRITE`; region rights ⊇ `perms` | `vaddr` unaligned, below the floor or outside the user half; overlap (never silent replacement); beyond the Region; `pages > 512`; `perms ∉ {R, RW, RX}`; W^X (§9); `PoolExhausted` | +| `unmap(as, vaddr)` | as `WRITE` | no mapping based at `vaddr` — the unit is the whole mapping | +| `protect(as, vaddr, perms)` | as `WRITE` | `perms` not a subset of the mapping's current `perms` | +| `reissue(region)` | region `REVOKE` (a proposed reading, §10) | stale handle | + +User numbers are validated *as numbers*. **The kernel never chooses a virtual address** *(seL4's VSpace +discipline)*: a kernel that picks addresses has an allocation policy in it. **`map` is atomic**: it counts +the table frames the range lacks and charges them, with the Mapping slot, *before* writing anything, so a +failed charge maps nothing; then it links tables and writes leaves. One call is bounded at 512 leaves and +six new tables — an unaligned range spans two leaf tables, each possibly with new parents at two levels. +Nothing splits a mapping. `protect` only narrows; widening is `map` again with the Region capability *(O-2's +shape; seL4's remap-with-capability)*. 4 KiB pages only; blocks later, invisible in the ABI. + +**Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as RFC-0003 §5 allows "when a subsystem +needs one". Folding execution into `READ` makes every readable Region potential code; a Region *kind* fails +the loader, which writes code before running it — a right in disguise. seL4 leaves execution an ungoverned +attribute; Fuchsia's `ZX_RIGHT_EXECUTE` *(Zircon)* is the lineage. The bit has two meanings, deliberately — +*may map executable* on a Region, *may mint executable Regions* on a Pool — one narrow authority over code, +refused on other objects. **`Rights::ALL` does not extend automatically**: `rights.rs` writes it as an +explicit union of five constants, and its doc comment claiming otherwise is misleading. The same PR edits +`ALL` and the comment, extends `Debug` and `rights_from_low_bits`, and widens the exhaustive test to `0..64`. + +The handshake RFC-0003 §14.3 asked for is one rule: **the installed permission is the requested `perms`, +refused unless a subset of the Region capability's rights at map time.** + +| perms | needs on the Region | AArch64 stage 1 | x86_64 | +|-------|---------------------|-----------------|--------| +| R | `READ` | `AP[2:1] = 0b11`, `UXN = 1`, `PXN = 1` | `P`, `U/S`, `R/W = 0`, `XD = 1` | +| RW | `READ`, `WRITE` | `AP[2:1] = 0b01`, `UXN = 1`, `PXN = 1` | `P`, `U/S`, `R/W = 1`, `XD = 1` | +| RX | `READ`, `EXECUTE` | `AP[2:1] = 0b11`, `UXN = 0`, `PXN = 1` | `P`, `U/S`, `R/W = 0`, `XD = 0` | + +Every user entry also sets `AF = 1`, `nG = 1`, inner-shareable, `AttrIndx` → Normal write-back. Write-only +is refused; execute-only exists on AArch64 but not on x86_64 without protection keys, so neither offers it. + +## 9. W^X (O-13), in four layers + +O-13's intent — a data-only bug cannot become code execution — needs more than its letter: + +1. **Unrepresentable.** `MapPerms` is an enum of `Read`, `ReadWrite`, `ReadExecute`; both encoders are total + functions from it, so no writable-executable entry can be built. Host-tested exhaustively. +2. **Per Region, across every address space.** A Region counts its writable uses (an IPC-buffer binding is + one) and its executable mappings; `map` refuses `RX` while any writable use exists anywhere, and `RW` + while any `RX` mapping does. Per-page W^X alone leaves frames writable here and executable there. +3. **Over time.** `EXECUTE` is minted only from a Pool carrying `EXECUTE`: the broker hands apps Pools + without it, the loader holds one, a JIT gets one by explicit grant *(Fuchsia moved executable memory from + ambient job policy to the `VmexResource` capability)*. O-13's Phase-3 load-path clause is policy on this. +4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); the + architecture already makes every EL0-writable page privileged-execute-never, and user entries set `PXN` + besides. x86_64 has no `WXN`; it sets `CR0.WP` and `SMEP` where present, and layers 1–3 are the defence. + +`map` with `RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` +on a PIPT instruction cache and `IC IALLUIS` otherwise (on VIPT-aliasing caches the physmap alias may miss +the user one — Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`. QEMU's cortex-a72 reports PIPT (`CTR_EL0 = +0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so errors here pass every test.** The +physmap aliases user code as writable at EL1; writing through it only to zero frames, fill tables and reach +IPC buffers is a convention (§17). + +## 10. Sharing and donation (RFC-0004 §9.1) + +pKVM's *share* (the owner keeps access) and *donate* (the donor loses it) — research/0002 Part 7 — need one op: + +- **Share** is `derive` then transfer over IPC: the borrower receives, say, `READ | WRITE` without + `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data then travels as RFC-0004 §5's + descriptor. No `share` syscall, and no page-table work during IPC — the long-IPC grave stays shut. +- **Donate** is transferring a capability carrying `REVOKE`, made exclusive by **`reissue(region)`**: tear + down every mapping and IPC-buffer binding of the Region in every address space (§7 says why the walk is + unavoidable), complete the invalidation (§12), advance the generation so every other capability is stale, + and return a fresh capability with the caller's rights over the same frames. **Destruction** is the same + walk without the re-mint; frames return to the payer only after the invalidation completes. + +`reissue` is **RFC-0003 §7's destruction case** — generation bump and re-mint, B2's shape on one object — and +pre-empts nothing in RFC-0003a's B1/B2/B3 choice. Gating it on `REVOKE` is a **proposed reading**: RFC-0003 +§5 defines `REVOKE` over capabilities derived from the holder's, and `reissue` also reaches holders that were +not. Copies the donor derived first die at the bump; a former sharer takes an ordinary fault (RFC-0006 +(proposed) provides delivery) — the "observable and survivable" revocation research/0002 Part 6 asks for. + +## 11. How the kernel touches user memory (O-5) + +**The kernel never dereferences a user virtual address** — not through a checked copy or a fixup table, not +at all *(seL4: the IPC buffer is a frame reached through the kernel window)*. Everything the kernel reads or +writes for a process is named by capability and reached through the physmap: + +- **The IPC buffer** — RFC-0004's spill area and slow-path copy — is one page of a Region bound to a thread + by capability (RFC-0006 (proposed) records it on the Thread; RFC-0007 (proposed) provides the syscall). + Binding needs `READ | WRITE`, holds a counted reference, is a writable use for §9, and `reissue` severs it. +- **No kernel path faults on user memory**: frames are eager and pinned, so the slow path copies frame to + frame, and the in-kernel fault concurrency that sank L4's long IPC cannot arise. +- **Peer-writable memory is never referenced.** No Rust reference is formed into it: one `kernel/src/mm/` + module copies with volatile accesses, and anything the kernel must *interpret* is copied once and validated + as that snapshot *(rust-vmm; virtio's double-fetch lesson, research/0002 Part 5)*. + +**The constraint RFC-0007 inherits: no syscall takes a pointer argument, ever.** Data travels in registers, the +IPC buffer, or as a Region handle plus offset; RFC-0003 §9 forbade naming *resources* by address, and this +extends it to data. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. +`SMAP` and `PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. + +## 12. TLBs, ASIDs and PCIDs + +- **Tags are kernel-private.** An ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8; + `TCR_EL1.A1 = 0`, so it rides in `TTBR0_EL1`) or a PCID (12 bits, where CPUID reports it). Tag 0 is + reserved. Exhaustion bumps a generation, invalidates all and reassigns lazily *(Linux + `arch/arm64/mm/context.c`)*. *Rejected:* seL4's ASID pools as capabilities — a cache tag is not authority. +- **Kernel entries are global, user entries never**; without PCID (QEMU's default x86_64 CPU) a `CR3` load + flushes the non-global ones. **`map` needs no invalidation**: neither architecture caches a faulting + entry (the Arm ARM; SDM Vol. 3A §4.10), though x86_64's paging-structure caches can cause one spurious + fault the handler re-checks. `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS` (ASID + and address, leaf only), `DSB ISH`, `ISB`; x86_64 uses `INVLPG`, and `INVPCID` or a stale-on-switch mark. + **Freeing a table frame needs more**: `VALE1IS` leaves walk caches alone, so any path that frees an + intermediate table (§18) uses `TLBI VAE1IS` or `ASIDE1IS`; x86_64's `INVLPG` and `INVPCID` already flush + the PCID's paging-structure caches (SDM Vol. 3A §4.10.4.1). Live remaps use break-before-make. +- **O-8's ordering: invalidate, then free.** No frame returns to a Pool while any core can translate to it. +- **Inner-shareable from day one.** The demo is `-smp 1`, but every `TLBI` is the broadcast form. x86_64 has + no broadcast; its shootdown IPIs are RFC-0006 (proposed)'s multi-core shape, walking §7's mapping lists. + +## 13. Early boot: turning the MMU on + +The image is linked at `0xFFFF_FFFF_8020_0000` and loaded at 0x4020_0000 (`AT()` in `aarch64.ld`); QEMU loads +segments at `p_paddr` and translates the entry point to its physical alias (`include/hw/elf_ops.h.inc`; +checked on 8.2.2), and the stub is already position-independent. With the MMU off, the stub: + +1. Parks secondaries, installs the stack and zeroes `.bss` at physical addresses, as today. +2. `MAIR_EL1`: index 0 = `0xFF` (Normal write-back, read/write-allocate), index 1 = `0x04` (Device-nGnRE). +3. `TCR_EL1`: `T0SZ = T1SZ = 17`; `TG0 = 0b00` and `TG1 = 0b10`, both 4 KiB (the trap); `SH = 0b11`; + `IRGN/ORGN = 0b01`; `IPS` from `PARange` (44 bits here); `AS = 1`; `A1 = 0`; `TBI = 0`; `HA = HD = 0`. +4. Four page-aligned boot tables in `.bss`: `TTBR0_EL1` identity-maps 0x4000_0000 as a 1 GiB Normal block and + 0x0000_0000 as a 1 GiB Device block, so the UART survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to + 0x4000_0000 as a 1 GiB Normal block. Five descriptors, all `AF = 1`. +5. `DC IVAC` over the tables, written non-cacheable, so no stale line shadows them once walks are cacheable + (Linux `head.S` does the same); `TLBI VMALLE1`; `DSB ISH`; both `TTBR`s; `ISB`; `SCTLR_EL1` written whole + with `M`, `C`, `I` (bits 0, 2, 12) and `nTWI = nTWE = 0`, as RFC-0006 (proposed) asks; `ISB`. +6. Branches to the high alias by absolute address, moves `SP` and `VBAR_EL1` there, and calls `rust_entry`. + **Rust never executes at the low alias.** + +Rust then, in an order that never leaves the exception reporter without a console: (1) reads the DTB through +the high block and builds the `MemoryMap`, bitmap and slot arrays; (2) builds §6's final tables; (3) swaps +`TTBR1_EL1` by an assembly routine run from the identity map — empty table, `TLBI VMALLE1`, final table — +since turning blocks into pages under a live translation needs break-before-make *(Linux +`idmap_cpu_replace_ttbr1`)*; (4) **repoints the console at the UART's MMIO-window address** — the physical +`UART0_BASE` in `uart.rs` dies silently once the identity map goes; (5) only then sets `TCR_EL1.EPD0 = 1`, +so nothing walks `TTBR0_EL1` until the first AddressSpace loads, sets `WXN`, and issues `TLBI VMALLE1`, +`DSB ISH`, `ISB` (`WXN` may be cached in the TLB); the boot tables' frames are freed. + +**From MMU-on to step 5 the kernel is one RWX gigabyte** — before any process exists, but a stray write in +early Rust goes uncaught. The reporter gains the decode this needs: aborts print `FAR_EL1` and the fault +level from `ESR_EL1.ISS.DFSC`; on x86_64, `#PF` prints `CR2` and its error code. **x86_64**, once the UEFI +stub exists, is already paging; Rust builds §6's PML4, sets `IA32_EFER.NXE` (bit 11 — without it `XD` is a +reserved-bit fault), `CR0.WP` (bit 16, so ring 0 honours read-only pages), `CR4.PGE` (bit 7) and +`CR4.PCIDE`/`SMEP`/`SMAP` (bits 17, 20, 21) where CPUID permits, and loads `CR3`. *Rejected:* a separate +loader *(seL4's elfloader)*; Rust building tables with the MMU off, sound only without absolute addresses. + +## 14. Lineage + +| Source | What is taken | What is left | +|--------|---------------|--------------| +| **seL4** | frames as capability objects; the kernel window; never touching user addresses; all remaining memory to the first task; user-chosen layout | Untyped/retype and the CDT; execute as an ungoverned attribute; ASID pools as capabilities | +| **pKVM** *(research/0002 Part 7)* | one owner per frame; share and donate as the only transitions | stage-2 mechanics; a separate page-state table | +| **Zircon / Fuchsia** | a region mapped into many spaces, kept alive by mappings; `EXECUTE` as a right; `VmexResource` | the kernel heap; lazily committed VMOs; VMARs | +| **Hubris** | build-time-sized kernel tables | a fully static task set | +| **Fiasco.OC, Genode** | kernel memory as a counted, tradeable quota | quotas on a general heap | +| **Linux** | the linear map; ASID rollover; arm64 descriptor headers as a checked reference; software PAN as an option | the buddy allocator; `copy_from_user` and its fixups | +| **Ge et al., EuroSys '19** | the colour-capable allocator | time protection itself, for later | + +## 15. Obligations + +| Obligation | This RFC | Status after implementation | +|-----------|----------|-----------------------------| +| O-3 revocability | destruction and `reissue` walk the Region's mapping list, invalidate, then bump; a page-table entry is a cached resolution RFC-0003's invariant cannot cover, so the generation bump alone is not enough for memory (§7, §10) | **destruction case extended to memory**; selective revocation stays RFC-0003a's | +| O-5 argument validation | no user virtual address ever dereferenced; memory reached by capability through the physmap; snapshot-validate peer-writable data (§8, §11) | **discharged for the memory surface**; RFC-0007 inherits "no pointer arguments" | +| O-6 kernel memory safety | new `unsafe` confined to `mm/**` (physmap, volatile copy) and `arch/**` (tables, TLB, boot); engine, allocator and store are safe host-tested code | **stays Built** — every new block listed per PR | +| O-7 no unprivileged exhaustion | typed slot arrays; every post-boot frame and slot charged to a named Pool; charge-first atomic `map`; bounded work per call (§5, §8) | **discharged for memory and object slots**; CPU time is RFC-0006's half | +| O-8 address-space disjointness | per-process tables; Regions only by capability; zero before exposure; invalidate before free; privileged-only kernel half (§4, §7, §12) | **discharged** | +| O-9 no ambient side channel | no shared mapping a capability did not establish (§10) | **contributes** | +| O-13 W^X | four layers, including per Region across all address spaces (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | +| O-2, O-4 | `perms` ⊆ capability rights; `protect` narrows; no `brk`, anonymous `mmap` or physical-address map | **maintained** | +| O-18 confined DMA | single-extent Regions; Device Regions minted at boot | **groundwork only** — Phase 2 | + +## 16. Graves checked (§3) + +- **Policy in the kernel.** No address choice, demand paging, copy-on-write, overcommit or OOM response. +- **Baroque capability hierarchies.** No Untyped tree; Pools are flat and balance moves rather than derives. +- **Multi-copy IPC.** The slow path copies once, frame to frame; bulk data is shared, never copied (§10). +- **Bolted-on multicore.** Broadcast invalidation from the first line; the tag allocator is the SMP design. +- **The catch-all right.** `EXECUTE` is narrow; there is deliberately no "map any physical address" right. +- **Compiled-in unused device paths (VENOM).** The kernel maps only the devices it drives itself. + +## 17. Costs — what this makes harder + +- **Capacities are ceilings**: raising a slot count is a rebuild, and `pool_move` is one call per resource. +- **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `TooFragmented`. +- **47-bit halves on AArch64** give up half of what `T0SZ = 16` allows, for one layout and one user ABI; + **the 64 KiB floor is a fixed constant**, where Linux lets the administrator tune it. +- **No pointer arguments** (§11): an ergonomic cost on every syscall RFC-0007 designs, for a bug class gone. +- **The physmap is a large target**: one kernel write bug reaches all RAM, user code included (§9) — a + software invariant, as are W^X on x86_64 and O-5 without PAN on cortex-a72. +- **Cache maintenance is untestable under QEMU** (§9, §13); and **the load address is a QEMU `virt` fact**. +- **Reclaim needs cooperation or revocation**: stale capabilities pin slots and transferred ones keep a Pool + charged, so `pool_merge` waits — RFC-0003a's problem made concrete. + +## 18. Open questions + +1. **Page-table frames on unmap.** Freed only with the AddressSpace; eager freeing needs §12's walk-cache rule. +2. **One accounting domain?** research/0002 Part 6 wants memory, object quotas and scheduling contexts under + one capability-named domain; the Pool can take time as another resource. RFC-0006 decides. +3. **Which right authorises `reissue` and destruction.** Proposed `REVOKE` (§10); RFC-0003a and RFC-0007 settle it. +4. **Hardware backing for two software rules:** software PAN on Armv8.0 (a `TTBR0_EL1` change per kernel + entry), and unmapping executable user frames from the physmap. +5. **Preemptible teardown.** The O(mappings) walk must be restartable to use RFC-0006 (proposed)'s preemption points. +6. **The firmware assumption** (§4) for threat model §10, via `docs/CHANGELOG.md`. + +## 19. Implementation increments + +One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate — +`no_std` in the kernel, `std` under `cargo test`, built for both targets in CI, no `unsafe`, as `capability/`. + +1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU still off. Boot-test `--expect "DTB 0xd00dfeed at 0x40000000"`. +2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs; + `CLAUDE.md` § Layout gains `memory/`. Boot-test `mm: 512 MiB usable at 0x40000000, N frames, M reserved`. +3. **[demo] Frame bitmap.** Host churn tests against a shadow model: no frame twice, none reserved. +4. **[demo] `EXECUTE` in `capability/`.** Bit 5, `ALL` edited by hand and its comment corrected, `Debug`, + the test widened to `0..64`. Host-only; demo-needed because the root maps the client's text. +5. **[demo] `MapPerms` and both encoders.** Exhaustive host tests: nothing writable and executable; every + user entry `PXN`, `AF`, not-global; every row of §8's table. +6. **[demo] The table engine** over a frame-access trait. Host tests on both formats: overlap, kernel-half + refusal, the 512-leaf and six-table bound, nothing mapped when a table cannot be paid for. +7. **[demo] Object store and Pools (§5).** Host tests: conservation under random create, destroy, split, + move and merge; `PoolExhausted` names the payer. +8. **[demo] MMU on (§13, stub).** Higher-half relink, boot tables, `VBAR_EL1` high, `FAR_EL1`/DFSC decoding. + Boot-test: `Kaya!` from the high alias, and the exception self-test still passes. +9. **[demo] Final kernel map (§13, Rust)**, the console repointed in the same PR; `EPD0`, `WXN`. Boot-test + `--features provoke-wx` writes to `.text`, expecting `data abort, same EL` and `FAR = 0xffffffff8…`. +10. **[demo] Region, AddressSpace, Mapping** (§7–§9, §12) charged to the root Pool, with per-Region W^X + counts, the tag allocator (rollover host-tested) and `TTBR0_EL1` switching. Boot-test: `AT S1E0R` under + two ASIDs resolves one shared Region to the same frame and private ones apart: `mm: shared region agrees`. +11. **User-memory module (§11).** Boot-test with a message beyond the register budget. Not demo. +12. **Destruction and `reissue`.** Boot-test that a reissued sharer faults. Not demo. +13. **PAN, SMAP, SMEP** where reported, with a `provoke-user-deref` self-test under `-cpu max`. Not demo. +14. **x86_64** tables, `CR3` and PCID, after the UEFI stub. Not demo. +15. **Device Regions** — Phase 2, with the first userspace driver. + +**Interims the demo carries, with their cost:** + +- **One root Pool.** `pool_split`/`move`/`merge` are host-tested, not yet syscalls: server and client + charges are indistinguishable. +- **No destruction or `reissue`**: a killed process's frames and slots stay charged; O-3's memory case is + designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. +- **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. +- **One RWX gigabyte** between MMU-on and increment 9, before any process exists; cache maintenance unverified. +- **Not an interim:** the kernel maps the root's image at boot — RFC-0007 (proposed)'s one bootstrap act — + through increment 10's `map` with kernel-minted Region capabilities, so §8's rights check runs even there. + +## 20. What this unblocks + +Increments 1–10 give RFC-0006 (proposed) address spaces, guarded kernel stacks and a Pool to charge threads +to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a constraint +it inherits rather than discovers: **no syscall takes a pointer argument**. The root task receives all memory +as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor; increment 12 closes RFC-0004 +§9.1 in code. Increments 1–3 change nothing about how the kernel runs and can land immediately. diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md new file mode 100644 index 0000000..6a3fef7 --- /dev/null +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -0,0 +1,482 @@ + + +# RFC-0006 — Threads and scheduling + +| Field | Value | +|-------|-------| +| Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | +| Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | +| Date | 2026-09-27 | +| Affects | Constitution §3 ("IPC is the product", priority inheritance, bolted-on multicore); the HAL (`kernel/src/arch/**`) and `aarch64.ld`'s stack plan; RFC-0003 §14.2; RFC-0004 §4, §8, §9.2 and its 2026-08-01 amendment; every userspace driver | +| Depends on | RFC-0003 (capability table); RFC-0004 (IPC) and its 2026-08-01 amendment; RFC-0005 (proposed) for object memory, kernel stacks and address-space switching; RFC-0007 (proposed) for the trap path, the syscall encoding, the Process object and the root task | +| Discharges | O-7 for CPU time and the scheduler's own structures; O-4 for time; both commitments of RFC-0004's 2026-08-01 amendment; answers RFC-0003 §14.2 and RFC-0004 §9.2 in shape; contributes to O-5, O-9, O-17 | + +> **Proposed verdicts.** +> +> - **V1 — Event kernel (the load-bearing choice).** One kernel stack per core; the kernel is non-preemptible; every wait is a thread state. +> - **V2 — Thread.** An RFC-0003 object with a fixed-size TCB holding its user frame; nine states; `yield` and `exit` are authority-free. +> - **V3 — Time is a capability.** A `SchedContext` (budget, period, ≤ 8 refills) is an object, configured only through a per-core `Core`. +> - **V4 — Priorities and dispatch.** 256 fixed levels; an MCP guard, no scheduling right; O(1) bitmap run queues; FIFO endpoint queues. +> - **V5 — Donation and the amendment.** `call` lends time to a passive server at `max(own, caller's)` priority; an RFC-14 threshold; one-hop `BudgetExpired`. +> - **V6 — Timer.** Tickless one-shot deadlines: the AArch64 EL1 virtual timer; the x86_64 LAPIC, TSC-deadline only with an invariant TSC. +> - **V7 — FP/SIMD.** Off by default, enabled per thread, loaded on first use, saved lazily by owner tracking; the kernel never touches it. +> - **V8 — Interrupts.** GICv3 first; LAPIC and I/O APIC; a device line is an `IrqHandler` that signals a notification, masked until `ack`. +> - **V9 — Multi-core shape.** Per-core state; a big lock when SMP lands; cores parked until funded; SMT placement classes; exclusivity as a degenerate case. + +## 1. The question + +**Who runs next, on which core, for how long, and on whose authority — such that no thread runs on time +it was not granted, a server never runs on time nobody paid for, and the kernel never waits on anything +a process controls?** + +That is O-7's scheduler half plus RFC-0004's three scheduling hazards: priority-aware direct switch, +donation, and budget expiry inside a server. This RFC designs the mechanism. It chooses no priorities, +admits no workloads and balances no load; that is for whoever holds the capabilities defined here. + +## 2. Which pillar + +**The kernel doctrine (§3) directly, and pillar 2 by consequence.** "IPC is the product" holds only if a +`call` hands over the CPU and the budget without a scheduler round trip (RFC-0004). Time is the one +resource every thread consumes; if it is not a capability, pillar 2 has a hole exactly where +availability (A5) lives. seL4 MCS made time a capability, and this RFC adopts that shape. + +## 3. The execution model — one kernel stack per core (V1) + +The load-bearing choice, because it decides what a context switch *is*. + +- **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). Any kernel path + may block, and switching threads switches kernel stacks. It costs a guarded stack per thread in a + kernel with no heap, and blocking in the kernel is how "hold a lock across a wait a process controls" + (O-7) becomes possible. **Rejected.** +- **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). + **Verdict sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with + interrupts masked, and leaves by restoring *some* thread's user frame. A syscall that must wait records + a state and the object waited on, and the kernel exits to someone else. On wake, the result is written + into the waiter's frame, or the call restarts by rewinding the saved PC one instruction (4 bytes for + `svc #0`, 2 for `syscall`). *(Heiser & Elphinstone, "L4 Microkernels: The Lessons from 20 Years of + Research and Deployment", TOCS 2016; Ford et al., "Interface and Execution Models in the Fluke Kernel", + OSDI 1999.)* + +**The cost:** every kernel path must be bounded. The long ones — object destruction and RFC-0005 +(proposed)'s `reissue` and address-space teardown — take **preemption points**: poll for a pending +interrupt (`ISR_EL1.I`; the LAPIC IRR) and, if one is pending, leave the operation restartable and exit. +The worst path is measured and published, because it is the interrupt latency. This **reverses +`aarch64.ld`'s plan** that the boot stack "will be replaced by per-thread stacks": it becomes core 0's +kernel stack, moved into RFC-0005 (proposed)'s guarded stack window and resized from a measured +high-water mark. + +## 4. The Thread object (V2) + +A **Thread** is an RFC-0003 object belonging to one Process, which RFC-0007 (proposed) provides. Its TCB +is a slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool presented at creation; +queue links are slot **indices**, not pointers. + +| Field | Meaning | +|-------|---------| +| `frame` | the full user register file, written at every kernel entry (§9) | +| `state`, `blocked_on`, links | a state below; the endpoint, reply object or notification waited on; intrusive queue links | +| `priority`, `mcp` | base priority and maximum controlled priority (§6) | +| `home_sc`, `sc`, `effective` | the bound context, if any; the context held now, own or donated; the priority run at now (§7) | +| `bound_notification`, `ipc_buffer` | the notification a `recv` may also wake on (RFC-0004 §10); the IPC-buffer page bound by capability (RFC-0005, RFC-0007, proposed) | +| `fault_ep`, `timeout_ep` | **handles in the thread's own process table**, resolved when raised — never cached capabilities | +| `fp_enabled`, `fp_state`, `class` | the FP/SIMD flag and save area (§10); the SMT placement class (§12) | + +Resolving fault handlers at fault time keeps RFC-0003's invariant that a resolved capability lives only +in a table, and makes a revoked handler fail closed. *(The seL4 master kernel looks its fault endpoint up +in the faulting thread's CSpace; MCS installs handler capabilities into the TCB.)* + +| State | Meaning | Left by | +|-------|---------|---------| +| `Inactive` | created or suspended; in no queue | `resume` | +| `Ready` | runnable with released budget; in its core's run queue | dispatch | +| `Running` | the current thread on its core; in no queue | preemption, block, `yield`, fault, `exit` | +| `Throttled` | runnable, but no released refill; in the release queue | refill release | +| `BlockedOnSend` | queued on an endpoint to `send` or `call` | rendezvous, endpoint destruction | +| `BlockedOnRecv` | waiting on an endpoint (and any bound notification) | rendezvous, signal | +| `BlockedOnReply` | waiting on a reply object after `call` or a fault message | reply, error completion (§7) | +| `BlockedOnNotification` | waiting on a notification alone | signal | +| `Exited` | finished; in no queue; the object lingers until destroyed | never | + +Faults reuse IPC: the kernel `call`s the fault endpoint on the thread's behalf and it waits in +`BlockedOnReply`; RFC-0007 (proposed) defines the message. Destroying a thread bumps its generation +(RFC-0003 §7), unlinks it in O(1) from whichever doubly linked queue holds it, destroys any reply object +it waits on (RFC-0004 §8) and returns any donated context as budget expiry does (§7). + +| Operation | Authority | Effect | +|-----------|-----------|--------| +| `set_priority(t, auth, p)` / `set_mcp(t, auth, m)` | `t`: `WRITE`; `auth`: `READ`; value `≤ mcp(auth)` | §6 | +| `bind_sc(t, sc)` / `unbind_sc(t)` | `t`, `sc`: `WRITE` | active or passive; class checked (§12) | +| `resume(t)` / `suspend(t)` | `t`: `WRITE` | `Inactive` ↔ runnable | +| `read_regs(t)` / `write_regs(t, …)` | `READ` / `WRITE` | frame and TLS bases, sanitised (§9) | +| `set_fp(t, on)` / `set_class(t, c)` | `t`: `WRITE` | §10 / §12 | +| `sc_configure(core, sc, B, T, refills, class)` / `sc_stats(sc)` | `core`, `sc`: `WRITE` / `sc`: `READ` | §5 | +| `yield()` / `exit()` | none — two of RFC-0003 §9's three authority-free exceptions | tail of own priority / `Exited` | + +No right joins RFC-0003's set; RFC-0007 (proposed) owns the encoding. + +## 5. Scheduling contexts and `Core` — time as a capability (V3) + +**Options.** A timeslice on the thread (classic L4, Zircon) makes creating a thread *acquiring CPU*, with +nothing to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are +**subsumed**: a partition is what a `Core` holder builds from several contexts. **Verdict sought:** +contexts as objects *(seL4 MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", +EuroSys 2018)*. A **SchedContext** holds budget `B`, period `T` (`B ≤ T`), at most eight refills, its core, its placement +class, its home thread and its current holder. A thread with a bound context is **active**; one without +is **passive** and runs only on donated time (§7). Its *memory* is charged to a Pool; its *time* comes +from a `Core`. Folding both into one accounting domain (RFC-0005 (proposed)'s open question) is left to +the resource RFC: they are different resources, and Phase 1 needs neither grouped. + +- **`Core`, one per core, minted to the root task** (RFC-0007 (proposed)). `sc_configure` through core + *c*'s `Core` binds the context to *c*: the authority to sell time on *c* and nothing else. `READ` + yields topology — package and SMT siblings, from devicetree `cpu-map` or ACPI PPTT via RFC-0005 + (proposed)'s discovery. No syscall names a core by number. *(seL4 `SchedControl`; research/0002's + "core-set capability".)* `B` below **`MIN_BUDGET`** — twice the measured worst-case kernel path, seL4 + MCS's own rule (`include/kernel/sporadic.h`) — is refused, so nobody configures a timer storm. +- **Full contexts** (`B = T`) are never throttled and round-robin with timeslice `T`. **Partial contexts** + (`B < T`) are sporadic servers: within any window of length `T` the holder consumes at most `B`. + Refills fall due a period after consumption; when all eight are in use they merge, forfeiting + bandwidth rather than exceeding it. *(Sprunt, Sha & Lehoczky 1989.)* +- **What a budget means** (research/0002: minima, not caps; burst and redistribution stated). The kernel + enforces the **cap**; dispatch is **work-conserving**; the **minimum** follows from admission, the `Core` + holder's policy. Unused budget never carries past its window. +- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21, research/0002). + The window is each context's own sliding period. Billing is exact, from the counter (`CNTVCT_EL0`; the + TSC) read at every kernel entry and exit: 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 + 62.5 MHz default (the flag is set in QEMU `target/arm/tcg/cpu64.c` and applied in `target/arm/cpu.c`). + Kernel time, interrupts included, is billed to the context running on entry. The ABI speaks + nanoseconds; budgets round down and periods up, so configured bandwidth is never exceeded. +- **Stall counters from the first version** (research/0002 Part 6, after PSI and oomd): per context, time + consumed, time and count throttled, time ready but waiting, and budget expiries, read-only via + `sc_stats` — the O-17 supervisor's wedged-server signal. The kernel measures; userspace acts. + +## 6. Priorities and dispatch (V4) + +**256 levels**, 255 most urgent, meaningful within a core. `set_priority(t, auth, p)` needs `WRITE` on +`t`, `READ` on `auth` and `p ≤ mcp(auth)`; MCP is set the same way, so authority over urgency only +narrows as it is delegated — O-2's shape. The root task's thread starts with MCP 255 and a full context +on core 0, the one bootstrap act; the kernel assigns no other priority. *(seL4 `TCB_SetPriority`.)* + +**Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all right in embryo, one bit whose meaning +grows with every scheduling operation (the CAP_SYS_ADMIN grave). The MCP is a monotone ceiling, a relation +between two capabilities, and needs no bit. **Rejected: dynamic priorities** (CFS, BSD decay, boosts): +kernel-resident policies churned in every kernel that carried them while accounting and enforcement +endured (research/0002 Part 1, "Mechanism/policy split"). + +- **Run queues.** Per core, 256 intrusive FIFO lists and a 256-bit bitmap: the next thread is a + count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; + one whose timeslice ended, at the tail. The running thread is never queued, and a thread switched to + directly is never enqueued at all. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* +- **Release queue.** Per core, `Throttled` threads sorted by next refill: O(n) insertion in the contexts + bound to that core, a number only the `Core` holder can raise. +- **Idle is a kernel loop, not a thread**, billing nobody. The subtlety is the lost wake-up. AArch64 runs + `wfi` with `PSTATE.I` set — a pending interrupt still wakes it — then unmasks briefly to take it; Linux + (`arch/arm64/kernel/idle.c`) relies on the same property and warns that masking at `ICC_PMR_EL1` would + *not* wake the core, so this kernel masks with DAIF only. On x86_64 `hlt` with `IF = 0` ignores maskable + interrupts, so the loop is `sti; hlt`, whose STI shadow issues the `hlt` before delivery. +- **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has + released budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority + is at least the highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also + strictly more urgent than *C*. Otherwise *W* is queued. A *W* without released budget goes to the + release queue; one on another core is enqueued there and sent an IPI (§12). +- **Endpoint queues are FIFO** — a refinement of RFC-0004, which fixed no order: O(1) insertion keeps the + hottest path bounded with interrupts masked. *(seL4's current MCS `tcbEPAppend` is FIFO.)* **Rejected: + priority order** *(QNX)* — an O(n) walk over a thread count any process can grow from its own Pool. + +## 7. Donation, inheritance and budget expiry (V5) + +**Donation.** When *C* `call`s and the receiver *S* is passive, the kernel moves *C*'s current context to +*S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves it back. +*S* runs at `max(prio(S), eff(C))`: the constitution's priority inheritance along the call chain, with +*S*'s own priority a **ceiling floor** that policy can raise to the highest client priority for the +priority-ceiling bound seL4 MCS relies on. Lending priority is safe because time is lent with it. A `call` +to an **active** receiver donates nothing. Chains compose one hop at a time. *(Ford & Lepreau, migrating +threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* + +**Rejected: a passive server at its own priority only** (seL4 MCS), which leaves inversion to a global +convention. **Rejected: boosting a busy server to a waiting sender's priority** (QNX): the kernel would +guess which thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. + +**The 2026-08-01 amendment, discharged**, following seL4 RFC-14 (Mitchell Johnston, "Budget limit +thresholds on endpoints for SC Donation", proposed 2023-08-15; seL4/rfcs pull request 24, opened +2024-06-17, still open; read in full for this RFC): + +- **Prevention — the threshold.** An endpoint carries a `threshold`, fixed at creation (0 means none; + RFC-0007 (proposed) carries the parameter). A `call` needs released budget of at least `threshold` + plus twice the kernel's worst-case path, paying for the `call` and `reply` themselves — RFC-14's rule. + A context whose `B` can never meet that fails with `BudgetBelowThreshold`; a non-donating `send` to a + thresholded endpoint is invalid. With `threshold` at the server's WCET, expiry inside it is a true error. +- **A flagged deviation.** The amendment says donation below the threshold is "refused up front". RFC-14, + which it cites, refuses only a context that can *never* pass; one that cannot pass *now* has its refills + merged and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in + an event kernel, O(refills) ≤ 8. This RFC proposes RFC-14's behaviour: no donation happens below the + threshold, so the amendment's purpose holds, whereas refusing a transient shortfall would make every + client spin on its own budget to retry. If accepted, it is logged against RFC-0004 as a dated amendment. +- **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* + runs on it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with + `BudgetExpired` (observed when the context next refills); *R* is consumed, so a late `reply` fails + closed. *S* is left runnable but unfunded, registers intact, and the context's expiry counter rises. If + *S*'s `timeout_ep` resolves, the kernel raises a **timeout fault** carrying *S*'s identity and consumed + time; the handler, active with time of its own, abandons the request (`write_regs`) or lends *S* a + context to finish. With no handler, *S* stays stranded, contained to its own clients. *(seL4 MCS: + `tcbTimeoutHandler`, `seL4_Fault_Timeout`.)* +- **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply + to A with an error. Unwinding to A would walk the reply stack faulting every level — the O(n) walk + RFC-14 also rejects. This kernel-initiated completion is the one the amendment left to this RFC: a + status on the reply object, no new object. +- **A reconciliation.** The amendment says seL4's timeout exceptions "have remained future work"; they + ship in MCS, and RFC-14 calls them the kernel's current mechanism — what is unmerged is RFC-14's + threshold. The phrase, from research/0002, wants a dated correction; the commitment is unchanged. + +**Rejected: the unwind without timeout faults**, where the stranded server is visible only through its +state and counters. Timeout faults are no second protocol — they travel RFC-0007 (proposed)'s fault path +— and a supervisor woken by a message beats one polling every passive server; the no-handler case *is* +that unwind. **Deferred: RFC-14's budget limits** (capping what a server spends of a donated context): +RFC-14 measured 22% extra fastpath cost on `call` and 21% on `reply_recv`, against 13% for thresholds. + +## 8. The timer and preemption (V6) + +One HAL surface: `now() -> Ticks`, `frequency() -> Hz`, `set_deadline(Ticks)`, `cancel()`, and a +`timer_fired()` up-call. The kernel is **tickless**: each exit programs one per-core deadline — the +earliest of a partial context's exhaustion, the timeslice boundary if an equal-priority thread is ready, +and the next refill due. With no competitor (§12's exclusive core) the timer is cancelled. + +- **AArch64: the EL1 virtual timer** — `CNTV_CVAL_EL0`, `CNTV_CTL_EL0` (`ENABLE`, `IMASK`, `ISTATUS`), + `CNTVCT_EL0`, and `CNTFRQ_EL0` always read, never assumed. PPI INTID 27, the Arm BSA assignment QEMU's + `virt` uses (`include/hw/arm/bsa.h`). Preferred to the physical timer, whose EL1 access an EL2 beneath + us can trap (`CNTHCTL_EL2`); the EL2-to-EL1 descent `boot.s` owes must grant EL1 timer access and zero + `CNTVOFF_EL2`. *(Arm ARM DDI 0487, "The Generic Timer".)* +- **x86_64: the local APIC timer**, x2APIC where offered (`IA32_APIC_BASE` bit 10). TSC-deadline mode (LVT + mode `10b` in bits 18:17; `IA32_TSC_DEADLINE`, MSR `0x6E0`) only when `CPUID.01H:ECX[24]` **and** + invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; otherwise one-shot through the initial-count + register, calibrated at boot, behind the same `set_deadline`. QEMU's TCG lacks TSC-deadline + (`target/i386/cpu.c` lists it as missing), so one-shot is what the emulator runs, not dead code. The + console prints the mode chosen. *(Intel SDM Vol. 3A, the APIC chapter.)* +- **Kernel-reserved.** The timer line is never minted (§11). EL0 `wfi` traps (`SCTLR_EL1.nTWI = 0`) and + completes as `yield`; `hlt` at CPL 3 faults. EL0 `wfe` does **not** trap (`nTWE = 1`): spin-lock backoff + uses it, trapping would make every spin a kernel entry, and the armed timer's interrupt wakes it anyway. + RFC-0005 (proposed) writes `SCTLR_EL1` and must carry these two bits. + +## 9. The context switch — what is saved, where + +In an event kernel a context switch is *which frame the exit path restores*: no kernel stack changes +hands. Assembly is confined to the entry and exit stubs in `kernel/src/arch/**`. **The user frame lives +in the TCB.** On AArch64, while a thread runs at EL0, `SP_EL1` points at the end of its TCB frame, so +the entry stub's stores land there; only then does it load the per-core kernel stack from the block +`TPIDR_EL1` addresses. On x86_64 an interrupt or exception from ring 3 loads `RSP` from `TSS.RSP0`, +which points at the TCB frame, so the CPU's pushes land there — but **`syscall` does not switch `RSP`**: +its stub must `swapgs`, park the user `RSP` and load the frame pointer itself to reach the same layout. +RFC-0007 (proposed) owns both entry sequences and `sysretq`'s canonical-`RIP` check; this RFC owns the +layout. The address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s call, +made on exit when the process differs — the single point where a time-protection flush would go (Ge et +al., EuroSys 2019; research/0002 Part 6). + +| What | AArch64 | x86_64 | +|------|---------|--------| +| Every entry, into the TCB | `x0`–`x30`, `SP_EL0`, `ELR_EL1`, `SPSR_EL1`: 34 words | `SS`, `RSP`, `RFLAGS`, `CS`, `RIP`, an error-or-vector slot, 15 GPRs: 21 words | +| Thread-local bases, per switch | `TPIDR_EL0` (EL0-writable) saved and restored; `TPIDRRO_EL0` restored | `FS_BASE` and user GS base restored; `CR4.FSGSBASE` off, so only the kernel sets them | +| Also per switch | `CPACR_EL1.FPEN` (§10) | `CR0.TS` (§10); `TSS.RSP0` | +| Lazily (§10) | `V0`–`V31`, `FPCR`, `FPSR`: 520 bytes, 16-byte aligned | XSAVE area, `XCR0` = x87, SSE, AVX: 832 bytes, 64-byte aligned | +| Never | kernel registers, kernel FP (soft-float), debug and PMU state | the same | + +**The TCB's real size.** AArch64: a 36-word frame (288 bytes), 528 bytes of FP area and about 150 bytes +of scheduling and IPC fields — about 970 bytes, allocated as **1 KiB**. x86_64: a 23-word frame (184 +bytes, padded to 192), the 832-byte XSAVE area and the same fields — about 1,180 bytes, allocated as +**1.25 KiB**. + +- **O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` + (EL0t, DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only + status flags (IOPL 0, IF set), and a non-canonical `RIP` or base is refused. `SCTLR_EL1.UMA = 0`. +- NMI, `#DB`, `#DF` and `#MC` run on IST stacks, as RFC-0007 (proposed) asks: they can arrive in the + `syscall` window before `RSP` is switched. AArch64 has no such window. + +## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (V7) + +Both kernel targets are soft-float (CLAUDE.md § Architectures), so FP/SIMD state is purely user state. + +- **Enabled per thread** by `set_fp`, off by default. A thread with FP off that touches FP takes an + ordinary fault, so IPC-heavy servers never own FP state and never force a save. +- **Loaded on first use, saved lazily.** Each core records an FP *owner*; every other thread runs with + access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or `CR0.TS = 1` + (`#NM`, vector 7). On that trap, for an FP-enabled thread, the kernel saves the owner's registers, + loads the new owner's (zeroed on first use), grants access (`FPEN = 0b11`; `clts`) and retries. A + `call` → FP-off server → `reply` round trip saves nothing. State is saved eagerly before a context moves. +- **AArch64's asymmetry, stated.** No `FPEN` value traps EL1 while allowing EL0 (`0b01` is the reverse), + so while an owner runs a stray kernel FP instruction would not trap; the soft-float target is the guarantee. +- **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format + area (512 legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. +- **The known cost — LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a + non-owner can speculatively read the owner's registers before `#NM` resolves. Hardware side channels are + out of scope (threat model §7) and CLAUDE.md asks for lazy saving; the cure is Open question 2, costed in §16. + +## 11. Interrupts — the controller, and device IRQs as notifications (V8) + +The kernel owns the interrupt controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`; Redox +and QNX userspace drivers.)* + +- **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. + Kernel-reserved lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. + `bind(irq, notification, bits)` needs `WRITE` on both; `ack(irq)` needs `WRITE`. Routing a line to core + *c* needs the `IrqHandler` and *c*'s `Core`. +- **Delivery.** The kernel acknowledges, **masks the line**, signals end of interrupt, ORs `bits` into the + notification and wakes a waiter through §6's dispatcher; the driver's `ack` unmasks. A level-triggered + line cannot storm. A destroyed notification leaves the line masked (fail closed); the `IrqHandler` + outlives a crashed driver, and a restarted one re-binds (O-17). +- **AArch64: GICv3 first.** System-register CPU interface (`ICC_SRE_EL1.SRE`, `ICC_IGRPEN1_EL1`; acknowledge + `ICC_IAR1_EL1`; end `ICC_EOIR1_EL1`; `ICC_PMR_EL1` open); SPIs through `GICD_ISENABLER`/`ICENABLER` + and `GICD_IROUTER`; SGIs and PPIs through the redistributor's `GICR_ISENABLER0`, woken by + `GICR_WAKER`. Each line the kernel enables is put in Group 1 explicitly, since Group 0 would arrive as + FIQ. On `virt`: distributor `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`). + Chosen for affinity routing past eight cores and an interface needing no MMIO mapping. *(Arm IHI 0069.)* + **Cost:** `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), so xtask gains + `gic-version=3`; and the Raspberry Pi 4 in Constitution §6's bring-up order has a GIC-400 (GICv2), so a + second driver behind the same HAL is a **known** debt, not a maybe. +- **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask + bit; end of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** + without IOMMU interrupt remapping a device can aim any vector at any core — O-18's precondition again. +- **Latency.** Interrupts are taken at EL0, in idle or at a preemption point, so latency is §3's worst + path — which includes RFC-0007 (proposed)'s kernel console write until a userspace UART driver exists. + +## 12. Multi-core shape, placement and exclusive cores (V9) + +Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. + +- **Everything scheduling-related is per core**: run and release queues, kernel stack, timer, FP owner, + current thread. A thread runs on the core of the context it holds; moving a context is `sc_configure` + through another `Core`. **The kernel never load-balances.** A passive server runs where its time is. +- **Cross-core wake-ups** (RFC-0004 §9.2): enqueue *W* on its core and send an IPI — an SGI through + `ICC_SGI1R_EL1`; a fixed vector through the LAPIC ICR. Direct switch is same-core only. +- **One big kernel lock when SMP lands**, taken on entry *(Peters, Danis, Elphinstone & Heiser, "For a + Microkernel, a Big Lock Is Fine", APSys 2015)*. **This answers RFC-0003 §14.2:** a table is touched only + under the lock, so the resolve→check→act window its 2026-07-30 amendment makes a correctness dependency + holds across the whole entry, with no per-table lock on the fast path. +- **Cores are parked until funded (default-deny).** Secondary cores stay in `boot.s`'s `wfe` loop until a + context is configured onto them through their `Core`; a core nobody funds never runs anything. +- **SMT placement invariant** (research/0002 Part 6, after L1TF and MDS). Contexts and threads carry a + placement class set by policy. `sc_configure` refuses a class that differs from any context on the + funded SMT sibling; `bind_sc` and **donation** refuse a thread of another class — the class travels with + the context, so a passive server serving two classes needs a thread per class. O(1) at bind and call; + no gang scheduling. SMT-off is "never fund the sibling"; one class is the vacuous default. +- **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one + thread, a full context, nothing else on the core. The bitmap has one bit, the timer is never armed, no + IPI arrives because nothing is queued there, and device lines are routed elsewhere with the `Core`. No + flag is needed: only a `Core` holder can bind another context, and a holder promising exclusivity to a + third party transfers the capability without keeping a copy. + +## 13. Lineage + +| Source | What is taken | What is left | +|--------|---------------|--------------| +| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl` as `Core`; MCP; sporadic refills; `MIN_BUDGET`; timeout faults; `IRQHandler` | own-priority-only passive servers; handlers cached in the TCB | +| **seL4 RFC-14** (Johnston) | the endpoint threshold, its 2 × WCET rule, deferral, one-layer return | budget limits, for now | +| **Heiser & Elphinstone 2016; Fluke 1999** | the event kernel | per-thread kernel stacks | +| **Elphinstone & Heiser 2013** | the running thread unqueued; priority-aware direct switch | lazy scheduling; L4's unconditional switch | +| **QNX Neutrino** | priority inheritance across the message; partitions built on budgets | full inheritance to busy servers; priority-ordered receive | +| **Ford & Lepreau 1994** | the migrating-thread picture of `call` | Mach | +| **Sprunt, Sha & Lehoczky 1989** | the sporadic server | an unbounded replenishment list | +| **Jailhouse** | refusing to share: parked cores, exclusive assignment | static-only partitioning | +| **Peters et al. 2015** | one big lock | fine-grained locking before measurement | +| **Linux** | the idle wake-up rule; PSI-style stall counters | dynamic priorities | + +## 14. Obligations + +| Obligation | This RFC | Status after implementation | +|-----------|----------|-----------------------------| +| O-7 no unprivileged exhaustion | time only through contexts; `MIN_BUDGET`; eight refills; O(1) expiry return; FIFO endpoint queues; no wait holding a lock; lines masked until `ack` | **discharged for CPU time and scheduler structures**; kernel memory is RFC-0005's half | +| O-4 no ambient authority | no thread runs without a context; priority through MCP; time through `Core`; interrupts through `IrqControl`; only `yield` and `exit` authority-free | **discharged for time** | +| O-5 argument validation | `write_regs` sanitises `SPSR_EL1`/`RFLAGS`, refuses non-canonical `RIP` and bases; every operation takes handles | **discharged for this surface** | +| O-9 no ambient side channel | no EL0 timer or PMU; SMT placement classes; parked cores | **contributes** — fine-grained timing channels stay out of scope | +| O-17 contained drivers | lines as notifications, masked until `ack`; `IrqHandler` outlives a crashed driver; stall counters | **enables** | +| O-18 confined DMA | MSI deferred until interrupt remapping is designed | **named, not discharged** | +| RFC-0004 amendment, 2026-08-01 | (1) threshold at `call`, deferral flagged; (2) `BudgetExpired` on the reply object, plus timeout faults (§7) | **both commitments discharged** | + +## 15. Graves checked (§3) + +- **Policy in the kernel.** No admission, balancing or budget choice. The one number the kernel chooses is 256. +- **The catch-all right.** Priority authority is a relation (MCP); time is per core; interrupts per line. +- **Bolted-on multicore.** Per-core state, IPIs, the lock and placement are stated before any SMP code. +- **Drivers in the kernel; compiled-in unused device paths.** The kernel masks and signals, never services + a device; one controller and one timer per target; unclaimed lines never enabled. +- **Baroque hierarchies; multi-copy IPC.** Contexts are flat; donation moves a reference to time. + +## 16. Costs — what this makes harder + +- **Every kernel path must be bounded and restartable**, and interrupt latency is the worst of them. +- **Every server needs someone to think about time.** A passive server cannot run until called; an active + one needs a configured context. "Spawn a thread and it runs" is gone — O-4's price, in time. +- **The one-hop unwind leaves a stranded server to userspace.** The kernel makes the failure visible and + restarts nothing: a supervisor is required before any passive server on partial contexts is trusted. +- **MCS is the least-verified part of seL4's lineage**, and sporadic refills are subtle — hence property tests. +- **About 1 KiB per TCB on AArch64 and 1.25 KiB on x86_64**, FP space included for threads that never use it. +- **FIFO endpoint queues** let a high-priority client wait behind earlier low-priority ones. +- **Lazy FP's cure, costed.** Eager restore on a switch into an FP-enabled thread of another process moves + 520 bytes (16 `ldp q` pairs) or an 832-byte `XRSTOR` — 9 or 13 cache lines — and nothing on IPC to FP-off + servers. The cycles are a measurement owed; beside a `call` round trip of roughly 380–630 cycles (twice + research/0001's one-way 188–316) it matters only for FP-heavy clients. + +## 17. Open questions + +1. **EL0 counter access.** Proposed default: a per-thread flag, off unless set with `WRITE` on the Thread, + switched with the thread (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`, written only when it differs). Denial costs + a web server a kernel entry per timestamp, and threat model §7 puts fine-grained timing out of scope. + EL0 timers, the physical counter and the PMU stay off regardless. +2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §16)? +3. **RFC-14 budget limits.** Worth 21–22% on the fastpath to stop a passive server overspending? +4. **Passive servers on notifications.** MCS binds a context to a notification; passive drivers need it. +5. **Userspace timers** (RFC-0004 §9.3): kernel deadline objects off the release queue, or a time server? +6. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst kernel path; x86_64's clock + when the TSC is not invariant. + +## 18. Implementation increments + +Pure logic lands in a new host-tested `no_std` workspace crate, `scheduler/`, generic over a thread index +as `capability/` is over `ObjectRef`, built for both Tier-1 targets in CI. Hardware behaviour is proved by +QEMU boot-test strings. Every new `unsafe` block lives in `kernel/src/arch/**`; no new designated tree. +The x86_64 HAL gains each signature as a documented no-op that returns — its entry still calls straight +through into the kernel proper (CLAUDE.md § Architectures), so the second build compiles the scheduler. +**[demo]** marks what the Phase-1 demo needs. + +1. **`scheduler` I — run queues, states, MCP. [demo]** O(1) pick, head/tail re-entry, legal transitions, + the MCP relation. Host unit tests and an exhaustive transition table. +2. **`scheduler` II — full contexts and billing. [demo]** Round-robin budgets, tick rounding, stall + counters, `MIN_BUDGET`. Host tests against a shadow model. +3. **GICv3 and the virtual timer, kernel-only. [demo]** xtask's machine line gains `gic-version=3`; the + kernel's own handler prints `timer: tick` twice — the GIC, the PPI and the idle loop, proved. +4. **Event-kernel dispatch. [demo]** Trap frame in the TCB; two kernel self-test threads (EL1t, on + `SP_EL0`, behind a `sched-selftest` feature as `provoke-exception` is) alternate under the timer: + `sched: A B A B`. The boot stack becomes core 0's kernel stack; `aarch64.ld`'s comment changes with it. +5. **EL0 entry and exit** (with RFC-0005 (proposed)'s user mappings and RFC-0007 (proposed)'s trap stub). + **[demo]** A spinning EL0 thread is preempted at arbitrary instructions and resumes with a register + pattern intact: `user: still here`. +6. **Thread, SchedContext and `Core` objects; `sc_configure`, `bind_sc`, `resume`, `set_priority`, `exit`. + [demo]** The root task configures the demo processes; two EL0 threads at equal priority interleave. +7. **IPC blocking states, FIFO endpoint queues, the direct switch** (with RFC-0004's endpoint PR). **[demo]** + Host tests for the switch decision; boot-test: the client `call`s an **active** server and RFC-0007 + (proposed)'s `Kaya!` strings appear. +8. **Passive servers, donation, inheritance.** A passive server runs only when called, at its client's + priority. A demo stretch goal if 1–7 land early. +9. **`scheduler` III — partial contexts.** Sporadic refills, merge, release queue. Property test: no + window of length `T` sees more than `B` consumed. +10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test: `BudgetBelowThreshold` + for an under-budgeted client, `BudgetExpired` and a timeout fault for an overrunning server. Must + land before any passive server on a partial context is trusted. +11. **Lazy FP/SIMD.** Two FP-enabled EL0 threads load different markers into `v0`, are preempted in turn + and each finds its own: `fp: isolated`. +12. **`IrqControl`/`IrqHandler`.** PL011 receive (SPI 1, INTID 33) reaches an EL0 driver as a notification + when xtask writes a byte to the serial port: `irq: notified`. +13. **x86_64 scheduling HAL** — IDT, TSS, IST, LAPIC, `#NM`, `XSAVE` — after its boot RFC; the strings of + 3–5 and 11 again, proving the HAL held. +14. **SMP** — secondary cores from the park loop, the big lock, IPIs, placement classes. + +**What the demo takes as interim, honestly.** The demo needs increments 1–7 and runs full contexts only. +Its `call` goes to an **active** server, so RFC-0004 §4's accepted `call` semantics — donation — are not +exercised by it; that is increment 8, one more PR, safe to pull in before 9 and 10 because a lent full +context (`B = T`) refills at once and merely re-queues its holder: expiry inside a server needs a partial +context, which the demo never configures. Its userspace stays soft-float, so 11 is not needed. The x86_64 +no-op HAL functions are the other interim, removed by 13. Nothing else is smuggled. + +## 19. What this unblocks + +RFC-0004's implementation gets threads to rendezvous between. The driver framework receives interrupts as +notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `Core`, MCP, +thresholds and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment (14), +and dated notes against RFC-0004 for §7's deviation and reconciliation. From 4c69180702a52d880de922636e0c25a333c74690 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:06:36 +0000 Subject: [PATCH 02/20] wip(docs): snapshot RFC-0005 and RFC-0006 revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 31 ++--- docs/rfcs/0006-threads-and-scheduling.md | 130 ++++++++---------- 2 files changed, 67 insertions(+), 94 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index 8fea185..b940bf9 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -104,10 +104,9 @@ as in seL4; a counted quota, as in Fiasco.OC's factory limits and Genode's quota **The object store is typed slot arrays.** Each object type has one fixed-capacity array in `.bss`; a slot holds the object, a generation and a reference count, and **the slot index is the object's identity**. `ObjectRef` (`capability/src/object.rs`) is `(type, index)` and `current_generation()` is a slot read — no -raw object pointer, slab or size class. A process's capability table lives in its slot -(`CapabilityTable` is fixed-capacity). A destroyed object frees its frames at once; its slot is reused -only when the count, stale capabilities' references included, reaches zero, and is retired, as RFC-0003 -requires, if its generation cannot advance. +raw pointer, slab or size class; a process's fixed-capacity `CapabilityTable` lives in its slot. A +destroyed object frees its frames at once; its slot is reused only when the count, stale capabilities' +references included, reaches zero, and is retired, as RFC-0003 requires, if its generation cannot advance. **A Pool is the accounting layer**: an RFC-0003 object holding balances — frames, and slots of each type. At boot every free frame and slot becomes the balance of one root Pool handed to the root task (RFC-0007 @@ -130,9 +129,8 @@ every unit of r in use == kept by the kernel at boot, or charged to exact when `from` has no outstanding charges. Balance is conserved, never minted — O-2's shape for quantity. - **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). -This is not option B's heap: nothing grows after boot, and the only variable memory — frames — comes from -the bitmap in page units, charged. The kernel keeps its image, the map, the bitmap, the slot arrays, its -final tables and the boot stack. +This is not option B's heap: nothing grows after boot, and frames come from the bitmap in page units, +charged. The kernel keeps only its image, the map, the bitmap, the slot arrays, its tables and boot stack. ## 6. The kernel's virtual layout @@ -155,10 +153,8 @@ top 2 GiB because `x86_64-unknown-none` uses `code-model: kernel`, which require (CVE-2012-0217) for RFC-0007's x86_64 exit path by layout, as Linux does. The physmap covers RAM, never MMIO, so no frame is aliased with conflicting attributes. - **The kernel half never changes after boot.** AArch64 shares `TTBR1_EL1` for free; x86_64 copies PML4 - entries 256–511 into each address space, safe only because they are fixed *(seL4 x86)*. - -*Rejected:* an **identity-mapped kernel** (impossible under the kernel code model); **no physmap** (an -invalidation per IPC-buffer touch, for a side-channel gain threat model §7 puts out of scope). + entries 256–511 into each address space, safe only because they are fixed *(seL4 x86)*. *Rejected:* an + identity-mapped kernel (impossible under the code model); no physmap (an invalidation per IPC-buffer touch). ## 7. The objects: Region, AddressSpace and Mapping @@ -254,7 +250,7 @@ IPC buffers is a convention (§17). ## 10. Sharing and donation (RFC-0004 §9.1) -pKVM's *share* (the owner keeps access) and *donate* (the donor loses it) — research/0002 Part 7 — need one op: +pKVM's *share* (the owner keeps access) and *donate* (the donor loses it), research/0002 Part 7's model, need one operation: - **Share** is `derive` then transfer over IPC: the borrower receives, say, `READ | WRITE` without `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data then travels as RFC-0004 §5's @@ -293,10 +289,9 @@ extends it to data. *Rejected:* `copy_from_user` with fixups (Linux), making ker ## 12. TLBs, ASIDs and PCIDs -- **Tags are kernel-private.** An ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8; - `TCR_EL1.A1 = 0`, so it rides in `TTBR0_EL1`) or a PCID (12 bits, where CPUID reports it). Tag 0 is - reserved. Exhaustion bumps a generation, invalidates all and reassigns lazily *(Linux - `arch/arm64/mm/context.c`)*. *Rejected:* seL4's ASID pools as capabilities — a cache tag is not authority. +- **Tags are kernel-private**: an ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8) or a PCID + (12 bits, where CPUID reports it); tag 0 reserved. Exhaustion bumps a generation, invalidates all and + reassigns lazily *(Linux `arch/arm64/mm/context.c`)*. *Rejected:* seL4's ASID pools — a tag is not authority. - **Kernel entries are global, user entries never**; without PCID (QEMU's default x86_64 CPU) a `CR3` load flushes the non-global ones. **`map` needs no invalidation**: neither architecture caches a faulting entry (the Arm ARM; SDM Vol. 3A §4.10), though x86_64's paging-structure caches can cause one spurious @@ -318,7 +313,7 @@ checked on 8.2.2), and the stub is already position-independent. With the MMU of 1. Parks secondaries, installs the stack and zeroes `.bss` at physical addresses, as today. 2. `MAIR_EL1`: index 0 = `0xFF` (Normal write-back, read/write-allocate), index 1 = `0x04` (Device-nGnRE). 3. `TCR_EL1`: `T0SZ = T1SZ = 17`; `TG0 = 0b00` and `TG1 = 0b10`, both 4 KiB (the trap); `SH = 0b11`; - `IRGN/ORGN = 0b01`; `IPS` from `PARange` (44 bits here); `AS = 1`; `A1 = 0`; `TBI = 0`; `HA = HD = 0`. + `IRGN/ORGN = 0b01`; `IPS` from `PARange` (44 bits here); `AS = 1`; `A1 = 0`; `TBI0 = TBI1 = 0`; `HA = HD = 0`. 4. Four page-aligned boot tables in `.bss`: `TTBR0_EL1` identity-maps 0x4000_0000 as a 1 GiB Normal block and 0x0000_0000 as a 1 GiB Device block, so the UART survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to 0x4000_0000 as a 1 GiB Normal block. Five descriptors, all `AF = 1`. @@ -451,4 +446,4 @@ Increments 1–10 give RFC-0006 (proposed) address spaces, guarded kernel stacks to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a constraint it inherits rather than discovers: **no syscall takes a pointer argument**. The root task receives all memory as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor; increment 12 closes RFC-0004 -§9.1 in code. Increments 1–3 change nothing about how the kernel runs and can land immediately. +§9.1 in code. Increments 3–7 are host-only and 1–2 change only where the kernel loads and what it prints: all can land now. diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 6a3fef7..f1bd81b 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -44,10 +44,9 @@ availability (A5) lives. seL4 MCS made time a capability, and this RFC adopts th The load-bearing choice, because it decides what a context switch *is*. -- **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). Any kernel path - may block, and switching threads switches kernel stacks. It costs a guarded stack per thread in a - kernel with no heap, and blocking in the kernel is how "hold a lock across a wait a process controls" - (O-7) becomes possible. **Rejected.** +- **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). **Rejected:** a + guarded stack per thread in a kernel with no heap, and blocking in the kernel is how "hold a lock across + a wait a process controls" (O-7) becomes possible. - **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). **Verdict sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with interrupts masked, and leaves by restoring *some* thread's user frame. A syscall that must wait records @@ -78,13 +77,9 @@ queue links are slot **indices**, not pointers. | `priority`, `mcp` | base priority and maximum controlled priority (§6) | | `home_sc`, `sc`, `effective` | the bound context, if any; the context held now, own or donated; the priority run at now (§7) | | `bound_notification`, `ipc_buffer` | the notification a `recv` may also wake on (RFC-0004 §10); the IPC-buffer page bound by capability (RFC-0005, RFC-0007, proposed) | -| `fault_ep`, `timeout_ep` | **handles in the thread's own process table**, resolved when raised — never cached capabilities | +| `fault_ep`, `timeout_ep` | **handles in the thread's own process table**, resolved when raised, so a resolved capability lives only in a table (RFC-0003) and a revoked handler fails closed *(seL4 master; MCS caches them in the TCB)* | | `fp_enabled`, `fp_state`, `class` | the FP/SIMD flag and save area (§10); the SMT placement class (§12) | -Resolving fault handlers at fault time keeps RFC-0003's invariant that a resolved capability lives only -in a table, and makes a revoked handler fail closed. *(The seL4 master kernel looks its fault endpoint up -in the faulting thread's CSpace; MCS installs handler capabilities into the TCB.)* - | State | Meaning | Left by | |-------|---------|---------| | `Inactive` | created or suspended; in no queue | `resume` | @@ -120,18 +115,18 @@ No right joins RFC-0003's set; RFC-0007 (proposed) owns the encoding. nothing to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are **subsumed**: a partition is what a `Core` holder builds from several contexts. **Verdict sought:** contexts as objects *(seL4 MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", -EuroSys 2018)*. A **SchedContext** holds budget `B`, period `T` (`B ≤ T`), at most eight refills, its core, its placement -class, its home thread and its current holder. A thread with a bound context is **active**; one without -is **passive** and runs only on donated time (§7). Its *memory* is charged to a Pool; its *time* comes -from a `Core`. Folding both into one accounting domain (RFC-0005 (proposed)'s open question) is left to -the resource RFC: they are different resources, and Phase 1 needs neither grouped. - -- **`Core`, one per core, minted to the root task** (RFC-0007 (proposed)). `sc_configure` through core - *c*'s `Core` binds the context to *c*: the authority to sell time on *c* and nothing else. `READ` - yields topology — package and SMT siblings, from devicetree `cpu-map` or ACPI PPTT via RFC-0005 - (proposed)'s discovery. No syscall names a core by number. *(seL4 `SchedControl`; research/0002's - "core-set capability".)* `B` below **`MIN_BUDGET`** — twice the measured worst-case kernel path, seL4 - MCS's own rule (`include/kernel/sporadic.h`) — is refused, so nobody configures a timer storm. +EuroSys 2018)*. + +A **SchedContext** holds budget `B`, period `T` (`B ≤ T`), at most eight refills, its core, its placement +class, its home thread and its holder. A thread with a bound context is **active**; one without is +**passive** and runs only on donated time (§7). Its *memory* is charged to a Pool; its *time* comes from +a `Core`. One accounting domain for both (RFC-0005 (proposed)'s open question) is left to the resource RFC. + +- **`Core`, one per core, minted to the root task** (RFC-0007 (proposed)). `sc_configure` through *c*'s + `Core` binds a context to *c*: the authority to sell time on *c* and nothing else. `READ` yields + topology — package and SMT siblings, via RFC-0005 (proposed)'s discovery. No syscall names a core by + number. *(seL4 `SchedControl`; research/0002's "core-set capability".)* `B` below **`MIN_BUDGET`** — + twice the measured worst-case kernel path, seL4 MCS's rule (`include/kernel/sporadic.h`) — is refused. - **Full contexts** (`B = T`) are never throttled and round-robin with timeslice `T`. **Partial contexts** (`B < T`) are sporadic servers: within any window of length `T` the holder consumes at most `B`. Refills fall due a period after consumption; when all eight are in use they merge, forfeiting @@ -139,12 +134,11 @@ the resource RFC: they are different resources, and Phase 1 needs neither groupe - **What a budget means** (research/0002: minima, not caps; burst and redistribution stated). The kernel enforces the **cap**; dispatch is **work-conserving**; the **minimum** follows from admission, the `Core` holder's policy. Unused budget never carries past its window. -- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21, research/0002). - The window is each context's own sliding period. Billing is exact, from the counter (`CNTVCT_EL0`; the - TSC) read at every kernel entry and exit: 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 - 62.5 MHz default (the flag is set in QEMU `target/arm/tcg/cpu64.c` and applied in `target/arm/cpu.c`). - Kernel time, interrupts included, is billed to the context running on entry. The ABI speaks - nanoseconds; budgets round down and periods up, so configured bandwidth is never exceeded. +- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21, research/0002): + the window is each context's own sliding period; billing is exact, from the counter (`CNTVCT_EL0`; the + TSC) read at every entry and exit — 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 62.5 MHz + default (flag set in QEMU `target/arm/tcg/cpu64.c`, applied in `target/arm/cpu.c`). Kernel time, + interrupts included, is billed to the context running on entry. Budgets round down, periods up. - **Stall counters from the first version** (research/0002 Part 6, after PSI and oomd): per context, time consumed, time and count throttled, time ready but waiting, and budget expiries, read-only via `sc_stats` — the O-17 supervisor's wedged-server signal. The kernel measures; userspace acts. @@ -168,11 +162,10 @@ endured (research/0002 Part 1, "Mechanism/policy split"). directly is never enqueued at all. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* - **Release queue.** Per core, `Throttled` threads sorted by next refill: O(n) insertion in the contexts bound to that core, a number only the `Core` holder can raise. -- **Idle is a kernel loop, not a thread**, billing nobody. The subtlety is the lost wake-up. AArch64 runs - `wfi` with `PSTATE.I` set — a pending interrupt still wakes it — then unmasks briefly to take it; Linux - (`arch/arm64/kernel/idle.c`) relies on the same property and warns that masking at `ICC_PMR_EL1` would - *not* wake the core, so this kernel masks with DAIF only. On x86_64 `hlt` with `IF = 0` ignores maskable - interrupts, so the loop is `sti; hlt`, whose STI shadow issues the `hlt` before delivery. +- **Idle is a kernel loop, not a thread**, billing nobody, and the lost wake-up is designed out. AArch64 + runs `wfi` with `PSTATE.I` set — a pending interrupt still wakes it — then unmasks briefly; Linux warns + masking at `ICC_PMR_EL1` would *not* wake the core (`arch/arm64/kernel/idle.c`), so this kernel masks + with DAIF only. x86_64 uses `sti; hlt`, whose STI shadow issues the `hlt` before delivery. - **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has released budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority is at least the highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also @@ -207,10 +200,10 @@ thresholds on endpoints for SC Donation", proposed 2023-08-15; seL4/rfcs pull re thresholded endpoint is invalid. With `threshold` at the server's WCET, expiry inside it is a true error. - **A flagged deviation.** The amendment says donation below the threshold is "refused up front". RFC-14, which it cites, refuses only a context that can *never* pass; one that cannot pass *now* has its refills - merged and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in - an event kernel, O(refills) ≤ 8. This RFC proposes RFC-14's behaviour: no donation happens below the - threshold, so the amendment's purpose holds, whereas refusing a transient shortfall would make every - client spin on its own budget to retry. If accepted, it is logged against RFC-0004 as a dated amendment. + merged and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in an + event kernel, O(refills) ≤ 8. Proposed: RFC-14's behaviour. No donation happens below the threshold, so + the amendment's purpose holds; refusing a transient shortfall would make every client spin to retry. If + accepted, it is logged against RFC-0004 as a dated amendment. - **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* runs on it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with `BudgetExpired` (observed when the context next refills); *R* is consumed, so a late `reply` fails @@ -220,12 +213,10 @@ thresholds on endpoints for SC Donation", proposed 2023-08-15; seL4/rfcs pull re context to finish. With no handler, *S* stays stranded, contained to its own clients. *(seL4 MCS: `tcbTimeoutHandler`, `seL4_Fault_Timeout`.)* - **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply - to A with an error. Unwinding to A would walk the reply stack faulting every level — the O(n) walk - RFC-14 also rejects. This kernel-initiated completion is the one the amendment left to this RFC: a - status on the reply object, no new object. + to A with an error; unwinding to A would walk the reply stack — the O(n) walk RFC-14 also rejects. This + kernel-initiated completion is the one the amendment left to this RFC: a status, no new object. - **A reconciliation.** The amendment says seL4's timeout exceptions "have remained future work"; they - ship in MCS, and RFC-14 calls them the kernel's current mechanism — what is unmerged is RFC-14's - threshold. The phrase, from research/0002, wants a dated correction; the commitment is unchanged. + ship in MCS, and what is unmerged is RFC-14's threshold. The phrase wants a dated correction only. **Rejected: the unwind without timeout faults**, where the stranded server is visible only through its state and counters. Timeout faults are no second protocol — they travel RFC-0007 (proposed)'s fault path @@ -277,11 +268,7 @@ al., EuroSys 2019; research/0002 Part 6). | Also per switch | `CPACR_EL1.FPEN` (§10) | `CR0.TS` (§10); `TSS.RSP0` | | Lazily (§10) | `V0`–`V31`, `FPCR`, `FPSR`: 520 bytes, 16-byte aligned | XSAVE area, `XCR0` = x87, SSE, AVX: 832 bytes, 64-byte aligned | | Never | kernel registers, kernel FP (soft-float), debug and PMU state | the same | - -**The TCB's real size.** AArch64: a 36-word frame (288 bytes), 528 bytes of FP area and about 150 bytes -of scheduling and IPC fields — about 970 bytes, allocated as **1 KiB**. x86_64: a 23-word frame (184 -bytes, padded to 192), the 832-byte XSAVE area and the same fields — about 1,180 bytes, allocated as -**1.25 KiB**. +| **TCB size** | 36-word frame (288 B) + FP area padded to 528 B + ~150 B of fields: **1 KiB** | 23-word frame (184 B, padded to 192) + 832 B XSAVE + ~150 B: **1.25 KiB** | - **O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` (EL0t, DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only @@ -291,10 +278,8 @@ bytes, padded to 192), the 832-byte XSAVE area and the same fields — about 1,1 ## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (V7) -Both kernel targets are soft-float (CLAUDE.md § Architectures), so FP/SIMD state is purely user state. - -- **Enabled per thread** by `set_fp`, off by default. A thread with FP off that touches FP takes an - ordinary fault, so IPC-heavy servers never own FP state and never force a save. +- **Enabled per thread** by `set_fp`, off by default; the kernel is soft-float, so this state is purely + user state. A thread with FP off that touches FP takes an ordinary fault, so servers never force a save. - **Loaded on first use, saved lazily.** Each core records an FP *owner*; every other thread runs with access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or `CR0.TS = 1` (`#NM`, vector 7). On that trap, for an FP-enabled thread, the kernel saves the owner's registers, @@ -326,10 +311,9 @@ and QNX userspace drivers.)* and `GICD_IROUTER`; SGIs and PPIs through the redistributor's `GICR_ISENABLER0`, woken by `GICR_WAKER`. Each line the kernel enables is put in Group 1 explicitly, since Group 0 would arrive as FIQ. On `virt`: distributor `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`). - Chosen for affinity routing past eight cores and an interface needing no MMIO mapping. *(Arm IHI 0069.)* - **Cost:** `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), so xtask gains - `gic-version=3`; and the Raspberry Pi 4 in Constitution §6's bring-up order has a GIC-400 (GICv2), so a - second driver behind the same HAL is a **known** debt, not a maybe. + Chosen for affinity routing past eight cores and an interface needing no MMIO. *(Arm IHI 0069.)* **Cost:** + `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), so xtask gains `gic-version=3`; + the Raspberry Pi 4 in Constitution §6's bring-up order has a GIC-400 (GICv2) — a **known** second driver. - **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask bit; end of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** without IOMMU interrupt remapping a device can aim any vector at any core — O-18's precondition again. @@ -340,9 +324,8 @@ and QNX userspace drivers.)* Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. -- **Everything scheduling-related is per core**: run and release queues, kernel stack, timer, FP owner, - current thread. A thread runs on the core of the context it holds; moving a context is `sc_configure` - through another `Core`. **The kernel never load-balances.** A passive server runs where its time is. +- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds; moving + a context is `sc_configure` through another `Core`. **The kernel never load-balances.** - **Cross-core wake-ups** (RFC-0004 §9.2): enqueue *W* on its core and send an IPI — an SGI through `ICC_SGI1R_EL1`; a fixed vector through the LAPIC ICR. Direct switch is same-core only. - **One big kernel lock when SMP lands**, taken on entry *(Peters, Danis, Elphinstone & Heiser, "For a @@ -358,9 +341,8 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. no gang scheduling. SMT-off is "never fund the sibling"; one class is the vacuous default. - **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one thread, a full context, nothing else on the core. The bitmap has one bit, the timer is never armed, no - IPI arrives because nothing is queued there, and device lines are routed elsewhere with the `Core`. No - flag is needed: only a `Core` holder can bind another context, and a holder promising exclusivity to a - third party transfers the capability without keeping a copy. + IPI arrives, and device lines are routed elsewhere. No flag: only a `Core` holder can bind another + context, and one promising exclusivity to a third party transfers the capability, keeping no copy. ## 13. Lineage @@ -374,8 +356,7 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. | **Ford & Lepreau 1994** | the migrating-thread picture of `call` | Mach | | **Sprunt, Sha & Lehoczky 1989** | the sporadic server | an unbounded replenishment list | | **Jailhouse** | refusing to share: parked cores, exclusive assignment | static-only partitioning | -| **Peters et al. 2015** | one big lock | fine-grained locking before measurement | -| **Linux** | the idle wake-up rule; PSI-style stall counters | dynamic priorities | +| **Peters et al. 2015; Linux** | one big lock; the idle wake-up rule; PSI-style stall counters | fine-grained locking before measurement; dynamic priorities | ## 14. Obligations @@ -400,7 +381,6 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. ## 16. Costs — what this makes harder -- **Every kernel path must be bounded and restartable**, and interrupt latency is the worst of them. - **Every server needs someone to think about time.** A passive server cannot run until called; an active one needs a configured context. "Spawn a thread and it runs" is gone — O-4's price, in time. - **The one-hop unwind leaves a stranded server to userspace.** The kernel makes the failure visible and @@ -421,24 +401,23 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. EL0 timers, the physical counter and the PMU stay off regardless. 2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §16)? 3. **RFC-14 budget limits.** Worth 21–22% on the fastpath to stop a passive server overspending? -4. **Passive servers on notifications.** MCS binds a context to a notification; passive drivers need it. -5. **Userspace timers** (RFC-0004 §9.3): kernel deadline objects off the release queue, or a time server? -6. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst kernel path; x86_64's clock +4. **Passive drivers** need a context bound to a notification (MCS); **userspace timers** (RFC-0004 §9.3) + need kernel deadline objects off the release queue, or a time server. Neither is Phase 1. +5. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst kernel path; x86_64's clock when the TSC is not invariant. ## 18. Implementation increments Pure logic lands in a new host-tested `no_std` workspace crate, `scheduler/`, generic over a thread index -as `capability/` is over `ObjectRef`, built for both Tier-1 targets in CI. Hardware behaviour is proved by -QEMU boot-test strings. Every new `unsafe` block lives in `kernel/src/arch/**`; no new designated tree. -The x86_64 HAL gains each signature as a documented no-op that returns — its entry still calls straight -through into the kernel proper (CLAUDE.md § Architectures), so the second build compiles the scheduler. -**[demo]** marks what the Phase-1 demo needs. +as `capability/` is over `ObjectRef`; hardware behaviour is proved by QEMU boot-test strings. New `unsafe` +stays in `kernel/src/arch/**`. The x86_64 HAL gains each signature as a documented no-op that returns — +its entry still calls straight through into the kernel proper (CLAUDE.md § Architectures), so the second +build compiles the scheduler. **[demo]** marks what the Phase-1 demo needs. 1. **`scheduler` I — run queues, states, MCP. [demo]** O(1) pick, head/tail re-entry, legal transitions, - the MCP relation. Host unit tests and an exhaustive transition table. -2. **`scheduler` II — full contexts and billing. [demo]** Round-robin budgets, tick rounding, stall - counters, `MIN_BUDGET`. Host tests against a shadow model. + the MCP relation; host tests and an exhaustive transition table. Built `no_std` for both targets in CI. +2. **`scheduler` II — full contexts and billing. [demo]** Round-robin budgets, rounding, stall counters, + `MIN_BUDGET`; host tests against a shadow model. 3. **GICv3 and the virtual timer, kernel-only. [demo]** xtask's machine line gains `gic-version=3`; the kernel's own handler prints `timer: tick` twice — the GIC, the PPI and the idle loop, proved. 4. **Event-kernel dispatch. [demo]** Trap frame in the TCB; two kernel self-test threads (EL1t, on @@ -463,8 +442,7 @@ through into the kernel proper (CLAUDE.md § Architectures), so the second build and each finds its own: `fp: isolated`. 12. **`IrqControl`/`IrqHandler`.** PL011 receive (SPI 1, INTID 33) reaches an EL0 driver as a notification when xtask writes a byte to the serial port: `irq: notified`. -13. **x86_64 scheduling HAL** — IDT, TSS, IST, LAPIC, `#NM`, `XSAVE` — after its boot RFC; the strings of - 3–5 and 11 again, proving the HAL held. +13. **x86_64 scheduling HAL** (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`) after its boot RFC: 3–5 and 11's strings. 14. **SMP** — secondary cores from the park loop, the big lock, IPIs, placement classes. **What the demo takes as interim, honestly.** The demo needs increments 1–7 and runs full contexts only. From 79d8583b5c26b6fa0477da40478f9b655d765fe2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:07:19 +0000 Subject: [PATCH 03/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 68 +-- ...0007-system-calls-and-process-bootstrap.md | 565 ++++++++++++++++++ 2 files changed, 595 insertions(+), 38 deletions(-) create mode 100644 docs/rfcs/0007-system-calls-and-process-bootstrap.md diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index f1bd81b..bdd5b75 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -50,19 +50,16 @@ The load-bearing choice, because it decides what a context switch *is*. - **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). **Verdict sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with interrupts masked, and leaves by restoring *some* thread's user frame. A syscall that must wait records - a state and the object waited on, and the kernel exits to someone else. On wake, the result is written - into the waiter's frame, or the call restarts by rewinding the saved PC one instruction (4 bytes for - `svc #0`, 2 for `syscall`). *(Heiser & Elphinstone, "L4 Microkernels: The Lessons from 20 Years of - Research and Deployment", TOCS 2016; Ford et al., "Interface and Execution Models in the Fluke Kernel", - OSDI 1999.)* - -**The cost:** every kernel path must be bounded. The long ones — object destruction and RFC-0005 -(proposed)'s `reissue` and address-space teardown — take **preemption points**: poll for a pending -interrupt (`ISR_EL1.I`; the LAPIC IRR) and, if one is pending, leave the operation restartable and exit. -The worst path is measured and published, because it is the interrupt latency. This **reverses -`aarch64.ld`'s plan** that the boot stack "will be replaced by per-thread stacks": it becomes core 0's -kernel stack, moved into RFC-0005 (proposed)'s guarded stack window and resized from a measured -high-water mark. + a state and the object waited on; on wake, the result is written into the waiter's frame, or the call + restarts by rewinding the saved PC one instruction (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & + Elphinstone, "L4 Microkernels: The Lessons from 20 Years of Research and Deployment", TOCS 2016; Ford + et al., "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* + +**The cost:** every kernel path must be bounded. The long ones — object destruction, RFC-0005 +(proposed)'s `reissue` and address-space teardown — take **preemption points**: on a pending interrupt +(`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. The worst path is measured and +published: it is the interrupt latency. This **reverses `aarch64.ld`'s plan** that the boot stack "will be +replaced by per-thread stacks": it becomes core 0's kernel stack, in RFC-0005 (proposed)'s guarded window. ## 4. The Thread object (V2) @@ -249,17 +246,15 @@ and the next refill due. With no competitor (§12's exclusive core) the timer is ## 9. The context switch — what is saved, where -In an event kernel a context switch is *which frame the exit path restores*: no kernel stack changes -hands. Assembly is confined to the entry and exit stubs in `kernel/src/arch/**`. **The user frame lives -in the TCB.** On AArch64, while a thread runs at EL0, `SP_EL1` points at the end of its TCB frame, so -the entry stub's stores land there; only then does it load the per-core kernel stack from the block -`TPIDR_EL1` addresses. On x86_64 an interrupt or exception from ring 3 loads `RSP` from `TSS.RSP0`, -which points at the TCB frame, so the CPU's pushes land there — but **`syscall` does not switch `RSP`**: -its stub must `swapgs`, park the user `RSP` and load the frame pointer itself to reach the same layout. -RFC-0007 (proposed) owns both entry sequences and `sysretq`'s canonical-`RIP` check; this RFC owns the -layout. The address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s call, -made on exit when the process differs — the single point where a time-protection flush would go (Ge et -al., EuroSys 2019; research/0002 Part 6). +In an event kernel a context switch is *which frame the exit path restores*; assembly is confined to the +entry and exit stubs in `kernel/src/arch/**`. **The user frame lives in the TCB.** On AArch64, while a +thread runs at EL0, `SP_EL1` points at the end of its TCB frame, so the entry stub's stores land there +before it loads the per-core kernel stack from the block `TPIDR_EL1` addresses. On x86_64 an interrupt or +exception from ring 3 loads `RSP` from `TSS.RSP0`, pointing at the TCB frame — but **`syscall` does not +switch `RSP`**: its stub must `swapgs`, park the user `RSP` and load the frame pointer itself. RFC-0007 +(proposed) owns both entry sequences; this RFC owns the layout. The address-space switch (`TTBR0_EL1` with +ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit when the process differs — the single point +where a time-protection flush would go (Ge et al., EuroSys 2019; research/0002 Part 6). | What | AArch64 | x86_64 | |------|---------|--------| @@ -395,10 +390,9 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. ## 17. Open questions -1. **EL0 counter access.** Proposed default: a per-thread flag, off unless set with `WRITE` on the Thread, - switched with the thread (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`, written only when it differs). Denial costs - a web server a kernel entry per timestamp, and threat model §7 puts fine-grained timing out of scope. - EL0 timers, the physical counter and the PMU stay off regardless. +1. **EL0 counter access.** Proposed default: a per-thread flag set with `WRITE` on the Thread and switched + with it (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`, written only when it differs). Denial costs a web server a + kernel entry per timestamp; threat model §7 puts fine-grained timing out of scope. EL0 timers stay off. 2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §16)? 3. **RFC-14 budget limits.** Worth 21–22% on the fastpath to stop a passive server overspending? 4. **Passive drivers** need a context bound to a notification (MCS); **userspace timers** (RFC-0004 §9.3) @@ -423,21 +417,19 @@ build compiles the scheduler. **[demo]** marks what the Phase-1 demo needs. 4. **Event-kernel dispatch. [demo]** Trap frame in the TCB; two kernel self-test threads (EL1t, on `SP_EL0`, behind a `sched-selftest` feature as `provoke-exception` is) alternate under the timer: `sched: A B A B`. The boot stack becomes core 0's kernel stack; `aarch64.ld`'s comment changes with it. -5. **EL0 entry and exit** (with RFC-0005 (proposed)'s user mappings and RFC-0007 (proposed)'s trap stub). - **[demo]** A spinning EL0 thread is preempted at arbitrary instructions and resumes with a register - pattern intact: `user: still here`. +5. **EL0 entry and exit** (on RFC-0005's user mappings, RFC-0007's trap stub, both proposed). **[demo]** A + spinning EL0 thread is preempted anywhere and resumes with its registers intact: `user: still here`. 6. **Thread, SchedContext and `Core` objects; `sc_configure`, `bind_sc`, `resume`, `set_priority`, `exit`. [demo]** The root task configures the demo processes; two EL0 threads at equal priority interleave. -7. **IPC blocking states, FIFO endpoint queues, the direct switch** (with RFC-0004's endpoint PR). **[demo]** - Host tests for the switch decision; boot-test: the client `call`s an **active** server and RFC-0007 - (proposed)'s `Kaya!` strings appear. +7. **IPC blocking states, FIFO endpoint queues, direct switch** (with RFC-0004's endpoint PR). **[demo]** + Host tests for the switch; the client `call`s an **active** server and RFC-0007's `Kaya!` strings appear. 8. **Passive servers, donation, inheritance.** A passive server runs only when called, at its client's - priority. A demo stretch goal if 1–7 land early. + priority; a demo stretch goal. 9. **`scheduler` III — partial contexts.** Sporadic refills, merge, release queue. Property test: no window of length `T` sees more than `B` consumed. -10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test: `BudgetBelowThreshold` - for an under-budgeted client, `BudgetExpired` and a timeout fault for an overrunning server. Must - land before any passive server on a partial context is trusted. +10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test `BudgetBelowThreshold`, + then `BudgetExpired` and a timeout fault for an overrunning server. Required before partial-context + passive servers are trusted. 11. **Lazy FP/SIMD.** Two FP-enabled EL0 threads load different markers into `v0`, are preempted in turn and each finds its own: `fp: isolated`. 12. **`IrqControl`/`IrqHandler`.** PL011 receive (SPI 1, INTID 33) reaches an EL0 driver as a notification diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md new file mode 100644 index 0000000..696647f --- /dev/null +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -0,0 +1,565 @@ + + +# RFC-0007 — System calls and process bootstrap + +| Field | Value | +|-------|-------| +| Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | +| Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | +| Date | 2026-09-27 | +| Affects | Constitution §3 (capabilities are the only authority), pillar 2; `CLAUDE.md` § `unsafe` policy and § Layout; the workspace (`user/`); `xtask` and `kernel/build.rs`; the kernel's trap path; the accepted `capability` crate (the handle word, a slot-generation bound, one error variant split) | +| Depends on | RFC-0003 (accepted; §9 binds every syscall); RFC-0004 (accepted; §5, §6, §8, §9.4, §10 and the 2026-08-01 amendment); RFC-0005 and RFC-0006 (proposed, drafted alongside) | +| Discharges | O-5 (argument validation) for the syscall surface; O-27 (explicit authority at spawn); binds O-1, O-2 and O-4 at B1; contributes to O-7, O-9 and O-13 | + +> **Proposed verdicts.** +> +> 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§5). +> 2. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§4). +> 3. **Messages:** four physical and sixty-four virtual message registers, three capabilities; a 1 KiB IPC buffer, every word fetched once (§6). +> 4. **IPC buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§6). +> 5. **Handle word:** 20-bit index, 44-bit *slot* generation, `0` is null; the crate retires slots at 2^44 − 1, objects keep 64 bits (§7). +> 6. **Errors in two layers:** transport status in a register, method result in the reply label; `BudgetRefused` beside `BudgetExpired`; faults are not errors (§8). +> 7. **Process object:** table + address space + threads, born empty, grant window shut by the first `resume` of any of its threads (§9). +> 8. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§10). +> 9. **Boot images:** `xtask` parses ELF on the host and embeds a bundle of fixed-shape descriptors in the kernel ELF; the kernel never parses ELF (§11). +> 10. **Console as a capability:** a kernel object with one `WRITE` method, scheduled for deletion when the Phase-2 UART driver lands (§12). +> 11. **`user/abi`** is the microkernel-core row, not libc/runtime; its `src/arch/**` is the third designated `unsafe` tree (§13). + +## 1. The question + +**How does userspace ask the kernel to do anything, and how does the first userspace come to exist — +such that every request names a capability, no argument is trusted, and no process is born holding +authority it was not explicitly handed?** + +RFC-0003 §9 fixed the rule and RFC-0004 §10 the IPC semantics; both left the encoding here. This RFC +designs the encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 +named: minting the first capabilities without an ambient grantor. It does not design address spaces +(RFC-0005), threads or scheduling (RFC-0006), or any policy about which programs run holding what — +that is the root task's, in userspace, and later the broker's. + +## 2. Which pillar + +**Pillar 2 and the kernel doctrine that capabilities are the only authority.** The syscall surface is the +whole of B1; one entry that names a resource without a handle makes RFC-0003's table decorative. Spawn +is where research/0002 found the freshest scar (Part 4: runc CVE-2024-21626, one inherited descriptor +became host traversal), and O-27 exists because of it. Pillar 1 is served in passing: boot images reach +userspace as read-only regions, immutable from their first instruction. + +## 3. What already binds this RFC + +- **RFC-0003 §9:** resources only by handle; authority-free are yielding, halting oneself, self-queries. +- **RFC-0004 §5–§10 and its 2026-08-01 amendment:** virtual message registers; one bounded copy; + cap-carrying messages off the fast path; all-or-nothing transfer; the IPC semantics; a minimum budget + refused up front at `call`, and an error completion when a budget expires inside a server. +- **`CLAUDE.md`:** `unsafe` only in designated trees; no kernel heap; both targets soft-float. + +## 4. Entering the kernel + +**AArch64 — `svc #0`.** From EL0, `svc` takes a synchronous exception to `VBAR_EL1 + 0x400` (lower EL, +AArch64 — entry 8 of the table `vectors.s` already installs) with `ESR_EL1.EC = 0x15`, the immediate in +`ESR_EL1.ISS[15:0]`, and `ELR_EL1` already past the `svc`; entry masks D, A, I and F and selects `SP_EL1` +*(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's +frame (RFC-0006 (proposed) provides its layout, in the TCB) and dispatches on EC: `0x15` to the syscall +dispatcher, anything else is a *fault* (§8). A non-zero immediate answers `InvalidArgument`, so a later +ABI revision is detectable rather than silently accepted. Return is `eret` with an `SPSR_EL1` the kernel +builds: `M` = EL0t, DAIF clear, NZCV from the frame; where implemented, `DIT` (Armv8.4) and `SSBS` +(Armv8.5) — which EL0 can already set with `msr` — are carried too; every other bit (`SS`, `IL`, `PAN`, +`UAO`, `TCO`, `BTYPE`) is zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. +*(Linux arm64 and seL4 both enter through `svc #0`.)* + +**x86_64 — `syscall` / `sysretq`.** `syscall` requires `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, +leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `RFLAGS` bits named in `IA32_FMASK` +and takes `CS`/`SS` from `IA32_STAR` — but does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System +Calls in 64-Bit Mode")*. The stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads +the kernel stack; the untrusted `rsp` is never dereferenced. `FMASK` clears IF, DF, TF and AC: interrupts +off, string direction sane, and — where SMAP exists — user access closed. Two hazards are designed in: + +- **Non-canonical return.** On Intel parts `sysretq` with a non-canonical `rcx` raises #GP *in ring 0* + on the user's stack — CVE-2012-0217, which took Xen, FreeBSD, NetBSD and Windows. RFC-0005 (proposed) + leaves the top user page unmapped and RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`; the exit + path checks `rcx` anyway and falls back to `iretq`, because the check is one compare. +- **The window before the stack switch.** An NMI or machine check there would run on a user-chosen + `rsp`; RFC-0006 (proposed) provides IST stacks for the exceptions that can land in it. + +`sysretq` also fixes the GDT order — user data at `STAR[63:48] + 8`, user code at `+ 16`. `sysenter` is +rejected: AMD does not implement it in long mode; so is an `int` gate, which costs a full IDT delivery. + +| Role | AArch64 | x86_64 | +|------|---------|--------| +| syscall number (in) | `x8` | `rax` | +| invoked handle (in) / status (out) | `x0` | `rdi` in, `rax` out | +| message tag (in/out) | `x1` | `rsi` | +| MR0–MR3 (in/out) | `x2`–`x5` | `rdx`, `r10`, `r8`, `r9` | +| second handle: reply object (in) / badge (out) | `x6` | `r12` | +| error detail (out) | `x7` | `r13` | +| clobbered by hardware | — | `rcx`, `r11` | + +Every register outside the result set is restored from the frame; the eight result registers (`x0`–`x7`; +`rax`, `rsi`, `rdx`, `r10`, `r8`, `r9`, `r12`, `r13`) carry a result or zero, so no kernel value ever +leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux, leaving `x0`–`x7`, the AAPCS64 argument +registers, to the message. The x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot +name (LLVM's base and frame pointers), and `rcx`/`r11`, which `syscall` destroys. seL4 has the same +shape in different seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. + +**Entry state of every thread,** root task included, on both architectures: PC = entry; SP as given; +`a0` and `a1` — `x0`/`x1`, `rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — +the two start values; every other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 +with `RFLAGS` = `0x202` (IF and the always-one bit 1); FP/SIMD trapped (`CPACR_EL1.FPEN` = 0, `CR0.TS` +set) until RFC-0006 (proposed) enables it for the thread; `TPIDR_EL0` and `FS_BASE` zero; the IPC-buffer +register set (§6). A Thread is created in exactly this state; the spawner writes PC, SP, `a0` and `a1` +through RFC-0006 (proposed)'s `write_regs`, and the kernel writes the root's (§10). + +**The kernel never touches user memory through an address userspace supplies.** No syscall argument is +a pointer; the only user memory the syscall path touches is the IPC buffer, reached by frame (§6). On the +demo CPU this discipline is the whole defence: the Cortex-A72 is Armv8.0 and lacks PAN (Armv8.1). + +## 5. The syscall set + +- **A — one syscall per operation** *(Zircon, Linux)*. Legible per call, but the table grows with every + object type, each entry is a new path to audit, and no server can ever stand in for a kernel object. +- **B — uniform invocation** *(seL4; KeyKOS before it)*. A handful of IPC syscalls; an operation on a + kernel object is a `call` on its capability with the method in the tag's label, answered by the kernel + as a server would answer. Kernel objects and servers are invoked identically. +- **C — B plus a separate `invoke` number.** Keeps the small table and loses the point: the caller can + again tell a kernel object from a server. + +**Verdict sought: B.** The payoff is interposition. A Console capability and an endpoint to a logging +server are invoked alike, so Phase 2's userspace UART driver takes over the console protocol without a +client changing, and the broker can hand out a forwarder in place of any kernel object — RFC-0003a option +(a)'s precondition. A `call` on a kernel object runs on the caller's own time: nothing is donated, +nothing blocks, no reply object is involved. + +| # | Syscall | Handles | Beyond the tag and message registers | +|---|---------|---------|--------------------------------------| +| 0 | `yield` | — | authority-free (RFC-0003 §9) | +| 1 | `send` | object | endpoint: blocking send; notification: signal (RFC-0004's `notify`) | +| 2 | `nb_send` | object | as `send`, failing `WouldBlock` rather than blocking | +| 3 | `call` | object | endpoint: send, await reply, donating (RFC-0004 §8); kernel object: invoke a method | +| 4 | `recv` | endpoint or notification; reply object | blocking receive; a bound notification may wake it | +| 5 | `nb_recv` | as `recv` | poll | +| 6 | `reply` | reply object | reply once to a caller | +| 7 | `reply_recv` | endpoint; reply object | `reply` then `recv` in one entry — the server loop *(seL4 `ReplyRecv`)* | +| 8 | `cap_derive` | object, with `DUPLICATE` | in: MR0 the rights wanted, MR1 a badge (reserved, zero); out: MR0 the new handle word | +| 9 | `cap_close` | object | nothing; the handle is dead on return | +| 10 | `cap_identify` | object | out: MR0 the object kind (an `abi` enumeration), MR1 the rights bits | +| 11 | `thread_exit` | — | halt the calling thread; authority-free | + +`reply_recv` fuses two RFC-0004 operations so a server loop is one kernel entry, which the direct-switch +fast path needs; it adds no semantics. `notify` is `send` on a notification, since RFC-0004 left "one +syscall with a mode, or several" here; a signal never blocks, so `nb_send` signals too. The `cap_*` calls +take a zero tag, return a zero tag, and act only on the caller's own table, gated by the handle they name, +so §9 needs no table capability. A rights word with an undefined bit answers `InvalidArgument`; the +dispatcher assembles `Rights` from the crate's named constants, so the crate still never accepts raw +bits. Transfer in a message is always a move; a sender that keeps a copy runs `cap_derive` first +(RFC-0003 §6). **Creation** is a method on a memory capability — RFC-0005 (proposed) provides the Pool +and its creation methods — and the per-type parameters are encoded here: an endpoint's creation carries +**`min_budget`** in MR1, in RFC-0006 (proposed)'s time unit, 0 for none, fixed for the endpoint's life. +**Rejected:** a `debug_putc` (§12); anything taking a PID, path or address; seL4's depth-addressed +capability pointers, which exist for CNode trees this kernel does not have. + +## 6. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) + +The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities attached (0–3); bit 9 +set by the kernel when a `recv` was woken by its bound notification (MR0 then holds the pending bits), +zero on input; bits 10–15 reserved, zero; bits 16–63 a 48-bit label — user-defined for endpoints, the +method number for kernel objects, the method result on a kernel reply. *(seL4's `seL4_MessageInfo`: +label, extra capabilities, length.)* + +**Four physical message registers on both architectures.** seL4 uses four on both 64-bit targets; +research/0001 found the in-register payoff small and shrinking (10% on ARM11, negative on modern x86); +and a count that differed by architecture would make one message fast on one Tier-1 target and slow on +the other. Messages up to **64 words** (512 bytes) spill MR4–MR63 to the IPC buffer — below seL4's 120 on +purpose: 512 bytes bounds the non-preemptible copy, and RFC-0004 §5 sends bulk data through shared +regions. **Three capabilities per message**, seL4's 2-bit field, so the field has no invalid encodings. + +```rust +/// 1 KiB and 1 KiB-aligned, so it never straddles a page. +#[repr(C, align(1024))] +pub struct IpcBuffer { + pub self_ptr: u64, // this buffer's user address, written by the kernel at binding + _reserved: [u64; 3], + pub msg: [u64; 64], // MR0..MR63; msg[0..4] never read, those travel in registers + pub caps: [u64; 3], // handle words sent or received, `tag.caps` of them + _tail: [u64; 57], // neither read nor written by the kernel +} +``` + +**The buffer is kernel-owned memory, reached by frame.** A thread's buffer is a 1 KiB-aligned slice of +one page of a Region, bound when the thread is created by the Region capability (`READ | WRITE`), the +offset, and the user address where the spawner mapped it; RFC-0006 (proposed) records the binding and +RFC-0005 (proposed) provides the Region and the kernel's physmap. The kernel reads and writes the buffer +only through its own mapping of that frame. The user address is used for two things — word 0 and the +discovery register — and is never dereferenced. Received capabilities land in fresh slots of the +receiver's table (RFC-0004 §6), so there is no seL4-style receive window. + +**Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's user address into +`TPIDRRO_EL0` (readable at EL0, writable only at EL1) and into the user GS base on x86_64, where +`CR4.FSGSBASE` stays off (RFC-0006 (proposed)) and `abi` reads the self-pointer at `gs:[0]` — the x86_64 +ELF TLS convention of a thread block whose first word holds its own address *(Drepper, "ELF Handling For +Thread-Local Storage")*. RFC-0006 (proposed) already restores both registers on every switch and leaves +their use here, so `_start` has its buffer at its first instruction, in any thread, with no TLS runtime. +The kernel trusts neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** +`TPIDR_EL0` and `FS_BASE` — the thread pointers of the AArch64 and x86_64 ELF TLS ABIs, which a relibc +port needs untouched; and seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 +`functions.h`) — sound, but it needs TLS before the first syscall. + +**Single fetch (O-5).** Once SMP exists, another thread of the same process can rewrite the buffer +mid-syscall. So every buffer word the kernel acts on is read **exactly once** into kernel memory before +any check depends on it; lengths and capability counts come from the tag register, already in the saved +frame, bounded by 64 and 3; the `caps[]` words are copied into a kernel array and every one is resolved +and checked for `TRANSFER` before any moves — all or nothing (RFC-0004 §6); payload words are copied +frame to frame by RFC-0005 (proposed)'s volatile-copy module and never interpreted, and no Rust reference +is ever formed into the buffer. *(research/0002 Part 5: virtio's double-fetch lesson — "validate a +copied-out snapshot; never re-read after validation".)* **Rejected:** six physical registers on AArch64 +(portability); seL4's 120-word maximum (a longer non-preemptible copy for payloads RFC-0004 routes +elsewhere). + +## 7. Handles in one register + +RFC-0003's `Handle` is a `u32` index and a 64-bit generation — ninety-six bits, which fit no register. +The ABI word is **`slot_generation << 20 | index`**: 2^20 slots per process, Linux's default +per-process descriptor ceiling (`fs.nr_open` = 1,048,576); and 2^44 slot generations, about two hundred +days of one close-and-reopen per microsecond on a single slot before it retires. `Generation::FIRST` is +1, so word `0` is never live and serves as null. This is the width RFC-0003's 2026-07-30 amendment asked +to be stated. **Only the slot generation crosses the ABI;** a capability's minted *object* generation +stays in kernel memory. So the accepted crate changes narrowly: `Handle::to_word`/`from_word` and a +constant `Handle::MAX_SLOT_GENERATION` (2^44 − 1); `CapabilityTable::remove` retires a slot whose next +generation would exceed it, through the `Retired` state the table already has, so a generation the ABI +cannot carry retires its slot rather than aliasing; a const assertion keeps every kernel table's `N` +≤ 2^20. `Generation` itself is untouched — objects keep all 64 bits — and its doc comment, which +promises 64 bits, gains the slot bound. Phase 1 tables have `N` = 64, fixed, failing closed with +`TableFull`. **Rejected:** two registers per handle (halves the message budget); 32/32 (2^32 reuses is +seventy minutes at one per microsecond, and the LIFO free list concentrates reuse on one slot); a +16-bit index (65,536 capabilities is too few for the server workloads the constitution targets). + +## 8. The error model + +**Transport status** (`x0`/`rax`) is what the syscall layer found, identical whether the target is a +kernel object or a server. Every failure is atomic — nothing moved, nothing half-sent — and the detail +register names the culprit: 0 the invoked handle, 1–3 an attached capability, 4 the tag, 5 the reply +handle. + +| Status | Meaning | +|--------|---------| +| `Ok` | done | +| `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total and needs no fault-handler policy | +| `InvalidArgument` | a reserved bit, a length over 64, a non-zero `svc` immediate, an undefined rights bit | +| `InvalidHandle` | word 0, index out of range, empty slot, or stale slot generation — the handle names nothing *of yours*: the crate's `OutOfBounds`, `Empty`, `StaleGeneration` | +| `Revoked` | the slot is live but its object was destroyed — the crate's new `ObjectDestroyed`; close the handle | +| `WrongType` | the handle is live but names the wrong kind of object for this syscall | +| `InsufficientRights` | the capability lacks the right the operation needs | +| `TableFull` | a moved capability found no slot in the receiving table; nothing moved | +| `WouldBlock` | `nb_send` or `nb_recv` found no partner | +| `PeerGone` | the object or party waited on was destroyed mid-wait | +| `BudgetRefused` | `call` whose donated context can never meet the endpoint's `min_budget`, refused up front (RFC-0004's amendment; seL4 RFC-14) | +| `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object (RFC-0004's amendment) | + +`Revoked` needs the second crate change. Today `StaleGeneration` covers both a reused slot — the +holder's own use-after-close — and a destroyed object; `resolve` already checks them in that order, so +the split is one new variant and one changed return. It lets a holder observe and survive destruction, +as research/0002 asks of revocation, without reporting its own use-after-close as something done to it. +When `BudgetRefused` and `BudgetExpired` arise is RFC-0006 (proposed)'s to define; its draft calls the +first `BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. + +**Method result** (the reply tag's label) is what the object said, written by the kernel for kernel +objects and by the server for endpoints, in one vocabulary — so a server standing in for a kernel object +reports errors the same way *(seL4 returns invocation errors in the reply's label)*. It reuses the status +numbering where meanings coincide and adds per-object results: `InvalidMethod`, RFC-0005 (proposed)'s +`PoolExhausted` and `Fragmented`, and §9's `WindowClosed`. + +**Faults** are EL0 exceptions that are not syscalls. They are not return values — the thread cannot +continue. The design suspends it and sends a fault message to a handler RFC-0006 (proposed) binds to the +thread, so a supervisor can restart a driver (O-17). **Interim:** until then the existing reporter prints +the fault, prefixed by the thread, and the thread stops; the core continues, and nothing restarts it. + +## 9. The Process object and spawning (O-27) + +A **Process** holds one RFC-0003 capability table, a reference to one AddressSpace (RFC-0005 (proposed)) +and the threads bound to it (RFC-0006 (proposed)). The per-process table makes the process a real object +*(Zircon)* rather than seL4's convention of a thread pointing at a CSpace and a VSpace. Three ways to give +it its first authority: + +- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: an inherited-by-default table is + ambient authority smuggled across creation. +- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves exactly one handle; a loader sends the + rest as a processargs message)*. Minimal, but it leans on a buffered channel; over Setonix's unbuffered + rendezvous the parent must serve its own child a protocol before the child can do anything, and every + spawner holds a bootstrap endpoint per child. +- **C — explicit grants before start** *(seL4's root task writing a child's CSpace, on a flat table)*. + **Verdict sought.** + +A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, and is born with an +**empty table**. Its methods: + +| Method | Right | Effect | +|--------|-------|--------| +| `GRANT` | `WRITE` on the process; `TRANSFER` on each attached capability | move up to three attached capabilities into the child's table; the reply's MR0–MR2 carry their handle words *in the child* | +| `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | + +`GRANT` is a `call` on the Process with capabilities attached: the kernel takes them from `caps[]` +exactly as an endpoint send does (§6) — fetched once, all resolved before any moves. + +**The grant window.** `GRANT` succeeds only until the first `resume` of any thread bound to the process — +a one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers +`WindowClosed`. From then on authority arrives only by IPC, which the process consents to by receiving. A +parent cannot reach into a running child's table, and a stolen Process capability cannot inject authority +into a live process. The window is **single-use** in research/0002's sense (Part 3, Vault's +response-wrapping) but gives none of its **prior-use detection**: the kernel does not tell the child who +granted what. Within the kernel the channel is the Process capability, minted to its creator alone; +detection across imperfectly trusted spawn channels belongs to the broker RFC, which owns those channels. + +The child learns its handles as any program learns its arguments: the spawner writes them into `a0` and +`a1` (for the demo, the console and the endpoint), or points `a0` at a start block it wrote — a +runtime-row convention, not kernel ABI. No right joins RFC-0003's set: `GRANT`, `KILL` and thread +creation all use `WRITE` on the process — authority over *that one object*, held by its creator, not a +system-wide catch-all. A process holds no capability to itself unless handed one, and is destroyed when +its last thread exits. + +## 10. The root task and boot info + +The one act the kernel performs unasked: at boot it builds **exactly one** process from **exactly one** +image, the first boot module, and gives it every boot capability. *(seL4's root task; research/0002 Part 3: +"the bootstrap seam is a one-time kernel act".)* In order: an AddressSpace; the descriptor's segments +copied into fresh frames and mapped with their stated permissions (§11); a stack; an IPC buffer; a +read-only **BootInfo** page; a Process and a Thread with the entry state of §4, `a0` = BootInfo's address +and `a1` = 0; `eret`. RFC-0006 (proposed) provides the root's first scheduling context, whose parameters +are a named boot constant — as seL4's root task starts at the maximum priority with one configured +timeslice. + +**BootInfo** is kernel ABI, defined in `user/abi`: a header (magic, version, length) and one entry per +initial capability — `{ kind, flags, handle, base, size }` — with a string table for module names. Every +initial capability is *listed*; none sits at a well-known slot, so the order can change without an ABI +break and a reviewer can enumerate the whole bootstrap grant. A host test pins its size within one page. +This RFC defines three kinds: the root's own Process, the Console (§12), and one read-only Region per +further boot module, with that module's load descriptor. RFC-0005 (proposed) provides the root's +AddressSpace, the root Pool covering all free RAM and the DTB Region; RFC-0006 (proposed) its Thread, +`SchedControl` and interrupt capabilities. + +**The root is not a superuser.** It holds everything at first, but as ordinary, droppable, +generation-checked capabilities, and the rule that makes that true is checkable: **no kernel code +branches on "is this the root task"** — after `eret` the kernel does not know which process came first. +The bootstrap graph is acyclic by construction: kernel to root, root to children, and nothing the root +needs waits on a userspace service. **Rejected:** a kernel boot manifest of programs and their grants — +policy in the kernel; a `spawn_from_module` syscall — a loader offered to userspace as a kernel service; +and a bootstrap superuser, research/0002's `system:masters`. + +## 11. Getting the first images into memory + +QEMU's `-kernel` loads one ELF. The candidates for the rest: + +- **Embed them in the kernel ELF** *(verdict sought)*. One artefact boots, and its hash pins every + program in it. +- **QEMU's `-initrd` or `-device loader`.** Two artefacts, QEMU-only — and neither reaches a bare ELF. In + QEMU's `hw/arm/boot.c` (read at 8.2.2), `arm_setup_direct_kernel_boot` loads an `-initrd` only for + images it treats as Linux, never an ELF; for a non-Linux image `do_cpu_reset` sets the PC and nothing + else, so no register names the DTB either, which QEMU places at the base of RAM only if the image leaves + room there. Today's image *is* the base of RAM, so it gets no DTB; RFC-0005 (proposed) relinks to + `0x4020_0000` for that reason. A `-device loader` blob lands where only the command line knows. +- **Link the children into the root's ELF** *(common seL4 practice: a CPIO archive in the root image)*. + The kernel knows one module, but the root becomes specific to each boot image and the children stop + being separate immutable objects it can hand on. + +**`xtask` parses ELF; the kernel does not.** `xtask` reads each program's headers with a dependency-free +ELF64 reader and emits a fixed-shape **load descriptor**: the entry point; a text segment (`R+X`), an +optional read-only segment (`R`) and a data segment (`R+W`, with a `bss` size), each page-aligned in the +user half; and the stack, IPC-buffer and BootInfo addresses. It **refuses** at build time any image with a +writable-and-executable segment, overlapping or unaligned segments, dynamic linking, TLS or any other +program header — W^X checked before the image exists (O-13). The kernel checks one fixed structure against +fixed bounds and never sees an ELF: Nexen's finding that kernel code validating structures built elsewhere +is the bug farm (research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` +crate — small and host-testable, but a parser in the TCB for no gain. + +**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned header (an `abi` type) +and one entry per module: name, descriptor, segment bytes — and builds the kernel with `SETONIX_PAYLOAD` +naming it; `kernel/build.rs` places it page-aligned in a read-only `.payload` section (`KEEP`). **With the +variable unset it emits an empty bundle**, so a bare `cargo build` or `cargo clippy --package +setonix-kernel` still links, boots and greets, printing `boot modules: 0` before halting as today. +*(Hubris's `xtask dist`; seL4's elfloader, which parses before the kernel runs.)* + +The kernel sees modules through one arch-neutral function, `arch::boot_modules()`. On AArch64 it returns +the embedded bundle; the x86_64 UEFI stub, when written, can return files it loaded — so embedding is +Phase 1's *source* of modules, not the design, and the kernel's contract is a range plus descriptors. The +kernel loads **entry 0 only**; every other entry reaches the root as a **read-only Region** over its pages +(RFC-0005 (proposed) provides those frames) with its descriptor in BootInfo, so the root copies segments +out without parsing anything and the image stays immutable (pillar 1). *(Genode's core serves boot modules +to init as read-only ROM modules.)* No "second process" logic exists in the kernel. + +**How the programs are built.** Each is a `no_std`, `no_main` workspace crate under `user/` — +`user/demo/server` and `user/demo/client` — depending only on `abi`, with no `unsafe`, linked by +`user/link/.ld` at a fixed address in RFC-0005 (proposed)'s user half, static and non-PIE. The +AArch64 target already defaults to static relocation; `x86_64-unknown-none` defaults to position-independent +output and the *kernel* code model, so `xtask` passes `-C relocation-model=static -C code-model=small` for +user crates, in a separate target directory. The programs stay soft-float: per-thread FP/SIMD is +RFC-0006's, and the demo needs none. + +## 12. The console as a capability + +Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, +invoked by `call`: the byte count in MR0 and the bytes from MR1 on — up to 504 per call, so a short line +never leaves the registers. It needs the `WRITE` right. The root receives the Console with +`DUPLICATE | TRANSFER | WRITE` — no `READ`: there is no input path — and derives `WRITE`-only copies for +the children it chooses. A program not handed one cannot print, which is the point. + +**Rejected:** seL4's `DebugPutChar`, which any thread may issue in a `CONFIG_PRINTING` build — ambient +authority, however small; and a console at a well-known handle number. **Adopted in spirit:** Zircon's +debuglog, a handle-rights object write-only by default. Because §5 makes invocation uniform, once a +userspace UART driver exists the root hands out an endpoint speaking `WRITE`, no client changes, and the +Console object is **deleted** (increment 13). **Interim, named:** until then this is a device path +reachable from userspace inside the kernel — tiny, write-only and exercised by every boot-test, so not +VENOM's shape — but on real hardware at 115,200 baud a 504-byte write holds the non-preemptible kernel +for about 44 ms, a bound RFC-0006's latency budget must carry until the driver moves out. + +## 13. `user/abi` — the kernel's own binding + +**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program a conventional +environment: allocation, formatting, start-up conventions, POSIX-shaped calls. It does not issue +syscalls. relibc sits on `redox_syscall`, a separate crate versioned with the Redox kernel; seL4 draws the +same line, keeping libsel4 in the kernel's repository, generated from the same interface specifications +as the kernel, with runtimes elsewhere. The binding is the user half of §4's table — there is nothing +anywhere to port it from. **Verdict sought: the microkernel-core row, write ourselves.** A future relibc +port adds a Setonix platform layer *above* it; anything in `user/abi` that would read the same on another +kernel belongs to the runtime row and moves out. The maintainer may wish to append "syscall ABI binding +(`user/abi`)" to that row's parenthesis — constitution text, and his. If he reads it as the runtime row +instead, it is write-ourselves-by-necessity there, and that row's "port code" verdict needs a recorded +exception. + +It holds syscall numbers; `Handle`, `Tag` and `Status`; the `IpcBuffer`, `BootInfo`, bundle and +descriptor layouts, with `const` layout assertions; and safe typed wrappers for the twelve syscalls and +each kernel object's methods — no allocator, no formatting, no start-block convention. **One source of +truth:** the kernel compiles the data half too, with `default-features = false` so the `syscalls` feature +that gates `src/arch/**` stays off, and `xtask` uses it for the bundle. Kernel, tools and userspace cannot +drift, because they compile the same constants *(STYLE.md § Single Source of Truth)*. + +**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is proposed as +`user/abi/src/arch/**`: the `svc`/`syscall` inline assembly; the `_start` stub, which turns `a0` into a +`&'static BootInfo` for the root and calls `main`; and the accessors whose soundness rests on a kernel +promise about this process's memory — the IPC-buffer view through its register, and the slice over a +Region just mapped into the caller (the root's loader writes child segments through it). User programs +contain no `unsafe`. The `CLAUDE.md` edit adds a third bullet to § `unsafe` policy — "`user/abi/src/arch/**` +— trap instructions, the process entry stub and the kernel-shared memory the ABI defines" — adds `user/` to +§ Layout, and changes `xtask`'s entry there from "no deps" to "no external deps"; the +`grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. + +## 14. Lineage + +| Source | What is taken | What is left | +|--------|---------------|--------------| +| **seL4** | the root task and boot info; invoking kernel objects as IPC, errors in the reply label; four physical message registers; `ReplyRecv`; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | +| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one handle at start as the only channel; negative status integers | +| **Linux** | `svc #0` + `x8`; `syscall` + `r10`; the canonical-`rcx` check; `nr_open` as a sizing reference | everything the registers carry | +| **x86_64 ELF TLS** | a thread block whose first word holds its own address | `FS` itself, left to TLS | +| **Hubris** | the build tool assembles one bootable image | a static task set with no spawn | +| **Redox** | `redox_syscall` versioned with the kernel, relibc above it | ambient path syscalls (research/0001) | +| **Genode** | boot modules handed to init as read-only memory | — | + +## 15. Obligations + +| Obligation | This RFC | Status after implementation | +|-----------|----------|-----------------------------| +| O-1 unforgeability | handles reach the kernel only as words resolved through the caller's table; word 0 and out-of-range indices fail closed (§7) | **binds at B1** — RFC-0003's mechanism made live | +| O-2 non-widenability | `cap_derive` is the only duplication and is subset-only; `GRANT` and transfer move, never widen (§5, §9) | **binds at B1** | +| O-4 no ambient authority | twelve syscalls, each handle-indexed or authority-free; console by capability (§5, §12) | **discharged at the ABI**; each new kernel method is checked at review | +| O-5 argument validation | no pointer arguments; reserved bits zero; single fetch; caps all-or-nothing; kernel-built `SPSR_EL1`; canonical `rcx` (§4, §6, §8) | **discharged for the syscall surface**, with RFC-0005 for user-memory access | +| O-6 kernel memory safety | one new designated `unsafe` tree, named and bounded (§13) | **stays Built**, with a wider grep | +| O-7 no unprivileged exhaustion | 512-byte copy bound; creation paid by a named Pool; fixed tables fail closed; the console's bounded hold named (§6, §7, §12) | **contributes** — RFC-0005 owns the accounting | +| O-9 no ambient side channel | result registers carry results or zero; no grant after start; no ambient console (§4, §9, §12) | **contributes** | +| O-13 W^X | the build refuses writable-and-executable segments; the kernel maps per descriptor (§11) | **contributes** — RFC-0005 owns enforcement | +| O-17 contained drivers | fault messages to a supervisor (§8) | **designed, deferred** — the interim stops the thread | +| O-27 explicit authority at spawn | empty table at birth; grants only by explicit move; window shut at first resume; root's grant enumerated; demo proves a denial (§9, §10, §19) | **discharged** | + +## 16. Graves checked (§3) + +- **Policy in the kernel.** The kernel runs one image and lists what it granted; what runs, holding what, + is the root task's. The rejected manifest and `spawn_from_module` are this grave's two doors; the root's + boot scheduling constant is the one named residue. +- **The catch-all right.** No root identity survives `eret`; no right is added; `WRITE` on a Process is + authority over one object. The root holds much, but droppably and enumerably. +- **Baroque hierarchies; multi-copy IPC.** Handles stay flat indices with no receive window; the slow path + copies once, frame to frame, and BootInfo is a flat list. +- **Bolted-on multicore.** Entry state is per-CPU from the first stub (`SP_EL1` and `TPIDR_EL1`; `swapgs` + and per-CPU stacks); the single-fetch rule is written for a second core rewriting the buffer; the + syscall path takes no global lock. Two cores on one table stays RFC-0003 §14.2's question. +- **Drivers pulled in; unused device paths.** The Console is the kernel's own diagnostic UART, exposed + write-only as a named interim with its deletion scheduled; it is the only device method reachable from + EL0, and every boot-test uses it. + +## 17. Costs — what this makes harder + +- **`call` is overloaded:** per-type method decoding and two error layers — the price of interposition. +- **Four physical registers:** past 32 bytes a message touches memory; raising the count breaks binaries. + A 65–120-word message seL4 would copy needs a Region here. +- **Handles are 20/44 for good,** and the accepted `capability` crate changes twice: a slot-generation + bound and the `ObjectDestroyed` split. +- **Two thread registers belong to the ABI:** `TPIDRRO_EL0` and the user GS base are no longer free for a + runtime. +- **The root task is the most sensitive process.** Before it drops authority, a bug there compromises the + system, as in seL4. It must stay small. +- **No ambient printing:** a process without a Console fails silently. **Late grants need a willing + child:** after the window closes, only IPC delivers authority; and the window detects no prior use. +- **Embedding couples builds:** a user change relinks the kernel, the kernel's reproducibility now + includes the programs', and `xtask` gains its first dependency, in-workspace. +- **No PAN on the demo CPU,** and a canonical check on every x86_64 `sysretq`. + +## 18. Open questions + +1. **Fault messages** — their layout (this RFC's) and the thread's fault binding (RFC-0006's). +2. **Observing death.** A supervisor needs to learn a process died (O-17); a notification bound at + creation is the likely shape. +3. **Badges.** `cap_derive`'s badge word and the badge output register are reserved zero until RFC-0003a. +4. **The console after Phase 2** — whether any Console capability stays mintable once a userspace driver + owns the UART. +5. **The `unsafe` tree's name.** Two of its items are memory, not architecture; the maintainer may prefer + a sibling `src/kernel_mem/` beside `src/arch/`. +6. **The 64-word budget** — a verdict now, to be revisited with the first measurements. + +## 19. Implementation increments + +Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect +prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo carries three named +interims, each costed above: faults print and stop rather than reaching a supervisor (§8); the Console is +a device path inside the kernel (§12); boot modules come from the kernel image (§11) — plus RFC-0005's +static object pools and the root's boot scheduling constant (§10). The root task doubling as the demo's +server is not an interim: it is userspace composition, and yields exactly the two EL0 processes asked for. + +1. **[demo] `capability`: the handle word.** `to_word`/`from_word`, the slot-generation bound, the + `ObjectDestroyed` split. Host tests: round trips; word 0 and index ≥ 2^20 rejected; a slot retiring at + the bound while an object generation passes it; use-after-close and destruction told apart. +2. **[demo] `user/abi`, data half.** Numbers, `Tag`, `Status`, `IpcBuffer`, `BootInfo`, bundle and + descriptor types, with `const` layout assertions; no `unsafe`. Host tests: every reserved-bit and + over-length encoding rejected. CI builds it for both targets. +3. **[demo] AArch64 trap path, MMU off.** Lower-EL frame save, dispatcher, restore, `eret`; `yield` and + `thread_exit`; every other number `InvalidSyscall`; lower-EL faults stop the thread, not the core. A + feature like `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU still off — the + softfloat target is `+strict-align`, so EL0's Device-memory data accesses are safe — and executes + `svc #0`. Boot-test `--features syscall-selftest --expect "syscall round trip"`. Lands before RFC-0005. +4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer register read — **with the + `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first executed by increment 7. +5. **[demo] The bundle.** `xtask`'s ELF reader and descriptor writer, host-tested on fixtures including + writable-and-executable, overlapping, truncated and wrong-machine images; `build.rs` and `.payload`. + Boot-tests: a bare build `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. +6. **[demo] Objects, `cap_*` and the Console,** on RFC-0005's static interim tables: the self-test's EL0 + code prints through a Console handle. Boot-test `--features syscall-selftest --expect "[el0] Kaya!"`. +7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads: construction + (§10), BootInfo, entry state. Boot-test `--expect "[server] up"`. +8. **[demo] The Process object.** Grant window and teardown as pure logic in a host-tested `process/` + crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first + resume; `KILL` empties the table. +9. **[demo] The root spawns the client,** granting a `WRITE`-only Console and the endpoint. The client + prints `[client] up`, then invokes a handle word it was never granted, receives `InvalidHandle` and + prints `[client] denied, as expected` — O-27 proved observably, not asserted. +10. **[demo] IPC syscalls** over RFC-0004 endpoints, with RFC-0006's blocking states. Boot-test lines + in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`; RFC-0006's increments add + the preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. +11. **Fault messages and death notification** (§18.1–2). Boot-test: a deliberately faulting child is + reported to the root, which prints `[server] child faulted`. +12. **x86_64 trap path and `user/abi` stubs:** `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the + canonical check with `iretq` fallback. Proves §4's second column links; it boots with the UEFI stub. +13. **Delete the Console object** when the Phase-2 UART driver lands — scheduled now so it stays interim. + +## 20. What this unblocks + +The first userspace, and with it every Phase-2 paper: the **scheme registry** and the **broker** (both +built from processes, grants and `call`), the **driver framework** (fault messages, interrupt +notifications, the console moving out), a **relibc port** (a platform layer over `user/abi`, with both +TLS registers still its own), and the **x86_64 bring-up**, whose trap path is already on paper. RFC-0003a +gains a concrete ABI to place badges in. From 62ec1828e962363429c2712bd0944c0353c0b949 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:08:38 +0000 Subject: [PATCH 04/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 40 +++++++++++------------- 1 file changed, 19 insertions(+), 21 deletions(-) diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index bdd5b75..796201a 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -98,7 +98,7 @@ it waits on (RFC-0004 §8) and returns any donated context as budget expiry does |-----------|-----------|--------| | `set_priority(t, auth, p)` / `set_mcp(t, auth, m)` | `t`: `WRITE`; `auth`: `READ`; value `≤ mcp(auth)` | §6 | | `bind_sc(t, sc)` / `unbind_sc(t)` | `t`, `sc`: `WRITE` | active or passive; class checked (§12) | -| `resume(t)` / `suspend(t)` | `t`: `WRITE` | `Inactive` ↔ runnable | +| `resume(t)` / `suspend(t)` | `t`: `WRITE` | `Inactive` ↔ runnable; a first `resume` closes the Process's grant window (RFC-0007, proposed) | | `read_regs(t)` / `write_regs(t, …)` | `READ` / `WRITE` | frame and TLS bases, sanitised (§9) | | `set_fp(t, on)` / `set_class(t, c)` | `t`: `WRITE` | §10 / §12 | | `sc_configure(core, sc, B, T, refills, class)` / `sc_stats(sc)` | `core`, `sc`: `WRITE` / `sc`: `READ` | §5 | @@ -157,8 +157,7 @@ endured (research/0002 Part 1, "Mechanism/policy split"). count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; one whose timeslice ended, at the tail. The running thread is never queued, and a thread switched to directly is never enqueued at all. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* -- **Release queue.** Per core, `Throttled` threads sorted by next refill: O(n) insertion in the contexts - bound to that core, a number only the `Core` holder can raise. +- **Release queue.** Per core, sorted by next refill: O(n) in contexts only the `Core` holder can add. - **Idle is a kernel loop, not a thread**, billing nobody, and the lost wake-up is designed out. AArch64 runs `wfi` with `PSTATE.I` set — a pending interrupt still wakes it — then unmasks briefly; Linux warns masking at `ICC_PMR_EL1` would *not* wake the core (`arch/arm64/kernel/idle.c`), so this kernel masks @@ -182,18 +181,18 @@ priority-ceiling bound seL4 MCS relies on. Lending priority is safe because time to an **active** receiver donates nothing. Chains compose one hop at a time. *(Ford & Lepreau, migrating threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* -**Rejected: a passive server at its own priority only** (seL4 MCS), which leaves inversion to a global -convention. **Rejected: boosting a busy server to a waiting sender's priority** (QNX): the kernel would -guess which thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. +**Rejected:** a passive server at its own priority only (seL4 MCS), leaving inversion to a global +convention; and boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess +which thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. **The 2026-08-01 amendment, discharged**, following seL4 RFC-14 (Mitchell Johnston, "Budget limit thresholds on endpoints for SC Donation", proposed 2023-08-15; seL4/rfcs pull request 24, opened 2024-06-17, still open; read in full for this RFC): - **Prevention — the threshold.** An endpoint carries a `threshold`, fixed at creation (0 means none; - RFC-0007 (proposed) carries the parameter). A `call` needs released budget of at least `threshold` + RFC-0007 (proposed) encodes it, as `min_budget`). A `call` needs released budget of at least `threshold` plus twice the kernel's worst-case path, paying for the `call` and `reply` themselves — RFC-14's rule. - A context whose `B` can never meet that fails with `BudgetBelowThreshold`; a non-donating `send` to a + A context whose `B` can never meet that fails with `BudgetRefused`; a non-donating `send` to a thresholded endpoint is invalid. With `threshold` at the server's WCET, expiry inside it is a true error. - **A flagged deviation.** The amendment says donation below the threshold is "refused up front". RFC-14, which it cites, refuses only a context that can *never* pass; one that cannot pass *now* has its refills @@ -278,20 +277,19 @@ where a time-protection flush would go (Ge et al., EuroSys 2019; research/0002 P - **Loaded on first use, saved lazily.** Each core records an FP *owner*; every other thread runs with access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or `CR0.TS = 1` (`#NM`, vector 7). On that trap, for an FP-enabled thread, the kernel saves the owner's registers, - loads the new owner's (zeroed on first use), grants access (`FPEN = 0b11`; `clts`) and retries. A - `call` → FP-off server → `reply` round trip saves nothing. State is saved eagerly before a context moves. + loads the new owner's (zeroed on first use), grants access (`FPEN = 0b11`; `clts`) and retries. A `call` + to an FP-off server saves nothing. State is saved eagerly before a context moves core. - **AArch64's asymmetry, stated.** No `FPEN` value traps EL1 while allowing EL0 (`0b01` is the reverse), so while an owner runs a stray kernel FP instruction would not trap; the soft-float target is the guarantee. - **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format area (512 legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. - **The known cost — LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a - non-owner can speculatively read the owner's registers before `#NM` resolves. Hardware side channels are - out of scope (threat model §7) and CLAUDE.md asks for lazy saving; the cure is Open question 2, costed in §16. + non-owner can read the owner's registers speculatively before `#NM` resolves. Hardware side channels are + out of scope (threat model §7); the cure is Open question 2, costed in §16. ## 11. Interrupts — the controller, and device IRQs as notifications (V8) -The kernel owns the interrupt controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`; Redox -and QNX userspace drivers.)* +The kernel owns the controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`; Redox; QNX.)* - **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. Kernel-reserved lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. @@ -367,7 +365,7 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. ## 15. Graves checked (§3) -- **Policy in the kernel.** No admission, balancing or budget choice. The one number the kernel chooses is 256. +- **Policy in the kernel.** No admission, balancing or budget choice; the one number chosen is 256. - **The catch-all right.** Priority authority is a relation (MCP); time is per core; interrupts per line. - **Bolted-on multicore.** Per-core state, IPIs, the lock and placement are stated before any SMP code. - **Drivers in the kernel; compiled-in unused device paths.** The kernel masks and signals, never services @@ -380,13 +378,13 @@ Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. one needs a configured context. "Spawn a thread and it runs" is gone — O-4's price, in time. - **The one-hop unwind leaves a stranded server to userspace.** The kernel makes the failure visible and restarts nothing: a supervisor is required before any passive server on partial contexts is trusted. -- **MCS is the least-verified part of seL4's lineage**, and sporadic refills are subtle — hence property tests. +- **MCS is seL4's least-verified part**, and sporadic refills are subtle — hence property tests. - **About 1 KiB per TCB on AArch64 and 1.25 KiB on x86_64**, FP space included for threads that never use it. - **FIFO endpoint queues** let a high-priority client wait behind earlier low-priority ones. - **Lazy FP's cure, costed.** Eager restore on a switch into an FP-enabled thread of another process moves - 520 bytes (16 `ldp q` pairs) or an 832-byte `XRSTOR` — 9 or 13 cache lines — and nothing on IPC to FP-off - servers. The cycles are a measurement owed; beside a `call` round trip of roughly 380–630 cycles (twice - research/0001's one-way 188–316) it matters only for FP-heavy clients. + 520 bytes (16 `ldp q` pairs) or an 832-byte `XRSTOR` — 9 or 13 cache lines — and nothing on IPC to + FP-off servers. Its cycles are a measurement owed, beside a `call` round trip of roughly 380–630 cycles + (twice research/0001's one-way 188–316). ## 17. Open questions @@ -427,14 +425,14 @@ build compiles the scheduler. **[demo]** marks what the Phase-1 demo needs. priority; a demo stretch goal. 9. **`scheduler` III — partial contexts.** Sporadic refills, merge, release queue. Property test: no window of length `T` sees more than `B` consumed. -10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test `BudgetBelowThreshold`, +10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test `BudgetRefused`, then `BudgetExpired` and a timeout fault for an overrunning server. Required before partial-context passive servers are trusted. 11. **Lazy FP/SIMD.** Two FP-enabled EL0 threads load different markers into `v0`, are preempted in turn and each finds its own: `fp: isolated`. 12. **`IrqControl`/`IrqHandler`.** PL011 receive (SPI 1, INTID 33) reaches an EL0 driver as a notification when xtask writes a byte to the serial port: `irq: notified`. -13. **x86_64 scheduling HAL** (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`) after its boot RFC: 3–5 and 11's strings. +13. **x86_64 scheduling HAL** (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`), after its boot RFC: 3–5, 11 again. 14. **SMP** — secondary cores from the park loop, the big lock, IPIs, placement classes. **What the demo takes as interim, honestly.** The demo needs increments 1–7 and runs full contexts only. From d93de1d70c0db06d616a217356f050c29a7c652e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:09:26 +0000 Subject: [PATCH 05/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0007-system-calls-and-process-bootstrap.md | 751 ++++++++---------- 1 file changed, 352 insertions(+), 399 deletions(-) diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index 696647f..8130c1e 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -16,7 +16,7 @@ > 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§5). > 2. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§4). > 3. **Messages:** four physical and sixty-four virtual message registers, three capabilities; a 1 KiB IPC buffer, every word fetched once (§6). -> 4. **IPC buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§6). +> 4. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§6). > 5. **Handle word:** 20-bit index, 44-bit *slot* generation, `0` is null; the crate retires slots at 2^44 − 1, objects keep 64 bits (§7). > 6. **Errors in two layers:** transport status in a register, method result in the reply label; `BudgetRefused` beside `BudgetExpired`; faults are not errors (§8). > 7. **Process object:** table + address space + threads, born empty, grant window shut by the first `resume` of any of its threads (§9). @@ -31,58 +31,49 @@ such that every request names a capability, no argument is trusted, and no process is born holding authority it was not explicitly handed?** -RFC-0003 §9 fixed the rule and RFC-0004 §10 the IPC semantics; both left the encoding here. This RFC -designs the encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 -named: minting the first capabilities without an ambient grantor. It does not design address spaces -(RFC-0005), threads or scheduling (RFC-0006), or any policy about which programs run holding what — -that is the root task's, in userspace, and later the broker's. +RFC-0003 §9 fixed the rule (resources only by handle; yielding, halting oneself and self-queries are the +authority-free exceptions) and RFC-0004 §10 the IPC semantics, including its 2026-08-01 amendment's +up-front minimum-budget refusal and error completion; both left the encoding here. This RFC designs the +encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 named: minting +the first capabilities without an ambient grantor. It does not design address spaces (RFC-0005), threads +or scheduling (RFC-0006), or any policy about which programs run holding what — that is the root task's, +in userspace, and later the broker's. ## 2. Which pillar **Pillar 2 and the kernel doctrine that capabilities are the only authority.** The syscall surface is the -whole of B1; one entry that names a resource without a handle makes RFC-0003's table decorative. Spawn -is where research/0002 found the freshest scar (Part 4: runc CVE-2024-21626, one inherited descriptor -became host traversal), and O-27 exists because of it. Pillar 1 is served in passing: boot images reach +whole of B1; one entry that names a resource without a handle makes RFC-0003's table decorative. Spawn is +where research/0002 found the freshest scar (Part 4: runc CVE-2024-21626, one inherited descriptor became +host traversal), and O-27 exists because of it. Pillar 1 is served in passing: boot images reach userspace as read-only regions, immutable from their first instruction. -## 3. What already binds this RFC - -- **RFC-0003 §9:** resources only by handle; authority-free are yielding, halting oneself, self-queries. -- **RFC-0004 §5–§10 and its 2026-08-01 amendment:** virtual message registers; one bounded copy; - cap-carrying messages off the fast path; all-or-nothing transfer; the IPC semantics; a minimum budget - refused up front at `call`, and an error completion when a budget expires inside a server. -- **`CLAUDE.md`:** `unsafe` only in designated trees; no kernel heap; both targets soft-float. - -## 4. Entering the kernel +## 3. Entering the kernel **AArch64 — `svc #0`.** From EL0, `svc` takes a synchronous exception to `VBAR_EL1 + 0x400` (lower EL, -AArch64 — entry 8 of the table `vectors.s` already installs) with `ESR_EL1.EC = 0x15`, the immediate in -`ESR_EL1.ISS[15:0]`, and `ELR_EL1` already past the `svc`; entry masks D, A, I and F and selects `SP_EL1` -*(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's -frame (RFC-0006 (proposed) provides its layout, in the TCB) and dispatches on EC: `0x15` to the syscall -dispatcher, anything else is a *fault* (§8). A non-zero immediate answers `InvalidArgument`, so a later -ABI revision is detectable rather than silently accepted. Return is `eret` with an `SPSR_EL1` the kernel -builds: `M` = EL0t, DAIF clear, NZCV from the frame; where implemented, `DIT` (Armv8.4) and `SSBS` -(Armv8.5) — which EL0 can already set with `msr` — are carried too; every other bit (`SS`, `IL`, `PAN`, -`UAO`, `TCO`, `BTYPE`) is zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. -*(Linux arm64 and seL4 both enter through `svc #0`.)* - -**x86_64 — `syscall` / `sysretq`.** `syscall` requires `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, -leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `RFLAGS` bits named in `IA32_FMASK` -and takes `CS`/`SS` from `IA32_STAR` — but does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System -Calls in 64-Bit Mode")*. The stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads -the kernel stack; the untrusted `rsp` is never dereferenced. `FMASK` clears IF, DF, TF and AC: interrupts -off, string direction sane, and — where SMAP exists — user access closed. Two hazards are designed in: - -- **Non-canonical return.** On Intel parts `sysretq` with a non-canonical `rcx` raises #GP *in ring 0* - on the user's stack — CVE-2012-0217, which took Xen, FreeBSD, NetBSD and Windows. RFC-0005 (proposed) - leaves the top user page unmapped and RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`; the exit - path checks `rcx` anyway and falls back to `iretq`, because the check is one compare. -- **The window before the stack switch.** An NMI or machine check there would run on a user-chosen - `rsp`; RFC-0006 (proposed) provides IST stacks for the exceptions that can land in it. - -`sysretq` also fixes the GDT order — user data at `STAR[63:48] + 8`, user code at `+ 16`. `sysenter` is -rejected: AMD does not implement it in long mode; so is an `int` gate, which costs a full IDT delivery. +AArch64 — entry 8 of the table `vectors.s` installs) with `ESR_EL1.EC = 0x15`, the immediate in +`ISS[15:0]` and `ELR_EL1` already past the `svc`; entry masks D, A, I and F and selects `SP_EL1` *(Arm ARM +DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's frame +(RFC-0006 (proposed) provides its layout, in the TCB) and dispatches on EC: `0x15` to the syscall +dispatcher, anything else is a *fault* (§7). A non-zero immediate answers `InvalidArgument`, so a later +ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the kernel builds: `M` = EL0t, DAIF clear, +NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5), which EL0 can set itself with `msr`, are +carried where implemented; every other bit (`SS`, `IL`, `PAN`, `UAO`, `TCO`, `BTYPE`) is zero. The +Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. *(Linux arm64 and seL4 both use +`svc #0`.)* + +**x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, +leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `RFLAGS` bits in `IA32_FMASK`, takes +`CS`/`SS` from `IA32_STAR` — and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in +64-Bit Mode")*. The stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads the +kernel stack, never dereferencing the user's. `FMASK` clears IF, DF, TF and AC: interrupts off, string +direction sane, user access closed where SMAP exists. Two hazards are designed in, not found later: a +**non-canonical `rcx`** makes `sysretq` raise #GP *in ring 0* on the user's stack on Intel parts — +CVE-2012-0217, which took Xen, FreeBSD, NetBSD and Windows — so although RFC-0005 (proposed) leaves the +top user page unmapped and RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, the exit path checks +anyway and falls back to `iretq`; and **the window before the stack switch**, where an NMI or machine +check would run on a user-chosen `rsp`, gets IST stacks from RFC-0006 (proposed). `sysretq` fixes the GDT +order (user data at `STAR[63:48] + 8`, code at `+ 16`). **Rejected:** `sysenter`, which AMD does not +implement in long mode, and an `int` gate, which pays a full IDT delivery per call. | Role | AArch64 | x86_64 | |------|---------|--------| @@ -95,39 +86,36 @@ rejected: AMD does not implement it in long mode; so is an `int` gate, which cos | clobbered by hardware | — | `rcx`, `r11` | Every register outside the result set is restored from the frame; the eight result registers (`x0`–`x7`; -`rax`, `rsi`, `rdx`, `r10`, `r8`, `r9`, `r12`, `r13`) carry a result or zero, so no kernel value ever -leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux, leaving `x0`–`x7`, the AAPCS64 argument -registers, to the message. The x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot -name (LLVM's base and frame pointers), and `rcx`/`r11`, which `syscall` destroys. seL4 has the same -shape in different seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. - -**Entry state of every thread,** root task included, on both architectures: PC = entry; SP as given; -`a0` and `a1` — `x0`/`x1`, `rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — -the two start values; every other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 -with `RFLAGS` = `0x202` (IF and the always-one bit 1); FP/SIMD trapped (`CPACR_EL1.FPEN` = 0, `CR0.TS` -set) until RFC-0006 (proposed) enables it for the thread; `TPIDR_EL0` and `FS_BASE` zero; the IPC-buffer -register set (§6). A Thread is created in exactly this state; the spawner writes PC, SP, `a0` and `a1` -through RFC-0006 (proposed)'s `write_regs`, and the kernel writes the root's (§10). - -**The kernel never touches user memory through an address userspace supplies.** No syscall argument is -a pointer; the only user memory the syscall path touches is the IPC buffer, reached by frame (§6). On the -demo CPU this discipline is the whole defence: the Cortex-A72 is Armv8.0 and lacks PAN (Armv8.1). - -## 5. The syscall set +`rax`, `rsi`, `rdx`, `r10`, `r8`, `r9`, `r12`, `r13`) carry a result or zero, so no kernel value leaves in +a register (O-9 hygiene). `x8`/`rax` follow Linux, leaving the AAPCS64 argument registers to the message; +the x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and the two `syscall` +destroys. seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. + +**Entry state of every thread,** root included, on both architectures: PC = entry; SP as given; `a0` and +`a1` — `x0`/`x1`, `rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — the two +start values; every other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 with +`RFLAGS` = `0x202`; FP/SIMD trapped (`CPACR_EL1.FPEN` = 0; `CR0.TS` set) until RFC-0006 (proposed) +enables it for the thread; `TPIDR_EL0` and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is +created in this state; the spawner writes PC, SP, `a0` and `a1` through RFC-0006 (proposed)'s +`write_regs`, and the kernel writes the root's (§9). + +**No syscall argument is a pointer.** The only user memory the syscall path touches is the IPC buffer, +reached by frame (§5). On the demo CPU that discipline is the whole defence: the Cortex-A72 is Armv8.0 +and lacks PAN (Armv8.1). + +## 4. The syscall set - **A — one syscall per operation** *(Zircon, Linux)*. Legible per call, but the table grows with every object type, each entry is a new path to audit, and no server can ever stand in for a kernel object. - **B — uniform invocation** *(seL4; KeyKOS before it)*. A handful of IPC syscalls; an operation on a - kernel object is a `call` on its capability with the method in the tag's label, answered by the kernel - as a server would answer. Kernel objects and servers are invoked identically. -- **C — B plus a separate `invoke` number.** Keeps the small table and loses the point: the caller can - again tell a kernel object from a server. + kernel object is a `call` on its capability, method in the tag's label, answered as a server would. +- **C — B plus an `invoke` number.** Keeps the small table and loses the point: the caller can again tell + a kernel object from a server. **Verdict sought: B.** The payoff is interposition. A Console capability and an endpoint to a logging -server are invoked alike, so Phase 2's userspace UART driver takes over the console protocol without a -client changing, and the broker can hand out a forwarder in place of any kernel object — RFC-0003a option -(a)'s precondition. A `call` on a kernel object runs on the caller's own time: nothing is donated, -nothing blocks, no reply object is involved. +server are invoked alike, so Phase 2's UART driver takes over the console protocol without a client +changing, and the broker can hand out a forwarder in place of any kernel object — RFC-0003a option (a)'s +precondition. A `call` on a kernel object runs on the caller's own time: nothing is donated or blocks. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| @@ -144,33 +132,31 @@ nothing blocks, no reply object is involved. | 10 | `cap_identify` | object | out: MR0 the object kind (an `abi` enumeration), MR1 the rights bits | | 11 | `thread_exit` | — | halt the calling thread; authority-free | -`reply_recv` fuses two RFC-0004 operations so a server loop is one kernel entry, which the direct-switch -fast path needs; it adds no semantics. `notify` is `send` on a notification, since RFC-0004 left "one -syscall with a mode, or several" here; a signal never blocks, so `nb_send` signals too. The `cap_*` calls -take a zero tag, return a zero tag, and act only on the caller's own table, gated by the handle they name, -so §9 needs no table capability. A rights word with an undefined bit answers `InvalidArgument`; the -dispatcher assembles `Rights` from the crate's named constants, so the crate still never accepts raw -bits. Transfer in a message is always a move; a sender that keeps a copy runs `cap_derive` first -(RFC-0003 §6). **Creation** is a method on a memory capability — RFC-0005 (proposed) provides the Pool -and its creation methods — and the per-type parameters are encoded here: an endpoint's creation carries -**`min_budget`** in MR1, in RFC-0006 (proposed)'s time unit, 0 for none, fixed for the endpoint's life. -**Rejected:** a `debug_putc` (§12); anything taking a PID, path or address; seL4's depth-addressed +`reply_recv` fuses two RFC-0004 operations so a server loop is one kernel entry, as the direct-switch +fast path needs; `notify` is `send` on a notification, which never blocks. The `cap_*` calls take and +return a zero tag and act only on the caller's own table, so §8 needs no table capability; a rights word +with an undefined bit answers `InvalidArgument`, and the dispatcher builds `Rights` from the crate's named +constants, so the crate never accepts raw bits. Transfer is always a move; a sender that keeps a copy +runs `cap_derive` first (RFC-0003 §6). **Creation** is a method on a Pool capability, which RFC-0005 +(proposed) provides; per-type parameters are encoded here, and an endpoint's creation carries +**`min_budget`** in MR1 — RFC-0006 (proposed)'s time unit, 0 for none, fixed for the endpoint's life. +**Rejected:** a `debug_putc` (§11); anything taking a PID, path or address; seL4's depth-addressed capability pointers, which exist for CNode trees this kernel does not have. -## 6. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) +## 5. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) -The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities attached (0–3); bit 9 -set by the kernel when a `recv` was woken by its bound notification (MR0 then holds the pending bits), -zero on input; bits 10–15 reserved, zero; bits 16–63 a 48-bit label — user-defined for endpoints, the -method number for kernel objects, the method result on a kernel reply. *(seL4's `seL4_MessageInfo`: -label, extra capabilities, length.)* +The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities attached (0–3); bit 9 set +by the kernel when a bound notification woke a `recv` (MR0 then holds the pending bits); bits 10–15 +reserved, zero; bits 16–63 a 48-bit label — user-defined for endpoints, the method number for kernel +objects, the method result on a kernel reply. *(seL4's `seL4_MessageInfo`: label, extra caps, length.)* -**Four physical message registers on both architectures.** seL4 uses four on both 64-bit targets; -research/0001 found the in-register payoff small and shrinking (10% on ARM11, negative on modern x86); -and a count that differed by architecture would make one message fast on one Tier-1 target and slow on -the other. Messages up to **64 words** (512 bytes) spill MR4–MR63 to the IPC buffer — below seL4's 120 on -purpose: 512 bytes bounds the non-preemptible copy, and RFC-0004 §5 sends bulk data through shared -regions. **Three capabilities per message**, seL4's 2-bit field, so the field has no invalid encodings. +**Four physical message registers on both architectures:** seL4 uses four on both 64-bit targets, +research/0001 found the in-register payoff small and shrinking (10% on ARM11, negative on modern x86), +and a count that differed by architecture would make one message fast on one Tier-1 target only. +Messages up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds +the non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a +2-bit field with no invalid encodings. **Rejected:** six physical registers on AArch64 (portability); +seL4's 120 words (a longer non-preemptible copy for payloads RFC-0004 routes elsewhere). ```rust /// 1 KiB and 1 KiB-aligned, so it never straddles a page. @@ -184,382 +170,349 @@ pub struct IpcBuffer { } ``` -**The buffer is kernel-owned memory, reached by frame.** A thread's buffer is a 1 KiB-aligned slice of -one page of a Region, bound when the thread is created by the Region capability (`READ | WRITE`), the -offset, and the user address where the spawner mapped it; RFC-0006 (proposed) records the binding and -RFC-0005 (proposed) provides the Region and the kernel's physmap. The kernel reads and writes the buffer -only through its own mapping of that frame. The user address is used for two things — word 0 and the -discovery register — and is never dereferenced. Received capabilities land in fresh slots of the -receiver's table (RFC-0004 §6), so there is no seL4-style receive window. +**The buffer is kernel-reachable memory, named by capability.** It is a 1 KiB-aligned slice of one page +of a Region, bound when the thread is created by the Region capability (`READ | WRITE`), the offset, and +the user address where the spawner mapped it; RFC-0006 (proposed) records the binding, RFC-0005 +(proposed) provides the Region and the physmap. The kernel reads and writes it only through its own +mapping of that frame; the user address serves word 0 and the discovery register and is never +dereferenced. Received capabilities land in fresh slots (RFC-0004 §6): no seL4-style receive window. **Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's user address into `TPIDRRO_EL0` (readable at EL0, writable only at EL1) and into the user GS base on x86_64, where `CR4.FSGSBASE` stays off (RFC-0006 (proposed)) and `abi` reads the self-pointer at `gs:[0]` — the x86_64 -ELF TLS convention of a thread block whose first word holds its own address *(Drepper, "ELF Handling For -Thread-Local Storage")*. RFC-0006 (proposed) already restores both registers on every switch and leaves -their use here, so `_start` has its buffer at its first instruction, in any thread, with no TLS runtime. -The kernel trusts neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** -`TPIDR_EL0` and `FS_BASE` — the thread pointers of the AArch64 and x86_64 ELF TLS ABIs, which a relibc -port needs untouched; and seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 -`functions.h`) — sound, but it needs TLS before the first syscall. - -**Single fetch (O-5).** Once SMP exists, another thread of the same process can rewrite the buffer -mid-syscall. So every buffer word the kernel acts on is read **exactly once** into kernel memory before -any check depends on it; lengths and capability counts come from the tag register, already in the saved -frame, bounded by 64 and 3; the `caps[]` words are copied into a kernel array and every one is resolved -and checked for `TRANSFER` before any moves — all or nothing (RFC-0004 §6); payload words are copied -frame to frame by RFC-0005 (proposed)'s volatile-copy module and never interpreted, and no Rust reference -is ever formed into the buffer. *(research/0002 Part 5: virtio's double-fetch lesson — "validate a -copied-out snapshot; never re-read after validation".)* **Rejected:** six physical registers on AArch64 -(portability); seL4's 120-word maximum (a longer non-preemptible copy for payloads RFC-0004 routes -elsewhere). - -## 7. Handles in one register +ELF TLS convention of a block whose first word holds its own address *(Drepper, "ELF Handling For +Thread-Local Storage")*. RFC-0006 (proposed) already restores both on every switch and leaves their use +here, so `_start` has its buffer at its first instruction, in any thread, with no TLS runtime. The kernel +trusts neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** `TPIDR_EL0` +and `FS_BASE`, the thread pointers of the AArch64 and x86_64 ELF TLS ABIs, which a relibc port needs +untouched; and seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`) — +sound, but it needs TLS before the first syscall. + +**Single fetch (O-5).** Once SMP exists another thread of the process can rewrite the buffer +mid-syscall. So every buffer word the kernel acts on is read **exactly once** into kernel memory before any +check depends on it; lengths and counts come from the tag register, already in the frame, bounded by 64 +and 3; the `caps[]` words are copied into a kernel array and all are resolved and checked for `TRANSFER` +before any moves (RFC-0004 §6); payload words go frame to frame through RFC-0005 (proposed)'s +volatile-copy module, never interpreted, and no Rust reference is ever formed into the buffer. +*(research/0002 Part 5, virtio's double-fetch lesson: "validate a copied-out snapshot; never re-read".)* + +## 6. Handles in one register RFC-0003's `Handle` is a `u32` index and a 64-bit generation — ninety-six bits, which fit no register. -The ABI word is **`slot_generation << 20 | index`**: 2^20 slots per process, Linux's default -per-process descriptor ceiling (`fs.nr_open` = 1,048,576); and 2^44 slot generations, about two hundred -days of one close-and-reopen per microsecond on a single slot before it retires. `Generation::FIRST` is -1, so word `0` is never live and serves as null. This is the width RFC-0003's 2026-07-30 amendment asked -to be stated. **Only the slot generation crosses the ABI;** a capability's minted *object* generation -stays in kernel memory. So the accepted crate changes narrowly: `Handle::to_word`/`from_word` and a -constant `Handle::MAX_SLOT_GENERATION` (2^44 − 1); `CapabilityTable::remove` retires a slot whose next -generation would exceed it, through the `Retired` state the table already has, so a generation the ABI -cannot carry retires its slot rather than aliasing; a const assertion keeps every kernel table's `N` -≤ 2^20. `Generation` itself is untouched — objects keep all 64 bits — and its doc comment, which -promises 64 bits, gains the slot bound. Phase 1 tables have `N` = 64, fixed, failing closed with -`TableFull`. **Rejected:** two registers per handle (halves the message budget); 32/32 (2^32 reuses is -seventy minutes at one per microsecond, and the LIFO free list concentrates reuse on one slot); a -16-bit index (65,536 capabilities is too few for the server workloads the constitution targets). - -## 8. The error model - -**Transport status** (`x0`/`rax`) is what the syscall layer found, identical whether the target is a -kernel object or a server. Every failure is atomic — nothing moved, nothing half-sent — and the detail -register names the culprit: 0 the invoked handle, 1–3 an attached capability, 4 the tag, 5 the reply -handle. +The ABI word is **`slot_generation << 20 | index`**: 2^20 slots per process, Linux's default descriptor +ceiling (`fs.nr_open` = 1,048,576); and 2^44 slot generations, about two hundred days of one +close-and-reopen per microsecond on one slot before it retires. `Generation::FIRST` is 1, so word `0` is +never live and serves as null. This is the width RFC-0003's 2026-07-30 amendment asked to be stated. +**Only the slot generation crosses the ABI;** a capability's object generation stays in kernel memory. So +the accepted crate changes narrowly: `Handle::to_word`/`from_word` and `Handle::MAX_SLOT_GENERATION` +(2^44 − 1); `CapabilityTable::remove` retires a slot whose next generation would exceed it, through the +`Retired` state it already has; a const assertion keeps every kernel table's `N` ≤ 2^20. `Generation` is +untouched — objects keep 64 bits — and its doc comment gains the slot bound. Phase 1 tables have +`N` = 64, fixed, failing closed. **Rejected:** two registers per handle (halves the message budget); 32/32 +(2^32 reuses is seventy minutes at one per microsecond, and the LIFO free list concentrates reuse on one +slot); a 16-bit index (too few for the server workloads the constitution targets). + +## 7. The error model + +**Transport status** (`x0`/`rax`) is what the syscall layer found, identical for a kernel object or a +server. Every failure is atomic — nothing moved, nothing half-sent — and the detail register names the +culprit: 0 the invoked handle, 1–3 an attached capability, 4 the tag, 5 the reply handle. | Status | Meaning | |--------|---------| | `Ok` | done | | `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total and needs no fault-handler policy | | `InvalidArgument` | a reserved bit, a length over 64, a non-zero `svc` immediate, an undefined rights bit | -| `InvalidHandle` | word 0, index out of range, empty slot, or stale slot generation — the handle names nothing *of yours*: the crate's `OutOfBounds`, `Empty`, `StaleGeneration` | +| `InvalidHandle` | word 0, index out of range, empty slot, stale slot generation — the handle names nothing *of yours* (the crate's `OutOfBounds`, `Empty`, `StaleGeneration`) | | `Revoked` | the slot is live but its object was destroyed — the crate's new `ObjectDestroyed`; close the handle | -| `WrongType` | the handle is live but names the wrong kind of object for this syscall | -| `InsufficientRights` | the capability lacks the right the operation needs | +| `WrongType` / `InsufficientRights` | live handle, wrong kind of object; or lacking the right the operation needs | | `TableFull` | a moved capability found no slot in the receiving table; nothing moved | | `WouldBlock` | `nb_send` or `nb_recv` found no partner | | `PeerGone` | the object or party waited on was destroyed mid-wait | -| `BudgetRefused` | `call` whose donated context can never meet the endpoint's `min_budget`, refused up front (RFC-0004's amendment; seL4 RFC-14) | +| `BudgetRefused` | a `call` whose donated context can never meet the endpoint's `min_budget`, refused up front (RFC-0004's amendment; seL4 RFC-14) | | `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object (RFC-0004's amendment) | -`Revoked` needs the second crate change. Today `StaleGeneration` covers both a reused slot — the -holder's own use-after-close — and a destroyed object; `resolve` already checks them in that order, so -the split is one new variant and one changed return. It lets a holder observe and survive destruction, -as research/0002 asks of revocation, without reporting its own use-after-close as something done to it. -When `BudgetRefused` and `BudgetExpired` arise is RFC-0006 (proposed)'s to define; its draft calls the -first `BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. - -**Method result** (the reply tag's label) is what the object said, written by the kernel for kernel -objects and by the server for endpoints, in one vocabulary — so a server standing in for a kernel object -reports errors the same way *(seL4 returns invocation errors in the reply's label)*. It reuses the status -numbering where meanings coincide and adds per-object results: `InvalidMethod`, RFC-0005 (proposed)'s -`PoolExhausted` and `Fragmented`, and §9's `WindowClosed`. - -**Faults** are EL0 exceptions that are not syscalls. They are not return values — the thread cannot -continue. The design suspends it and sends a fault message to a handler RFC-0006 (proposed) binds to the -thread, so a supervisor can restart a driver (O-17). **Interim:** until then the existing reporter prints -the fault, prefixed by the thread, and the thread stops; the core continues, and nothing restarts it. - -## 9. The Process object and spawning (O-27) - -A **Process** holds one RFC-0003 capability table, a reference to one AddressSpace (RFC-0005 (proposed)) -and the threads bound to it (RFC-0006 (proposed)). The per-process table makes the process a real object -*(Zircon)* rather than seL4's convention of a thread pointing at a CSpace and a VSpace. Three ways to give -it its first authority: - -- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: an inherited-by-default table is - ambient authority smuggled across creation. -- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves exactly one handle; a loader sends the - rest as a processargs message)*. Minimal, but it leans on a buffered channel; over Setonix's unbuffered - rendezvous the parent must serve its own child a protocol before the child can do anything, and every - spawner holds a bootstrap endpoint per child. -- **C — explicit grants before start** *(seL4's root task writing a child's CSpace, on a flat table)*. - **Verdict sought.** - -A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, and is born with an -**empty table**. Its methods: +`Revoked` needs the second crate change: `StaleGeneration` today covers both a reused slot — the holder's +own use-after-close — and a destroyed object. `resolve` already checks them in that order, so the split is +one variant and one changed return, and it lets a holder observe and survive destruction, as +research/0002 asks of revocation, without reporting its own bug as something done to it. When the two +budget statuses arise is RFC-0006 (proposed)'s to define; its draft names the first +`BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. + +**Method result** (the reply label) is what the object said, written by the kernel for kernel objects and +by the server for endpoints, in one vocabulary — so a stand-in server reports errors the same way *(seL4 +returns invocation errors in the reply's label)*. It reuses the status numbers where meanings coincide and +adds `InvalidMethod`, RFC-0005 (proposed)'s `PoolExhausted` and `Fragmented`, and §8's `WindowClosed`. + +**Faults** — EL0 exceptions that are not syscalls — are not return values. The design suspends the thread +and sends a fault message to a handler RFC-0006 (proposed) binds, so a supervisor can restart a driver +(O-17). **Interim:** the reporter prints the fault, prefixed by the thread; the thread stops, the core +continues, nothing restarts it. + +## 8. The Process object and spawning (O-27) + +A **Process** holds one RFC-0003 capability table, one AddressSpace (RFC-0005 (proposed)) and its threads +(RFC-0006 (proposed)) — a real object *(Zircon)*, not seL4's convention of a thread naming a CSpace and a +VSpace. Three ways to give it its first authority: + +- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority smuggled across + creation. +- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves one handle; a loader sends the rest as + processargs)*. Minimal, but over Setonix's unbuffered rendezvous the parent must serve its own child a + protocol before the child can act, holding a bootstrap endpoint per child. +- **C — explicit grants before start** *(seL4's root task writing a child's CSpace)*. **Verdict sought.** + +A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, with an **empty table**: | Method | Right | Effect | |--------|-------|--------| | `GRANT` | `WRITE` on the process; `TRANSFER` on each attached capability | move up to three attached capabilities into the child's table; the reply's MR0–MR2 carry their handle words *in the child* | | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | -`GRANT` is a `call` on the Process with capabilities attached: the kernel takes them from `caps[]` -exactly as an endpoint send does (§6) — fetched once, all resolved before any moves. +`GRANT` is a `call` on the Process with capabilities attached: the kernel takes them from `caps[]` exactly +as an endpoint send does (§5) — fetched once, all resolved before any moves. **The grant window.** `GRANT` succeeds only until the first `resume` of any thread bound to the process — a one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers -`WindowClosed`. From then on authority arrives only by IPC, which the process consents to by receiving. A +`WindowClosed`. From then on authority arrives only by IPC, which the process consents to by receiving: a parent cannot reach into a running child's table, and a stolen Process capability cannot inject authority -into a live process. The window is **single-use** in research/0002's sense (Part 3, Vault's -response-wrapping) but gives none of its **prior-use detection**: the kernel does not tell the child who -granted what. Within the kernel the channel is the Process capability, minted to its creator alone; -detection across imperfectly trusted spawn channels belongs to the broker RFC, which owns those channels. - -The child learns its handles as any program learns its arguments: the spawner writes them into `a0` and -`a1` (for the demo, the console and the endpoint), or points `a0` at a start block it wrote — a -runtime-row convention, not kernel ABI. No right joins RFC-0003's set: `GRANT`, `KILL` and thread -creation all use `WRITE` on the process — authority over *that one object*, held by its creator, not a -system-wide catch-all. A process holds no capability to itself unless handed one, and is destroyed when -its last thread exits. - -## 10. The root task and boot info - -The one act the kernel performs unasked: at boot it builds **exactly one** process from **exactly one** -image, the first boot module, and gives it every boot capability. *(seL4's root task; research/0002 Part 3: -"the bootstrap seam is a one-time kernel act".)* In order: an AddressSpace; the descriptor's segments -copied into fresh frames and mapped with their stated permissions (§11); a stack; an IPC buffer; a -read-only **BootInfo** page; a Process and a Thread with the entry state of §4, `a0` = BootInfo's address -and `a1` = 0; `eret`. RFC-0006 (proposed) provides the root's first scheduling context, whose parameters -are a named boot constant — as seL4's root task starts at the maximum priority with one configured -timeslice. - -**BootInfo** is kernel ABI, defined in `user/abi`: a header (magic, version, length) and one entry per -initial capability — `{ kind, flags, handle, base, size }` — with a string table for module names. Every -initial capability is *listed*; none sits at a well-known slot, so the order can change without an ABI -break and a reviewer can enumerate the whole bootstrap grant. A host test pins its size within one page. -This RFC defines three kinds: the root's own Process, the Console (§12), and one read-only Region per -further boot module, with that module's load descriptor. RFC-0005 (proposed) provides the root's -AddressSpace, the root Pool covering all free RAM and the DTB Region; RFC-0006 (proposed) its Thread, -`SchedControl` and interrupt capabilities. - -**The root is not a superuser.** It holds everything at first, but as ordinary, droppable, -generation-checked capabilities, and the rule that makes that true is checkable: **no kernel code -branches on "is this the root task"** — after `eret` the kernel does not know which process came first. -The bootstrap graph is acyclic by construction: kernel to root, root to children, and nothing the root -needs waits on a userspace service. **Rejected:** a kernel boot manifest of programs and their grants — -policy in the kernel; a `spawn_from_module` syscall — a loader offered to userspace as a kernel service; -and a bootstrap superuser, research/0002's `system:masters`. - -## 11. Getting the first images into memory +into a live process. The window is **single-use** in research/0002's sense (Part 3, from Vault's +response-wrapping) but gives none of its **prior-use detection** — the child is not told who granted what. +Inside the kernel the channel is the Process capability, minted to its creator alone; detection across +imperfectly trusted spawn channels belongs to the broker RFC, which owns those channels. + +The child learns its handles as a program learns its arguments: the spawner writes them into `a0` and +`a1` (the demo's console and endpoint), or points `a0` at a start block — a runtime-row convention, not +kernel ABI. No right joins RFC-0003's set: `GRANT`, `KILL` and thread creation all use `WRITE` on that one +Process, held by its creator — not a catch-all. A process holds no capability to itself unless handed one, +and is destroyed when its last thread exits. + +## 9. The root task and boot info + +The one act the kernel performs unasked: at boot it builds **exactly one** process from the first boot +module and gives it every boot capability *(seL4's root task; research/0002 Part 3: "the bootstrap seam is +a one-time kernel act")*: an AddressSpace; the descriptor's segments copied into fresh frames and mapped +as stated (§10); a stack; an IPC buffer; a read-only **BootInfo** page; a Process and a Thread in §3's +entry state with `a0` = BootInfo's address and `a1` = 0; `eret`. RFC-0006 (proposed) provides the root's +first scheduling context, its parameters a named boot constant — as seL4's root starts at the maximum +priority with one configured timeslice. + +**BootInfo** is kernel ABI, in `user/abi`: a header (magic, version, length) and one entry per initial +capability — `{ kind, flags, handle, base, size }` — with a string table for module names, pinned within +one page by a host test. Every initial capability is *listed*; none sits at a well-known slot, so the +order can change without an ABI break and a reviewer can enumerate the whole bootstrap grant. This RFC +defines three kinds: the root's own Process, the Console (§11), and one read-only Region per further boot +module, with its load descriptor. RFC-0005 (proposed) provides the AddressSpace, the root Pool over all +free RAM and the DTB Region; RFC-0006 (proposed) the Thread, `SchedControl` and interrupt capabilities. + +**The root is not a superuser.** It holds everything at first, as ordinary, droppable, generation-checked +capabilities, and the rule that keeps it so is checkable: **no kernel code branches on "is this the root +task"**. The bootstrap graph is acyclic by construction — kernel to root, root to children — and nothing +the root needs waits on a userspace service. **Rejected:** a kernel boot manifest of programs and grants +(policy in the kernel); a `spawn_from_module` syscall (a loader offered as a kernel service); and a +bootstrap superuser, research/0002's `system:masters`. + +## 10. Getting the first images into memory QEMU's `-kernel` loads one ELF. The candidates for the rest: -- **Embed them in the kernel ELF** *(verdict sought)*. One artefact boots, and its hash pins every - program in it. +- **Embed them in the kernel ELF** *(verdict sought)*. One artefact boots; its hash pins every program. - **QEMU's `-initrd` or `-device loader`.** Two artefacts, QEMU-only — and neither reaches a bare ELF. In - QEMU's `hw/arm/boot.c` (read at 8.2.2), `arm_setup_direct_kernel_boot` loads an `-initrd` only for - images it treats as Linux, never an ELF; for a non-Linux image `do_cpu_reset` sets the PC and nothing - else, so no register names the DTB either, which QEMU places at the base of RAM only if the image leaves - room there. Today's image *is* the base of RAM, so it gets no DTB; RFC-0005 (proposed) relinks to - `0x4020_0000` for that reason. A `-device loader` blob lands where only the command line knows. -- **Link the children into the root's ELF** *(common seL4 practice: a CPIO archive in the root image)*. - The kernel knows one module, but the root becomes specific to each boot image and the children stop - being separate immutable objects it can hand on. - -**`xtask` parses ELF; the kernel does not.** `xtask` reads each program's headers with a dependency-free -ELF64 reader and emits a fixed-shape **load descriptor**: the entry point; a text segment (`R+X`), an -optional read-only segment (`R`) and a data segment (`R+W`, with a `bss` size), each page-aligned in the -user half; and the stack, IPC-buffer and BootInfo addresses. It **refuses** at build time any image with a -writable-and-executable segment, overlapping or unaligned segments, dynamic linking, TLS or any other -program header — W^X checked before the image exists (O-13). The kernel checks one fixed structure against -fixed bounds and never sees an ELF: Nexen's finding that kernel code validating structures built elsewhere -is the bug farm (research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` + `hw/arm/boot.c` (read at QEMU 8.2.2), `arm_setup_direct_kernel_boot` loads an `-initrd` only for images + it treats as Linux, never an ELF, and for a non-Linux image `do_cpu_reset` sets the PC and nothing else: + no register names the DTB either, which QEMU puts at the base of RAM only if the image leaves room. + Today's image *is* the base of RAM, so it gets none; RFC-0005 (proposed) relinks to `0x4020_0000` for + that. A `-device loader` blob lands where only the command line knows. +- **Link the children into the root's ELF** *(common seL4 practice, a CPIO archive)*. The root becomes + specific to each image, and the children stop being separate immutable objects it can hand on. + +**`xtask` parses ELF; the kernel does not.** `xtask` reads each program's headers with a dependency-free, +host-tested ELF64 reader and emits a fixed-shape **load descriptor**: the entry point; a text segment +(`R+X`), an optional read-only segment (`R`) and a data segment (`R+W`, with a `bss` size), page-aligned in +the user half; the stack, IPC-buffer and BootInfo addresses. It **refuses** at build time any +writable-and-executable, overlapping or unaligned segment, dynamic linking, TLS or any other program +header — W^X checked before the image exists (O-13). The kernel checks one fixed structure against fixed +bounds and never sees an ELF: Nexen's finding that kernel code validating structures built elsewhere is +the bug farm (research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` crate — small and host-testable, but a parser in the TCB for no gain. -**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned header (an `abi` type) -and one entry per module: name, descriptor, segment bytes — and builds the kernel with `SETONIX_PAYLOAD` -naming it; `kernel/build.rs` places it page-aligned in a read-only `.payload` section (`KEEP`). **With the -variable unset it emits an empty bundle**, so a bare `cargo build` or `cargo clippy --package -setonix-kernel` still links, boots and greets, printing `boot modules: 0` before halting as today. -*(Hubris's `xtask dist`; seL4's elfloader, which parses before the kernel runs.)* - -The kernel sees modules through one arch-neutral function, `arch::boot_modules()`. On AArch64 it returns -the embedded bundle; the x86_64 UEFI stub, when written, can return files it loaded — so embedding is -Phase 1's *source* of modules, not the design, and the kernel's contract is a range plus descriptors. The -kernel loads **entry 0 only**; every other entry reaches the root as a **read-only Region** over its pages -(RFC-0005 (proposed) provides those frames) with its descriptor in BootInfo, so the root copies segments -out without parsing anything and the image stays immutable (pillar 1). *(Genode's core serves boot modules -to init as read-only ROM modules.)* No "second process" logic exists in the kernel. - -**How the programs are built.** Each is a `no_std`, `no_main` workspace crate under `user/` — -`user/demo/server` and `user/demo/client` — depending only on `abi`, with no `unsafe`, linked by -`user/link/.ld` at a fixed address in RFC-0005 (proposed)'s user half, static and non-PIE. The -AArch64 target already defaults to static relocation; `x86_64-unknown-none` defaults to position-independent -output and the *kernel* code model, so `xtask` passes `-C relocation-model=static -C code-model=small` for -user crates, in a separate target directory. The programs stay soft-float: per-thread FP/SIMD is -RFC-0006's, and the demo needs none. - -## 12. The console as a capability +**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned `abi` header and per +module a name, descriptor and segment bytes — and builds the kernel with `SETONIX_PAYLOAD` naming it; +`kernel/build.rs` places it page-aligned in a read-only `.payload` section (`KEEP`). **Unset, it emits an +empty bundle**, so a bare `cargo build` or `cargo clippy --package setonix-kernel` still links, boots and +greets, printing `boot modules: 0` before halting as today. *(Hubris's `xtask dist`; seL4's elfloader, +which parses before the kernel runs.)* + +The kernel sees modules through one arch-neutral function, `arch::boot_modules()`: the embedded bundle on +AArch64, files the UEFI stub loaded on x86_64 — so embedding is Phase 1's *source*, and the kernel's +contract is a range plus descriptors. It loads **entry 0 only**; every other entry reaches the root as a +**read-only Region** over its pages (RFC-0005 (proposed) provides the frames) with its descriptor in +BootInfo, so the root copies segments out without parsing anything and the image stays immutable +(pillar 1). *(Genode's core hands boot modules to init as read-only ROM modules.)* + +**The programs** are `no_std`, `no_main` workspace crates — `user/demo/server`, `user/demo/client` — +depending only on `abi`, with no `unsafe`, linked static and non-PIE by `user/link/.ld` in RFC-0005 +(proposed)'s user half. `x86_64-unknown-none` defaults to position-independent output and the *kernel* +code model, so `xtask` passes `-C relocation-model=static -C code-model=small` for user crates, in their +own target directory; the AArch64 target is static already. They stay soft-float, as the demo needs. + +## 11. The console as a capability Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, -invoked by `call`: the byte count in MR0 and the bytes from MR1 on — up to 504 per call, so a short line -never leaves the registers. It needs the `WRITE` right. The root receives the Console with -`DUPLICATE | TRANSFER | WRITE` — no `READ`: there is no input path — and derives `WRITE`-only copies for -the children it chooses. A program not handed one cannot print, which is the point. +by `call`: the byte count in MR0, the bytes from MR1 on — up to 504 per call, so a short line never leaves +the registers — needing the `WRITE` right. The root receives it with `DUPLICATE | TRANSFER | WRITE` (no +`READ`: there is no input path) and derives `WRITE`-only copies for the children it chooses. A program not +handed one cannot print, which is the point. **Rejected:** seL4's `DebugPutChar`, which any thread may issue in a `CONFIG_PRINTING` build — ambient -authority, however small; and a console at a well-known handle number. **Adopted in spirit:** Zircon's -debuglog, a handle-rights object write-only by default. Because §5 makes invocation uniform, once a -userspace UART driver exists the root hands out an endpoint speaking `WRITE`, no client changes, and the -Console object is **deleted** (increment 13). **Interim, named:** until then this is a device path -reachable from userspace inside the kernel — tiny, write-only and exercised by every boot-test, so not -VENOM's shape — but on real hardware at 115,200 baud a 504-byte write holds the non-preemptible kernel -for about 44 ms, a bound RFC-0006's latency budget must carry until the driver moves out. - -## 13. `user/abi` — the kernel's own binding - -**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program a conventional -environment: allocation, formatting, start-up conventions, POSIX-shaped calls. It does not issue -syscalls. relibc sits on `redox_syscall`, a separate crate versioned with the Redox kernel; seL4 draws the -same line, keeping libsel4 in the kernel's repository, generated from the same interface specifications -as the kernel, with runtimes elsewhere. The binding is the user half of §4's table — there is nothing -anywhere to port it from. **Verdict sought: the microkernel-core row, write ourselves.** A future relibc -port adds a Setonix platform layer *above* it; anything in `user/abi` that would read the same on another -kernel belongs to the runtime row and moves out. The maintainer may wish to append "syscall ABI binding -(`user/abi`)" to that row's parenthesis — constitution text, and his. If he reads it as the runtime row -instead, it is write-ourselves-by-necessity there, and that row's "port code" verdict needs a recorded -exception. +authority, however small — and a console at a well-known handle. **Adopted in spirit:** Zircon's debuglog, +a handle-rights object write-only by default. Because §4 makes invocation uniform, once a userspace UART +driver exists the root hands out an endpoint speaking `WRITE`, no client changes, and the Console object is +**deleted** (increment 13). **Interim, named:** until then a device path inside the kernel is reachable +from userspace — tiny, write-only, exercised by every boot-test — and on real hardware at 115,200 baud a +504-byte write holds the non-preemptible kernel for about 44 ms, a bound RFC-0006's latency budget carries. + +## 12. `user/abi` — the kernel's own binding + +**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment: +allocation, formatting, start-up conventions, POSIX-shaped calls. It does not issue syscalls. relibc sits +on `redox_syscall`, versioned with the Redox kernel; seL4 keeps libsel4 in the kernel's repository, +generated from the kernel's own interface specifications, with runtimes elsewhere. The binding is the user +half of §3's table — there is nothing to port it from. **Verdict sought: the microkernel-core row, write +ourselves.** A future relibc port adds a platform layer *above* it; anything in `user/abi` that would read +the same on another kernel moves out. The maintainer may wish to append "syscall ABI binding +(`user/abi`)" to that row — constitution text, and his. If he reads it as the runtime row instead, it is +write-ourselves-by-necessity there, and that row's "port code" verdict needs a recorded exception. It holds syscall numbers; `Handle`, `Tag` and `Status`; the `IpcBuffer`, `BootInfo`, bundle and -descriptor layouts, with `const` layout assertions; and safe typed wrappers for the twelve syscalls and -each kernel object's methods — no allocator, no formatting, no start-block convention. **One source of -truth:** the kernel compiles the data half too, with `default-features = false` so the `syscalls` feature -that gates `src/arch/**` stays off, and `xtask` uses it for the bundle. Kernel, tools and userspace cannot -drift, because they compile the same constants *(STYLE.md § Single Source of Truth)*. +descriptor layouts, with `const` layout assertions; and safe wrappers for the twelve syscalls and each +kernel object's methods — no allocator, no formatting. **One source of truth:** the kernel compiles the +data half with `default-features = false`, so the `syscalls` feature gating `src/arch/**` stays off, and +`xtask` uses it for the bundle; kernel, tools and userspace compile the same constants. **The third designated `unsafe` tree**, approved in principle on 2026-09-27, is proposed as -`user/abi/src/arch/**`: the `svc`/`syscall` inline assembly; the `_start` stub, which turns `a0` into a -`&'static BootInfo` for the root and calls `main`; and the accessors whose soundness rests on a kernel -promise about this process's memory — the IPC-buffer view through its register, and the slice over a -Region just mapped into the caller (the root's loader writes child segments through it). User programs -contain no `unsafe`. The `CLAUDE.md` edit adds a third bullet to § `unsafe` policy — "`user/abi/src/arch/**` -— trap instructions, the process entry stub and the kernel-shared memory the ABI defines" — adds `user/` to -§ Layout, and changes `xtask`'s entry there from "no deps" to "no external deps"; the -`grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. +`user/abi/src/arch/**`: the `svc`/`syscall` assembly; the `_start` stub, which turns `a0` into the root's +`&'static BootInfo` and calls `main`; and accessors whose soundness rests on a kernel promise about this +process's memory — the IPC-buffer view, and the slice over a Region just mapped into the caller (the root's +loader writes child segments through it). User programs contain no `unsafe`. **The `CLAUDE.md` edit** +adds to § `unsafe` policy "`user/abi/src/arch/**` — trap instructions, the process entry stub and the +kernel-shared memory the ABI defines", adds `user/` to § Layout and changes `xtask`'s "no deps" to "no +external deps"; the `grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. -## 14. Lineage +## 13. Lineage | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4** | the root task and boot info; invoking kernel objects as IPC, errors in the reply label; four physical message registers; `ReplyRecv`; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | -| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one handle at start as the only channel; negative status integers | +| **seL4** | the root task and boot info; kernel objects invoked as IPC, errors in the reply label; four physical message registers; `ReplyRecv`; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | +| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one start handle as the only channel; negative status integers | | **Linux** | `svc #0` + `x8`; `syscall` + `r10`; the canonical-`rcx` check; `nr_open` as a sizing reference | everything the registers carry | -| **x86_64 ELF TLS** | a thread block whose first word holds its own address | `FS` itself, left to TLS | +| **x86_64 ELF TLS** | a block whose first word holds its own address | `FS` itself, left to TLS | | **Hubris** | the build tool assembles one bootable image | a static task set with no spawn | | **Redox** | `redox_syscall` versioned with the kernel, relibc above it | ambient path syscalls (research/0001) | | **Genode** | boot modules handed to init as read-only memory | — | -## 15. Obligations +## 14. Obligations | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| -| O-1 unforgeability | handles reach the kernel only as words resolved through the caller's table; word 0 and out-of-range indices fail closed (§7) | **binds at B1** — RFC-0003's mechanism made live | -| O-2 non-widenability | `cap_derive` is the only duplication and is subset-only; `GRANT` and transfer move, never widen (§5, §9) | **binds at B1** | -| O-4 no ambient authority | twelve syscalls, each handle-indexed or authority-free; console by capability (§5, §12) | **discharged at the ABI**; each new kernel method is checked at review | -| O-5 argument validation | no pointer arguments; reserved bits zero; single fetch; caps all-or-nothing; kernel-built `SPSR_EL1`; canonical `rcx` (§4, §6, §8) | **discharged for the syscall surface**, with RFC-0005 for user-memory access | -| O-6 kernel memory safety | one new designated `unsafe` tree, named and bounded (§13) | **stays Built**, with a wider grep | -| O-7 no unprivileged exhaustion | 512-byte copy bound; creation paid by a named Pool; fixed tables fail closed; the console's bounded hold named (§6, §7, §12) | **contributes** — RFC-0005 owns the accounting | -| O-9 no ambient side channel | result registers carry results or zero; no grant after start; no ambient console (§4, §9, §12) | **contributes** | -| O-13 W^X | the build refuses writable-and-executable segments; the kernel maps per descriptor (§11) | **contributes** — RFC-0005 owns enforcement | -| O-17 contained drivers | fault messages to a supervisor (§8) | **designed, deferred** — the interim stops the thread | -| O-27 explicit authority at spawn | empty table at birth; grants only by explicit move; window shut at first resume; root's grant enumerated; demo proves a denial (§9, §10, §19) | **discharged** | - -## 16. Graves checked (§3) +| O-1 unforgeability | handles reach the kernel only as words resolved through the caller's table; word 0 and out-of-range indices fail closed (§6) | **binds at B1** — RFC-0003's mechanism made live | +| O-2 non-widenability | `cap_derive` is the only duplication and is subset-only; `GRANT` and transfer move, never widen (§4, §8) | **binds at B1** | +| O-4 no ambient authority | twelve syscalls, each handle-indexed or authority-free; console by capability (§4, §11) | **discharged at the ABI**; each new kernel method checked at review | +| O-5 argument validation | no pointer arguments; reserved bits zero; single fetch; caps all-or-nothing; kernel-built `SPSR_EL1`; canonical `rcx` (§3, §5, §7) | **discharged for the syscall surface**, with RFC-0005 for user memory | +| O-6 kernel memory safety | one new designated `unsafe` tree, named and bounded (§12) | **stays Built**, with a wider grep | +| O-7 no unprivileged exhaustion | 512-byte copy bound; creation paid by a named Pool; fixed tables fail closed; the console's hold bounded (§5, §6, §11) | **contributes** — RFC-0005 owns the accounting | +| O-9 no ambient side channel | result registers carry results or zero; no grant after start; no ambient console (§3, §8, §11) | **contributes** | +| O-13 W^X | the build refuses writable-and-executable segments; the kernel maps per descriptor (§10) | **contributes** — RFC-0005 owns enforcement | +| O-17 contained drivers | fault messages to a supervisor (§7) | **designed, deferred** — the interim stops the thread | +| O-27 explicit authority at spawn | empty table at birth; grants only by explicit move; window shut at first resume; root's grant enumerated; the demo proves a denial (§8, §9, §18) | **discharged** | + +## 15. Graves checked (§3) - **Policy in the kernel.** The kernel runs one image and lists what it granted; what runs, holding what, - is the root task's. The rejected manifest and `spawn_from_module` are this grave's two doors; the root's - boot scheduling constant is the one named residue. + is the root's. The rejected manifest and `spawn_from_module` are this grave's two doors; the root's boot + scheduling constant is the one named residue. - **The catch-all right.** No root identity survives `eret`; no right is added; `WRITE` on a Process is authority over one object. The root holds much, but droppably and enumerably. -- **Baroque hierarchies; multi-copy IPC.** Handles stay flat indices with no receive window; the slow path - copies once, frame to frame, and BootInfo is a flat list. -- **Bolted-on multicore.** Entry state is per-CPU from the first stub (`SP_EL1` and `TPIDR_EL1`; `swapgs` - and per-CPU stacks); the single-fetch rule is written for a second core rewriting the buffer; the - syscall path takes no global lock. Two cores on one table stays RFC-0003 §14.2's question. -- **Drivers pulled in; unused device paths.** The Console is the kernel's own diagnostic UART, exposed - write-only as a named interim with its deletion scheduled; it is the only device method reachable from - EL0, and every boot-test uses it. +- **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; the slow path + copies once, frame to frame. +- **Bolted-on multicore.** Entry state is per-CPU from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`); + single fetch is written for a second core rewriting the buffer; the syscall path takes no global lock. + Two cores on one table stays RFC-0003 §14.2's question. +- **Drivers pulled in; unused device paths.** The Console is the kernel's own diagnostic UART, write-only, + used by every boot-test and scheduled for deletion; it is the only device method reachable from EL0. -## 17. Costs — what this makes harder +## 16. Costs — what this makes harder - **`call` is overloaded:** per-type method decoding and two error layers — the price of interposition. -- **Four physical registers:** past 32 bytes a message touches memory; raising the count breaks binaries. - A 65–120-word message seL4 would copy needs a Region here. -- **Handles are 20/44 for good,** and the accepted `capability` crate changes twice: a slot-generation - bound and the `ObjectDestroyed` split. -- **Two thread registers belong to the ABI:** `TPIDRRO_EL0` and the user GS base are no longer free for a - runtime. -- **The root task is the most sensitive process.** Before it drops authority, a bug there compromises the +- **Four physical registers:** past 32 bytes a message touches memory, and a 65–120-word message seL4 + would copy needs a Region here; raising either count later breaks binaries. +- **Handles are 20/44 for good,** and the accepted `capability` crate changes twice. +- **Two thread registers belong to the ABI:** `TPIDRRO_EL0` and the user GS base are not a runtime's. +- **The root task is the most sensitive process:** until it drops authority, a bug there compromises the system, as in seL4. It must stay small. -- **No ambient printing:** a process without a Console fails silently. **Late grants need a willing - child:** after the window closes, only IPC delivers authority; and the window detects no prior use. -- **Embedding couples builds:** a user change relinks the kernel, the kernel's reproducibility now - includes the programs', and `xtask` gains its first dependency, in-workspace. +- **No ambient printing,** so a process without a Console fails silently; **late grants need a willing + child**, and the grant window detects no prior use. +- **Embedding couples builds:** a user change relinks the kernel, the kernel's reproducibility includes the + programs', and `xtask` gains its first, in-workspace dependency. - **No PAN on the demo CPU,** and a canonical check on every x86_64 `sysretq`. -## 18. Open questions +## 17. Open questions 1. **Fault messages** — their layout (this RFC's) and the thread's fault binding (RFC-0006's). -2. **Observing death.** A supervisor needs to learn a process died (O-17); a notification bound at - creation is the likely shape. -3. **Badges.** `cap_derive`'s badge word and the badge output register are reserved zero until RFC-0003a. -4. **The console after Phase 2** — whether any Console capability stays mintable once a userspace driver - owns the UART. -5. **The `unsafe` tree's name.** Two of its items are memory, not architecture; the maintainer may prefer - a sibling `src/kernel_mem/` beside `src/arch/`. +2. **Observing death** — a supervisor must learn a process died (O-17); a notification bound at creation + is the likely shape. +3. **Badges** — `cap_derive`'s badge word and the badge register are reserved zero until RFC-0003a. +4. **The console after Phase 2** — whether any Console capability stays mintable once a driver owns the UART. +5. **The `unsafe` tree's name** — two of its items are memory, not architecture; the maintainer may prefer + a sibling `src/kernel_mem/`. 6. **The 64-word budget** — a verdict now, to be revisited with the first measurements. -## 19. Implementation increments +## 18. Implementation increments Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect -prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo carries three named -interims, each costed above: faults print and stop rather than reaching a supervisor (§8); the Console is -a device path inside the kernel (§12); boot modules come from the kernel image (§11) — plus RFC-0005's -static object pools and the root's boot scheduling constant (§10). The root task doubling as the demo's -server is not an interim: it is userspace composition, and yields exactly the two EL0 processes asked for. - -1. **[demo] `capability`: the handle word.** `to_word`/`from_word`, the slot-generation bound, the - `ObjectDestroyed` split. Host tests: round trips; word 0 and index ≥ 2^20 rejected; a slot retiring at - the bound while an object generation passes it; use-after-close and destruction told apart. -2. **[demo] `user/abi`, data half.** Numbers, `Tag`, `Status`, `IpcBuffer`, `BootInfo`, bundle and - descriptor types, with `const` layout assertions; no `unsafe`. Host tests: every reserved-bit and - over-length encoding rejected. CI builds it for both targets. -3. **[demo] AArch64 trap path, MMU off.** Lower-EL frame save, dispatcher, restore, `eret`; `yield` and - `thread_exit`; every other number `InvalidSyscall`; lower-EL faults stop the thread, not the core. A - feature like `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU still off — the - softfloat target is `+strict-align`, so EL0's Device-memory data accesses are safe — and executes - `svc #0`. Boot-test `--features syscall-selftest --expect "syscall round trip"`. Lands before RFC-0005. -4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer register read — **with the - `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first executed by increment 7. -5. **[demo] The bundle.** `xtask`'s ELF reader and descriptor writer, host-tested on fixtures including - writable-and-executable, overlapping, truncated and wrong-machine images; `build.rs` and `.payload`. - Boot-tests: a bare build `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. -6. **[demo] Objects, `cap_*` and the Console,** on RFC-0005's static interim tables: the self-test's EL0 - code prints through a Console handle. Boot-test `--features syscall-selftest --expect "[el0] Kaya!"`. -7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads: construction - (§10), BootInfo, entry state. Boot-test `--expect "[server] up"`. -8. **[demo] The Process object.** Grant window and teardown as pure logic in a host-tested `process/` - crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first - resume; `KILL` empties the table. +prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo's interims, each +costed above: faults print and stop (§7); the Console is a device path in the kernel (§11); modules come +from the kernel image (§10); RFC-0005's static object pools; the root's boot scheduling constant (§9). The +root doubling as the server is not an interim: it is userspace composition, and yields the two processes. + +1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, the slot bound, `ObjectDestroyed`. + Host tests: round trips; word 0 and index ≥ 2^20 rejected; a slot retiring at the bound while an + object generation passes it; use-after-close and destruction told apart. +2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, `IpcBuffer`, `BootInfo`, bundle and + descriptor types, `const` layout assertions, no `unsafe`. Host tests reject every reserved-bit and + over-length encoding; CI builds it for both targets. +3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatcher, restore, `eret`; `yield`, + `thread_exit`, `InvalidSyscall`; faults stop the thread, not the core. A feature like + `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off — the softfloat target is + `+strict-align`, so EL0's Device-memory accesses are safe — and runs `svc #0`. Boot-test + `--features syscall-selftest --expect "syscall round trip"`. Lands before RFC-0005's first mapping. +4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer-register read — **with the + `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first run by increment 7. +5. **[demo] The bundle** — `xtask`'s ELF reader and descriptor writer, host-tested on writable-and-executable, + overlapping, truncated and wrong-machine fixtures; `build.rs`, `.payload`. Boot-tests: a bare build + `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. +6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static interim tables. Boot-test + `--features syscall-selftest --expect "[el0] Kaya!"`, printed from EL0 through a Console handle. +7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads: construction, + BootInfo, entry state (§9). Boot-test `--expect "[server] up"`. +8. **[demo] The Process object** — grant window and teardown as pure logic in a host-tested `process/` + crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first resume; + `KILL` empties the table. 9. **[demo] The root spawns the client,** granting a `WRITE`-only Console and the endpoint. The client - prints `[client] up`, then invokes a handle word it was never granted, receives `InvalidHandle` and - prints `[client] denied, as expected` — O-27 proved observably, not asserted. -10. **[demo] IPC syscalls** over RFC-0004 endpoints, with RFC-0006's blocking states. Boot-test lines - in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`; RFC-0006's increments add - the preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. -11. **Fault messages and death notification** (§18.1–2). Boot-test: a deliberately faulting child is + prints `[client] up`, invokes a handle word it was never granted, receives `InvalidHandle` and prints + `[client] denied, as expected` — O-27 proved observably, not asserted. +10. **[demo] IPC syscalls** over RFC-0004 endpoints with RFC-0006's blocking states. Boot-test lines in + order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. RFC-0006's increments add the + preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. +11. **Fault messages and death notification** (§17.1–2). Boot-test: a deliberately faulting child is reported to the root, which prints `[server] child faulted`. -12. **x86_64 trap path and `user/abi` stubs:** `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the - canonical check with `iretq` fallback. Proves §4's second column links; it boots with the UEFI stub. +12. **x86_64 trap path and `user/abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the + canonical check with `iretq` fallback. Proves §3's second column links; boots with the UEFI stub. 13. **Delete the Console object** when the Phase-2 UART driver lands — scheduled now so it stays interim. -## 20. What this unblocks +## 19. What this unblocks -The first userspace, and with it every Phase-2 paper: the **scheme registry** and the **broker** (both -built from processes, grants and `call`), the **driver framework** (fault messages, interrupt -notifications, the console moving out), a **relibc port** (a platform layer over `user/abi`, with both -TLS registers still its own), and the **x86_64 bring-up**, whose trap path is already on paper. RFC-0003a -gains a concrete ABI to place badges in. +The first userspace, and with it every Phase-2 paper: the **scheme registry** and the **broker** (built +from processes, grants and `call`), the **driver framework** (fault messages, interrupt notifications, the +console moving out), a **relibc port** (a platform layer over `user/abi`, both TLS registers still its +own), and the **x86_64 bring-up**, whose trap path is on paper. RFC-0003a gains an ABI to place badges in. From dd53cf5d1fdee11ba7b35c61cbf1e07f35019622 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:12:44 +0000 Subject: [PATCH 06/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0007-system-calls-and-process-bootstrap.md | 541 ++++++++---------- 1 file changed, 246 insertions(+), 295 deletions(-) diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index 8130c1e..cae37bd 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -13,17 +13,17 @@ > **Proposed verdicts.** > -> 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§5). -> 2. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§4). -> 3. **Messages:** four physical and sixty-four virtual message registers, three capabilities; a 1 KiB IPC buffer, every word fetched once (§6). -> 4. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§6). -> 5. **Handle word:** 20-bit index, 44-bit *slot* generation, `0` is null; the crate retires slots at 2^44 − 1, objects keep 64 bits (§7). -> 6. **Errors in two layers:** transport status in a register, method result in the reply label; `BudgetRefused` beside `BudgetExpired`; faults are not errors (§8). -> 7. **Process object:** table + address space + threads, born empty, grant window shut by the first `resume` of any of its threads (§9). -> 8. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§10). -> 9. **Boot images:** `xtask` parses ELF on the host and embeds a bundle of fixed-shape descriptors in the kernel ELF; the kernel never parses ELF (§11). -> 10. **Console as a capability:** a kernel object with one `WRITE` method, scheduled for deletion when the Phase-2 UART driver lands (§12). -> 11. **`user/abi`** is the microkernel-core row, not libc/runtime; its `src/arch/**` is the third designated `unsafe` tree (§13). +> 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§4). +> 2. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§3). +> 3. **Messages:** four physical and sixty-four virtual message registers, three capabilities; a 1 KiB IPC buffer, every word fetched once (§5). +> 4. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§5). +> 5. **Handle word:** 20-bit index, 44-bit *slot* generation, `0` is null; the crate retires slots at 2^44 − 1, objects keep 64 bits (§6). +> 6. **Errors in two layers:** transport status in a register, method result in the reply label; `BudgetRefused` beside `BudgetExpired`; faults are not errors (§7). +> 7. **Process object:** table + address space + threads, born empty, grant window shut by the first `resume` of any of its threads (§8). +> 8. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§9). +> 9. **Boot images:** `xtask` parses ELF on the host and embeds a bundle of fixed-shape descriptors in the kernel ELF; the kernel never parses ELF (§10). +> 10. **Console as a capability:** a kernel object with one `WRITE` method, scheduled for deletion when the Phase-2 UART driver lands (§11). +> 11. **`user/abi`** is the microkernel-core row, not libc/runtime; its `src/arch/**` is the third designated `unsafe` tree (§12). ## 1. The question @@ -31,49 +31,41 @@ such that every request names a capability, no argument is trusted, and no process is born holding authority it was not explicitly handed?** -RFC-0003 §9 fixed the rule (resources only by handle; yielding, halting oneself and self-queries are the -authority-free exceptions) and RFC-0004 §10 the IPC semantics, including its 2026-08-01 amendment's -up-front minimum-budget refusal and error completion; both left the encoding here. This RFC designs the -encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 named: minting -the first capabilities without an ambient grantor. It does not design address spaces (RFC-0005), threads -or scheduling (RFC-0006), or any policy about which programs run holding what — that is the root task's, -in userspace, and later the broker's. +RFC-0003 §9 fixed the rule and RFC-0004 §10 the IPC semantics; both left the encoding here. This RFC +designs the encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 +named: minting the first capabilities without an ambient grantor. It does not design address spaces +(RFC-0005), threads or scheduling (RFC-0006), or any policy about which programs run holding what — that +is the root task's, in userspace, and later the broker's. ## 2. Which pillar -**Pillar 2 and the kernel doctrine that capabilities are the only authority.** The syscall surface is the -whole of B1; one entry that names a resource without a handle makes RFC-0003's table decorative. Spawn is -where research/0002 found the freshest scar (Part 4: runc CVE-2024-21626, one inherited descriptor became -host traversal), and O-27 exists because of it. Pillar 1 is served in passing: boot images reach -userspace as read-only regions, immutable from their first instruction. +**Pillar 2 and the doctrine that capabilities are the only authority.** The syscall surface is the whole +of B1; one entry naming a resource without a handle makes RFC-0003's table decorative. Spawn is where +research/0002 found the freshest scar (Part 4: runc CVE-2024-21626, one inherited descriptor became host +traversal), and O-27 exists because of it. Boot images reach userspace read-only (pillar 1). ## 3. Entering the kernel -**AArch64 — `svc #0`.** From EL0, `svc` takes a synchronous exception to `VBAR_EL1 + 0x400` (lower EL, -AArch64 — entry 8 of the table `vectors.s` installs) with `ESR_EL1.EC = 0x15`, the immediate in -`ISS[15:0]` and `ELR_EL1` already past the `svc`; entry masks D, A, I and F and selects `SP_EL1` *(Arm ARM -DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's frame -(RFC-0006 (proposed) provides its layout, in the TCB) and dispatches on EC: `0x15` to the syscall -dispatcher, anything else is a *fault* (§7). A non-zero immediate answers `InvalidArgument`, so a later -ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the kernel builds: `M` = EL0t, DAIF clear, -NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5), which EL0 can set itself with `msr`, are -carried where implemented; every other bit (`SS`, `IL`, `PAN`, `UAO`, `TCO`, `BTYPE`) is zero. The -Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. *(Linux arm64 and seL4 both use -`svc #0`.)* +**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` (lower EL, AArch64 — entry 8 of the +table `vectors.s` installs) with `ESR_EL1.EC = 0x15`, the immediate in `ISS[15:0]` and `ELR_EL1` past the +`svc`; entry masks DAIF and selects `SP_EL1` *(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, +`ELR_EL1` and `SPSR_EL1` into the thread's frame (RFC-0006 (proposed) provides its layout, in the TCB) and +dispatches on EC: `0x15` to the syscall dispatcher, anything else is a *fault* (§7). A non-zero immediate +answers `InvalidArgument`, so a later ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the +kernel builds: `M` = EL0t, DAIF clear, NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5), which +EL0 sets itself with `msr`, carried where implemented; every other bit (`SS`, `IL`, `PAN`, `UAO`, `TCO`, +`BTYPE`) zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. **x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, -leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `RFLAGS` bits in `IA32_FMASK`, takes -`CS`/`SS` from `IA32_STAR` — and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in -64-Bit Mode")*. The stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads the -kernel stack, never dereferencing the user's. `FMASK` clears IF, DF, TF and AC: interrupts off, string -direction sane, user access closed where SMAP exists. Two hazards are designed in, not found later: a -**non-canonical `rcx`** makes `sysretq` raise #GP *in ring 0* on the user's stack on Intel parts — -CVE-2012-0217, which took Xen, FreeBSD, NetBSD and Windows — so although RFC-0005 (proposed) leaves the -top user page unmapped and RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, the exit path checks -anyway and falls back to `iretq`; and **the window before the stack switch**, where an NMI or machine -check would run on a user-chosen `rsp`, gets IST stacks from RFC-0006 (proposed). `sysretq` fixes the GDT -order (user data at `STAR[63:48] + 8`, code at `+ 16`). **Rejected:** `sysenter`, which AMD does not -implement in long mode, and an `int` gate, which pays a full IDT delivery per call. +leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits, takes `CS`/`SS` from +`IA32_STAR`, and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. The +stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads the kernel stack, never +dereferencing the user's. `FMASK` clears IF, DF, TF and AC — user access closed where SMAP exists. A +**non-canonical `rcx`** makes Intel's `sysretq` raise #GP *in ring 0* on the user's stack (CVE-2012-0217, +which took Xen, FreeBSD, NetBSD and Windows): RFC-0005 (proposed) leaves the top user page unmapped and +RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, and the exit path checks anyway, falling back to +`iretq`. The window before the stack switch gets IST stacks from RFC-0006 (proposed). **Rejected:** +`sysenter`, which AMD does not implement in long mode; an `int` gate, a full IDT delivery per call. | Role | AArch64 | x86_64 | |------|---------|--------| @@ -85,37 +77,33 @@ implement in long mode, and an `int` gate, which pays a full IDT delivery per ca | error detail (out) | `x7` | `r13` | | clobbered by hardware | — | `rcx`, `r11` | -Every register outside the result set is restored from the frame; the eight result registers (`x0`–`x7`; -`rax`, `rsi`, `rdx`, `r10`, `r8`, `r9`, `r12`, `r13`) carry a result or zero, so no kernel value leaves in -a register (O-9 hygiene). `x8`/`rax` follow Linux, leaving the AAPCS64 argument registers to the message; -the x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and the two `syscall` -destroys. seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. +Every register outside the result set is restored from the frame; the eight result registers carry a +result or zero, so no kernel value leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux; the +x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and what `syscall` destroys. +seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. -**Entry state of every thread,** root included, on both architectures: PC = entry; SP as given; `a0` and -`a1` — `x0`/`x1`, `rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — the two -start values; every other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 with -`RFLAGS` = `0x202`; FP/SIMD trapped (`CPACR_EL1.FPEN` = 0; `CR0.TS` set) until RFC-0006 (proposed) -enables it for the thread; `TPIDR_EL0` and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is -created in this state; the spawner writes PC, SP, `a0` and `a1` through RFC-0006 (proposed)'s -`write_regs`, and the kernel writes the root's (§9). +**Entry state of every thread,** root included: PC = entry; SP as given; `a0`, `a1` — `x0`/`x1`, +`rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — the two start values; every +other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 with `RFLAGS` = `0x202`; +FP/SIMD trapped (`CPACR_EL1.FPEN` = 0; `CR0.TS` set) until RFC-0006 (proposed) enables it; `TPIDR_EL0` +and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is created so; the spawner writes PC, SP, +`a0` and `a1` through RFC-0006 (proposed)'s `write_regs`, and the kernel writes the root's (§9). -**No syscall argument is a pointer.** The only user memory the syscall path touches is the IPC buffer, -reached by frame (§5). On the demo CPU that discipline is the whole defence: the Cortex-A72 is Armv8.0 -and lacks PAN (Armv8.1). +**No syscall argument is a pointer.** The syscall path touches user memory only in the IPC buffer, by +frame (§5) — on the demo CPU the whole defence, since the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). ## 4. The syscall set -- **A — one syscall per operation** *(Zircon, Linux)*. Legible per call, but the table grows with every - object type, each entry is a new path to audit, and no server can ever stand in for a kernel object. -- **B — uniform invocation** *(seL4; KeyKOS before it)*. A handful of IPC syscalls; an operation on a - kernel object is a `call` on its capability, method in the tag's label, answered as a server would. -- **C — B plus an `invoke` number.** Keeps the small table and loses the point: the caller can again tell - a kernel object from a server. +- **A — one syscall per operation** *(Zircon, Linux)*. Legible, but every object type grows the table, + each entry is a new path to audit, and no server can ever stand in for a kernel object. +- **B — uniform invocation** *(seL4; KeyKOS)*. A handful of IPC syscalls; a kernel-object operation is a + `call` on its capability, the method in the tag's label, answered as a server would answer. +- **C — B plus an `invoke` number.** The small table without the point: callers can tell the two apart. -**Verdict sought: B.** The payoff is interposition. A Console capability and an endpoint to a logging -server are invoked alike, so Phase 2's UART driver takes over the console protocol without a client -changing, and the broker can hand out a forwarder in place of any kernel object — RFC-0003a option (a)'s -precondition. A `call` on a kernel object runs on the caller's own time: nothing is donated or blocks. +**Verdict sought: B.** The payoff is interposition: a Console and an endpoint to a logging server are +invoked alike, so Phase 2's UART driver takes over the console protocol without a client changing, and +the broker can hand out a forwarder for any kernel object — RFC-0003a option (a)'s precondition. A `call` +on a kernel object runs on the caller's own time: nothing is donated, nothing blocks. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| @@ -132,31 +120,26 @@ precondition. A `call` on a kernel object runs on the caller's own time: nothing | 10 | `cap_identify` | object | out: MR0 the object kind (an `abi` enumeration), MR1 the rights bits | | 11 | `thread_exit` | — | halt the calling thread; authority-free | -`reply_recv` fuses two RFC-0004 operations so a server loop is one kernel entry, as the direct-switch -fast path needs; `notify` is `send` on a notification, which never blocks. The `cap_*` calls take and -return a zero tag and act only on the caller's own table, so §8 needs no table capability; a rights word -with an undefined bit answers `InvalidArgument`, and the dispatcher builds `Rights` from the crate's named -constants, so the crate never accepts raw bits. Transfer is always a move; a sender that keeps a copy -runs `cap_derive` first (RFC-0003 §6). **Creation** is a method on a Pool capability, which RFC-0005 -(proposed) provides; per-type parameters are encoded here, and an endpoint's creation carries -**`min_budget`** in MR1 — RFC-0006 (proposed)'s time unit, 0 for none, fixed for the endpoint's life. -**Rejected:** a `debug_putc` (§11); anything taking a PID, path or address; seL4's depth-addressed -capability pointers, which exist for CNode trees this kernel does not have. +`reply_recv` fuses two RFC-0004 operations so a server loop is one entry, as direct switch needs. The +`cap_*` calls take and return a zero tag and act only on the caller's own table; an undefined rights bit +answers `InvalidArgument`, and the dispatcher builds `Rights` from the crate's named constants, which +still never accept raw bits. Transfer is a move; a sender keeping a copy derives first (RFC-0003 §6). +**Creation** is a method on a Pool, which RFC-0005 (proposed) provides; per-type parameters are encoded +here, and an endpoint's carries **`min_budget`** in MR1 (RFC-0006 (proposed)'s time unit, 0 for none, +fixed for the endpoint's life). **Rejected:** a `debug_putc` (§11); anything taking a PID, path or +address; seL4's depth-addressed capability pointers, made for CNode trees this kernel does not have. ## 5. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) -The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities attached (0–3); bit 9 set -by the kernel when a bound notification woke a `recv` (MR0 then holds the pending bits); bits 10–15 -reserved, zero; bits 16–63 a 48-bit label — user-defined for endpoints, the method number for kernel -objects, the method result on a kernel reply. *(seL4's `seL4_MessageInfo`: label, extra caps, length.)* - -**Four physical message registers on both architectures:** seL4 uses four on both 64-bit targets, -research/0001 found the in-register payoff small and shrinking (10% on ARM11, negative on modern x86), -and a count that differed by architecture would make one message fast on one Tier-1 target only. -Messages up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds -the non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a -2-bit field with no invalid encodings. **Rejected:** six physical registers on AArch64 (portability); -seL4's 120 words (a longer non-preemptible copy for payloads RFC-0004 routes elsewhere). +The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities (0–3); bit 9 set by the +kernel when a bound notification woke a `recv` (MR0 then holds the bits); bits 10–15 reserved, zero; bits +16–63 a 48-bit label — user-defined for endpoints, the method for kernel objects, the method result on a +kernel reply. *(seL4's `seL4_MessageInfo`.)* **Four physical message registers on both architectures:** +seL4 uses four on both, research/0001 found the in-register payoff small and shrinking (10% on ARM11, +negative on modern x86), and a per-architecture count would make a message fast on one Tier-1 target only. +Up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds the +non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a 2-bit +field with no invalid encodings. **Rejected:** six physical registers on AArch64; seL4's 120 words. ```rust /// 1 KiB and 1 KiB-aligned, so it never straddles a page. @@ -170,53 +153,49 @@ pub struct IpcBuffer { } ``` -**The buffer is kernel-reachable memory, named by capability.** It is a 1 KiB-aligned slice of one page -of a Region, bound when the thread is created by the Region capability (`READ | WRITE`), the offset, and -the user address where the spawner mapped it; RFC-0006 (proposed) records the binding, RFC-0005 -(proposed) provides the Region and the physmap. The kernel reads and writes it only through its own -mapping of that frame; the user address serves word 0 and the discovery register and is never -dereferenced. Received capabilities land in fresh slots (RFC-0004 §6): no seL4-style receive window. - -**Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's user address into -`TPIDRRO_EL0` (readable at EL0, writable only at EL1) and into the user GS base on x86_64, where -`CR4.FSGSBASE` stays off (RFC-0006 (proposed)) and `abi` reads the self-pointer at `gs:[0]` — the x86_64 -ELF TLS convention of a block whose first word holds its own address *(Drepper, "ELF Handling For -Thread-Local Storage")*. RFC-0006 (proposed) already restores both on every switch and leaves their use -here, so `_start` has its buffer at its first instruction, in any thread, with no TLS runtime. The kernel -trusts neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** `TPIDR_EL0` -and `FS_BASE`, the thread pointers of the AArch64 and x86_64 ELF TLS ABIs, which a relibc port needs -untouched; and seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`) — -sound, but it needs TLS before the first syscall. - -**Single fetch (O-5).** Once SMP exists another thread of the process can rewrite the buffer -mid-syscall. So every buffer word the kernel acts on is read **exactly once** into kernel memory before any -check depends on it; lengths and counts come from the tag register, already in the frame, bounded by 64 -and 3; the `caps[]` words are copied into a kernel array and all are resolved and checked for `TRANSFER` -before any moves (RFC-0004 §6); payload words go frame to frame through RFC-0005 (proposed)'s -volatile-copy module, never interpreted, and no Rust reference is ever formed into the buffer. -*(research/0002 Part 5, virtio's double-fetch lesson: "validate a copied-out snapshot; never re-read".)* +**The buffer is named by capability and reached by frame.** It is a 1 KiB-aligned slice of one page of a +Region, bound at thread creation by the Region capability (`READ | WRITE`), the offset and the user address +the spawner mapped it at; RFC-0006 (proposed) records the binding, RFC-0005 (proposed) provides the Region +and the physmap through which alone the kernel reads and writes it. The user address serves word 0 and +the discovery register and is never dereferenced. Received capabilities take fresh slots (RFC-0004 §6). + +**Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's address into +`TPIDRRO_EL0` (readable at EL0, writable only at EL1) and, on x86_64, into the user GS base, with +`CR4.FSGSBASE` off (RFC-0006 (proposed)) and `abi` reading the self-pointer at `gs:[0]` — the x86_64 ELF +TLS convention of a block whose first word is its own address *(Drepper, "ELF Handling For Thread-Local +Storage")*. RFC-0006 (proposed) already restores both on every switch and leaves their use here, so +`_start` has its buffer at its first instruction, in any thread, with no TLS runtime. The kernel trusts +neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** `TPIDR_EL0` and +`FS_BASE`, the ELF TLS thread pointers a relibc port needs untouched; and seL4's current answer, a +thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`) — sound, but it needs TLS before any syscall. + +**Single fetch (O-5).** Once SMP exists another thread can rewrite the buffer mid-syscall, so every buffer +word the kernel acts on is read **exactly once** into kernel memory before any check depends on it. +Lengths and counts come from the tag register, bounded by 64 and 3; `caps[]` is copied into a kernel array +and every word resolved and checked for `TRANSFER` before any moves (RFC-0004 §6); payload goes frame to +frame through RFC-0005 (proposed)'s volatile-copy module, never interpreted, and no Rust reference is +formed into the buffer. *(research/0002 Part 5, virtio's double-fetch lesson.)* ## 6. Handles in one register -RFC-0003's `Handle` is a `u32` index and a 64-bit generation — ninety-six bits, which fit no register. -The ABI word is **`slot_generation << 20 | index`**: 2^20 slots per process, Linux's default descriptor -ceiling (`fs.nr_open` = 1,048,576); and 2^44 slot generations, about two hundred days of one -close-and-reopen per microsecond on one slot before it retires. `Generation::FIRST` is 1, so word `0` is -never live and serves as null. This is the width RFC-0003's 2026-07-30 amendment asked to be stated. -**Only the slot generation crosses the ABI;** a capability's object generation stays in kernel memory. So -the accepted crate changes narrowly: `Handle::to_word`/`from_word` and `Handle::MAX_SLOT_GENERATION` -(2^44 − 1); `CapabilityTable::remove` retires a slot whose next generation would exceed it, through the -`Retired` state it already has; a const assertion keeps every kernel table's `N` ≤ 2^20. `Generation` is -untouched — objects keep 64 bits — and its doc comment gains the slot bound. Phase 1 tables have -`N` = 64, fixed, failing closed. **Rejected:** two registers per handle (halves the message budget); 32/32 -(2^32 reuses is seventy minutes at one per microsecond, and the LIFO free list concentrates reuse on one -slot); a 16-bit index (too few for the server workloads the constitution targets). +RFC-0003's `Handle` is a `u32` index and a 64-bit generation, which fit no register. The ABI word is +**`slot_generation << 20 | index`**: 2^20 slots, Linux's default descriptor ceiling (`fs.nr_open` = +1,048,576); 2^44 slot generations, about two hundred days of one close-and-reopen per microsecond on one +slot before it retires. `Generation::FIRST` is 1, so word `0` is never live and is null. This is the width +RFC-0003's 2026-07-30 amendment asked to be stated. **Only the slot generation crosses the ABI;** object +generations stay in kernel memory. So the accepted crate changes narrowly: `Handle::to_word`/`from_word` +and `Handle::MAX_SLOT_GENERATION` (2^44 − 1); `CapabilityTable::remove` retires a slot whose next +generation would exceed it, through its existing `Retired` state; a const assertion bounds every kernel +table's `N` by 2^20. `Generation` keeps 64 bits for objects, and its doc comment gains the slot bound. +Phase 1 tables have `N` = 64, fixed, failing closed. **Rejected:** two registers per handle (halves the +budget); 32/32 (2^32 reuses is seventy minutes at one per microsecond, and the LIFO free list +concentrates reuse on one slot); a 16-bit index (too few for the server workloads the constitution targets). ## 7. The error model **Transport status** (`x0`/`rax`) is what the syscall layer found, identical for a kernel object or a -server. Every failure is atomic — nothing moved, nothing half-sent — and the detail register names the -culprit: 0 the invoked handle, 1–3 an attached capability, 4 the tag, 5 the reply handle. +server. Every failure is atomic, and the detail register names the culprit: 0 the invoked handle, 1–3 an +attached capability, 4 the tag, 5 the reply handle. | Status | Meaning | |--------|---------| @@ -232,34 +211,29 @@ culprit: 0 the invoked handle, 1–3 an attached capability, 4 the tag, 5 the re | `BudgetRefused` | a `call` whose donated context can never meet the endpoint's `min_budget`, refused up front (RFC-0004's amendment; seL4 RFC-14) | | `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object (RFC-0004's amendment) | -`Revoked` needs the second crate change: `StaleGeneration` today covers both a reused slot — the holder's -own use-after-close — and a destroyed object. `resolve` already checks them in that order, so the split is -one variant and one changed return, and it lets a holder observe and survive destruction, as -research/0002 asks of revocation, without reporting its own bug as something done to it. When the two -budget statuses arise is RFC-0006 (proposed)'s to define; its draft names the first +`Revoked` needs the second crate change: `StaleGeneration` today covers both the holder's own +use-after-close and a destroyed object; `resolve` checks them in that order, so the split is one variant +and one changed return, and a holder observes destruction without its own bug being reported as one. When +the budget statuses arise is RFC-0006 (proposed)'s to define; its draft calls the first `BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. -**Method result** (the reply label) is what the object said, written by the kernel for kernel objects and -by the server for endpoints, in one vocabulary — so a stand-in server reports errors the same way *(seL4 -returns invocation errors in the reply's label)*. It reuses the status numbers where meanings coincide and -adds `InvalidMethod`, RFC-0005 (proposed)'s `PoolExhausted` and `Fragmented`, and §8's `WindowClosed`. +**Method result** (the reply label) is what the object said — the kernel's for kernel objects, the +server's for endpoints — in one vocabulary, so a stand-in server reports errors the same way *(seL4 returns +invocation errors in the reply's label)*. It reuses status numbers where meanings coincide and adds +`InvalidMethod`, RFC-0005 (proposed)'s `PoolExhausted` and `Fragmented`, and §8's `WindowClosed`. -**Faults** — EL0 exceptions that are not syscalls — are not return values. The design suspends the thread -and sends a fault message to a handler RFC-0006 (proposed) binds, so a supervisor can restart a driver -(O-17). **Interim:** the reporter prints the fault, prefixed by the thread; the thread stops, the core -continues, nothing restarts it. +**Faults** — EL0 exceptions that are not syscalls — are not return values: the thread is suspended and a +fault message goes to a handler RFC-0006 (proposed) binds, so a supervisor can restart a driver (O-17). +**Interim:** the reporter prints it, prefixed by the thread; the thread stops and the core continues. ## 8. The Process object and spawning (O-27) -A **Process** holds one RFC-0003 capability table, one AddressSpace (RFC-0005 (proposed)) and its threads -(RFC-0006 (proposed)) — a real object *(Zircon)*, not seL4's convention of a thread naming a CSpace and a -VSpace. Three ways to give it its first authority: +A **Process** holds one RFC-0003 table, one AddressSpace (RFC-0005 (proposed)) and its threads (RFC-0006 +(proposed)) — a real object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. First authority: -- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority smuggled across - creation. -- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves one handle; a loader sends the rest as - processargs)*. Minimal, but over Setonix's unbuffered rendezvous the parent must serve its own child a - protocol before the child can act, holding a bootstrap endpoint per child. +- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority across creation. +- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves one handle; a loader sends the rest)*. + Over an unbuffered rendezvous the parent must serve its own child a protocol before the child can act. - **C — explicit grants before start** *(seL4's root task writing a child's CSpace)*. **Verdict sought.** A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, with an **empty table**: @@ -269,135 +243,118 @@ A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, | `GRANT` | `WRITE` on the process; `TRANSFER` on each attached capability | move up to three attached capabilities into the child's table; the reply's MR0–MR2 carry their handle words *in the child* | | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | -`GRANT` is a `call` on the Process with capabilities attached: the kernel takes them from `caps[]` exactly -as an endpoint send does (§5) — fetched once, all resolved before any moves. +`GRANT` is a `call` whose capabilities come from `caps[]` exactly as an endpoint send's do (§5) — fetched +once, all resolved before any moves. **The grant window.** `GRANT` succeeds only until the first `resume` of any thread bound to the process — a one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers -`WindowClosed`. From then on authority arrives only by IPC, which the process consents to by receiving: a -parent cannot reach into a running child's table, and a stolen Process capability cannot inject authority -into a live process. The window is **single-use** in research/0002's sense (Part 3, from Vault's -response-wrapping) but gives none of its **prior-use detection** — the child is not told who granted what. -Inside the kernel the channel is the Process capability, minted to its creator alone; detection across -imperfectly trusted spawn channels belongs to the broker RFC, which owns those channels. - -The child learns its handles as a program learns its arguments: the spawner writes them into `a0` and -`a1` (the demo's console and endpoint), or points `a0` at a start block — a runtime-row convention, not -kernel ABI. No right joins RFC-0003's set: `GRANT`, `KILL` and thread creation all use `WRITE` on that one -Process, held by its creator — not a catch-all. A process holds no capability to itself unless handed one, -and is destroyed when its last thread exits. +`WindowClosed`, and authority arrives only by IPC, which the process consents to by receiving. No parent +reaches into a running child's table; a stolen Process capability injects nothing into a live one. The +window is **single-use** in research/0002's sense (Part 3, from Vault's response-wrapping) but lacks its +**prior-use detection**: the child is not told who granted what. Here the channel is a Process capability +minted to its creator alone; detection across imperfectly trusted spawn channels is the broker RFC's. + +The spawner passes the child's handle words in `a0` and `a1` (the demo's console and endpoint), or points +`a0` at a start block — a runtime convention, not kernel ABI. No right joins RFC-0003's set: `GRANT`, +`KILL` and thread creation use `WRITE` on that one Process. A process dies with its last thread. ## 9. The root task and boot info The one act the kernel performs unasked: at boot it builds **exactly one** process from the first boot -module and gives it every boot capability *(seL4's root task; research/0002 Part 3: "the bootstrap seam is -a one-time kernel act")*: an AddressSpace; the descriptor's segments copied into fresh frames and mapped +module and gives it every boot capability *(seL4's root task; research/0002 Part 3, "the bootstrap seam is +a one-time kernel act")* — an AddressSpace; the descriptor's segments copied into fresh frames and mapped as stated (§10); a stack; an IPC buffer; a read-only **BootInfo** page; a Process and a Thread in §3's -entry state with `a0` = BootInfo's address and `a1` = 0; `eret`. RFC-0006 (proposed) provides the root's -first scheduling context, its parameters a named boot constant — as seL4's root starts at the maximum -priority with one configured timeslice. - -**BootInfo** is kernel ABI, in `user/abi`: a header (magic, version, length) and one entry per initial -capability — `{ kind, flags, handle, base, size }` — with a string table for module names, pinned within -one page by a host test. Every initial capability is *listed*; none sits at a well-known slot, so the -order can change without an ABI break and a reviewer can enumerate the whole bootstrap grant. This RFC -defines three kinds: the root's own Process, the Console (§11), and one read-only Region per further boot -module, with its load descriptor. RFC-0005 (proposed) provides the AddressSpace, the root Pool over all -free RAM and the DTB Region; RFC-0006 (proposed) the Thread, `SchedControl` and interrupt capabilities. - -**The root is not a superuser.** It holds everything at first, as ordinary, droppable, generation-checked -capabilities, and the rule that keeps it so is checkable: **no kernel code branches on "is this the root -task"**. The bootstrap graph is acyclic by construction — kernel to root, root to children — and nothing -the root needs waits on a userspace service. **Rejected:** a kernel boot manifest of programs and grants -(policy in the kernel); a `spawn_from_module` syscall (a loader offered as a kernel service); and a -bootstrap superuser, research/0002's `system:masters`. +entry state, `a0` = BootInfo's address, `a1` = 0; `eret`. Its scheduling context (RFC-0006 (proposed)) +takes a named boot constant, as seL4's root starts at maximum priority with one configured timeslice. + +**BootInfo**, in `user/abi`, is a header (magic, version, length) and one `{ kind, flags, handle, base, +size }` entry per initial capability, with a string table for module names, pinned within a page by a host +test. Every initial capability is *listed*, none at a well-known slot, so the order can change without an +ABI break and a reviewer can enumerate the whole grant. Kinds here: the root's Process, the Console (§11), +a read-only Region per further module with its descriptor; RFC-0005 (proposed) adds the AddressSpace, the +root Pool and the DTB, RFC-0006 (proposed) the Thread, `SchedControl` and interrupt capabilities. + +**The root is not a superuser.** Its capabilities are ordinary, droppable and generation-checked, and the +rule is checkable: **no kernel code branches on "is this the root task"**. The bootstrap graph is acyclic, +and nothing the root needs waits on a userspace service. **Rejected:** a kernel boot manifest (policy in +the kernel); a `spawn_from_module` syscall (a loader as a kernel service); a `system:masters` superuser. ## 10. Getting the first images into memory QEMU's `-kernel` loads one ELF. The candidates for the rest: - **Embed them in the kernel ELF** *(verdict sought)*. One artefact boots; its hash pins every program. -- **QEMU's `-initrd` or `-device loader`.** Two artefacts, QEMU-only — and neither reaches a bare ELF. In - `hw/arm/boot.c` (read at QEMU 8.2.2), `arm_setup_direct_kernel_boot` loads an `-initrd` only for images - it treats as Linux, never an ELF, and for a non-Linux image `do_cpu_reset` sets the PC and nothing else: - no register names the DTB either, which QEMU puts at the base of RAM only if the image leaves room. - Today's image *is* the base of RAM, so it gets none; RFC-0005 (proposed) relinks to `0x4020_0000` for - that. A `-device loader` blob lands where only the command line knows. -- **Link the children into the root's ELF** *(common seL4 practice, a CPIO archive)*. The root becomes - specific to each image, and the children stop being separate immutable objects it can hand on. - -**`xtask` parses ELF; the kernel does not.** `xtask` reads each program's headers with a dependency-free, -host-tested ELF64 reader and emits a fixed-shape **load descriptor**: the entry point; a text segment -(`R+X`), an optional read-only segment (`R`) and a data segment (`R+W`, with a `bss` size), page-aligned in -the user half; the stack, IPC-buffer and BootInfo addresses. It **refuses** at build time any -writable-and-executable, overlapping or unaligned segment, dynamic linking, TLS or any other program -header — W^X checked before the image exists (O-13). The kernel checks one fixed structure against fixed -bounds and never sees an ELF: Nexen's finding that kernel code validating structures built elsewhere is -the bug farm (research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` -crate — small and host-testable, but a parser in the TCB for no gain. - -**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned `abi` header and per +- **QEMU's `-initrd` or `-device loader`.** Two artefacts, QEMU-only, and neither reaches a bare ELF. In + `hw/arm/boot.c` (read at QEMU 8.2.2), `arm_setup_direct_kernel_boot` loads `-initrd` only for images it + treats as Linux, never an ELF; for a non-Linux image `do_cpu_reset` sets the PC and nothing else, so no + register names the DTB either, which goes at the base of RAM only if the image leaves room — ours does + not, hence RFC-0005 (proposed)'s relink to `0x4020_0000`. A loader blob lands where only QEMU's command + line knows. +- **Link the children into the root's ELF** *(seL4 practice, a CPIO archive)*. The root becomes specific + to each image, and the children stop being separate immutable objects it can hand on. + +**`xtask` parses ELF; the kernel does not.** A dependency-free, host-tested ELF64 reader in `xtask` emits +a fixed-shape **load descriptor** per program: entry point; a text segment (`R+X`), an optional read-only +segment (`R`), a data segment (`R+W`, with a `bss` size), page-aligned in the user half; stack, IPC-buffer +and BootInfo addresses. It **refuses** at build time any writable-and-executable, overlapping or unaligned +segment, dynamic linking, TLS or other program header — W^X checked before the image exists (O-13). The +kernel checks one fixed structure against fixed bounds and never sees an ELF: Nexen's finding that kernel +code validating structures built elsewhere is the bug farm (research/0002 Part 7), applied to our own boot +path. **Rejected:** an in-kernel `elf/` crate — small and testable, but a parser in the TCB for no gain. + +**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned `abi` header, then per module a name, descriptor and segment bytes — and builds the kernel with `SETONIX_PAYLOAD` naming it; `kernel/build.rs` places it page-aligned in a read-only `.payload` section (`KEEP`). **Unset, it emits an empty bundle**, so a bare `cargo build` or `cargo clippy --package setonix-kernel` still links, boots and -greets, printing `boot modules: 0` before halting as today. *(Hubris's `xtask dist`; seL4's elfloader, -which parses before the kernel runs.)* - -The kernel sees modules through one arch-neutral function, `arch::boot_modules()`: the embedded bundle on -AArch64, files the UEFI stub loaded on x86_64 — so embedding is Phase 1's *source*, and the kernel's -contract is a range plus descriptors. It loads **entry 0 only**; every other entry reaches the root as a -**read-only Region** over its pages (RFC-0005 (proposed) provides the frames) with its descriptor in -BootInfo, so the root copies segments out without parsing anything and the image stays immutable -(pillar 1). *(Genode's core hands boot modules to init as read-only ROM modules.)* - -**The programs** are `no_std`, `no_main` workspace crates — `user/demo/server`, `user/demo/client` — -depending only on `abi`, with no `unsafe`, linked static and non-PIE by `user/link/.ld` in RFC-0005 -(proposed)'s user half. `x86_64-unknown-none` defaults to position-independent output and the *kernel* -code model, so `xtask` passes `-C relocation-model=static -C code-model=small` for user crates, in their -own target directory; the AArch64 target is static already. They stay soft-float, as the demo needs. +greets, printing `boot modules: 0`. *(Hubris's `xtask dist`; seL4's elfloader, parsing before the +kernel runs.)* The kernel reaches modules through `arch::boot_modules()` — the bundle on AArch64, files +the UEFI stub loaded on x86_64 — so embedding is Phase 1's *source*, not the design. It loads **entry 0 +only**; the rest reach the root as **read-only Regions** with descriptors in BootInfo, so the root parses +nothing and the images stay immutable. *(Genode's core hands boot modules to init read-only.)* -## 11. The console as a capability +**The programs** — `user/demo/server`, `user/demo/client` — are soft-float `no_std` crates on `abi` alone, +with no `unsafe`, linked static by `user/link/.ld` in the user half; `x86_64-unknown-none` defaults +to PIE and the *kernel* code model, so `xtask` passes `-C relocation-model=static -C code-model=small`. -Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, -by `call`: the byte count in MR0, the bytes from MR1 on — up to 504 per call, so a short line never leaves -the registers — needing the `WRITE` right. The root receives it with `DUPLICATE | TRANSFER | WRITE` (no -`READ`: there is no input path) and derives `WRITE`-only copies for the children it chooses. A program not -handed one cannot print, which is the point. +## 11. The console as a capability +Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, by +`call`: the byte count in MR0, the bytes from MR1 on — up to 504, so a short line never leaves the +registers. The root receives it with `DUPLICATE | TRANSFER | WRITE` (no `READ`: there is no input path) +and derives `WRITE`-only copies for the children it chooses. A program not handed one cannot print. **Rejected:** seL4's `DebugPutChar`, which any thread may issue in a `CONFIG_PRINTING` build — ambient authority, however small — and a console at a well-known handle. **Adopted in spirit:** Zircon's debuglog, -a handle-rights object write-only by default. Because §4 makes invocation uniform, once a userspace UART -driver exists the root hands out an endpoint speaking `WRITE`, no client changes, and the Console object is -**deleted** (increment 13). **Interim, named:** until then a device path inside the kernel is reachable -from userspace — tiny, write-only, exercised by every boot-test — and on real hardware at 115,200 baud a -504-byte write holds the non-preemptible kernel for about 44 ms, a bound RFC-0006's latency budget carries. +write-only by default. Because invocation is uniform, once a userspace UART driver exists the root hands +out an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** (increment 13). +**Interim, named:** until then a device path in the kernel is reachable from EL0 — tiny, write-only, +exercised by every boot-test — and at 115,200 baud a 504-byte write holds the non-preemptible kernel about +44 ms on real hardware, a bound RFC-0006's latency budget carries. ## 12. `user/abi` — the kernel's own binding -**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment: -allocation, formatting, start-up conventions, POSIX-shaped calls. It does not issue syscalls. relibc sits -on `redox_syscall`, versioned with the Redox kernel; seL4 keeps libsel4 in the kernel's repository, -generated from the kernel's own interface specifications, with runtimes elsewhere. The binding is the user -half of §3's table — there is nothing to port it from. **Verdict sought: the microkernel-core row, write -ourselves.** A future relibc port adds a platform layer *above* it; anything in `user/abi` that would read -the same on another kernel moves out. The maintainer may wish to append "syscall ABI binding -(`user/abi`)" to that row — constitution text, and his. If he reads it as the runtime row instead, it is -write-ourselves-by-necessity there, and that row's "port code" verdict needs a recorded exception. - -It holds syscall numbers; `Handle`, `Tag` and `Status`; the `IpcBuffer`, `BootInfo`, bundle and -descriptor layouts, with `const` layout assertions; and safe wrappers for the twelve syscalls and each -kernel object's methods — no allocator, no formatting. **One source of truth:** the kernel compiles the -data half with `default-features = false`, so the `syscalls` feature gating `src/arch/**` stays off, and -`xtask` uses it for the bundle; kernel, tools and userspace compile the same constants. +**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment — +allocation, formatting, start-up, POSIX-shaped calls — and issues no syscalls. relibc sits on +`redox_syscall`, versioned with the Redox kernel; seL4 keeps libsel4 in the kernel's repository, generated +from the kernel's own interface specifications. The binding is the user half of §3's table, with nothing to +port it from. **Verdict sought: the microkernel-core row, write ourselves.** A relibc port adds a layer +*above* it; anything here that would read the same on another kernel moves out. The maintainer may append +"syscall ABI binding (`user/abi`)" to that row — constitution text, and his; if he reads it as the runtime +row instead, it is write-ourselves-by-necessity there, and that row's verdict needs a recorded exception. + +It holds syscall numbers; `Handle`, `Tag` and `Status`; the `IpcBuffer`, `BootInfo`, bundle and descriptor +layouts with `const` assertions; safe wrappers for the syscalls and each kernel object's methods — no +allocator, no formatting. **One source of truth:** the kernel compiles the data half with +`default-features = false`, keeping the `syscalls` feature that gates `src/arch/**` off, and `xtask` uses it +for the bundle, so kernel, tools and userspace compile the same constants. **The third designated `unsafe` tree**, approved in principle on 2026-09-27, is proposed as -`user/abi/src/arch/**`: the `svc`/`syscall` assembly; the `_start` stub, which turns `a0` into the root's -`&'static BootInfo` and calls `main`; and accessors whose soundness rests on a kernel promise about this -process's memory — the IPC-buffer view, and the slice over a Region just mapped into the caller (the root's -loader writes child segments through it). User programs contain no `unsafe`. **The `CLAUDE.md` edit** -adds to § `unsafe` policy "`user/abi/src/arch/**` — trap instructions, the process entry stub and the -kernel-shared memory the ABI defines", adds `user/` to § Layout and changes `xtask`'s "no deps" to "no -external deps"; the `grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. +`user/abi/src/arch/**`: the `svc`/`syscall` assembly; `_start`, which turns `a0` into the root's +`&'static BootInfo` and calls `main`; and accessors resting on a kernel promise about this process's +memory — the IPC-buffer view, and the slice over a Region just mapped into the caller (the root's loader +writes child segments through it). User programs contain no `unsafe`. **The `CLAUDE.md` edit** adds to +§ `unsafe` policy "`user/abi/src/arch/**` — trap instructions, the process entry stub and the kernel-shared +memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no external deps"; the +`grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. ## 13. Lineage @@ -432,72 +389,66 @@ external deps"; the `grep -rn "allow(unsafe_code)"` audit widens from `kernel/sr is the root's. The rejected manifest and `spawn_from_module` are this grave's two doors; the root's boot scheduling constant is the one named residue. - **The catch-all right.** No root identity survives `eret`; no right is added; `WRITE` on a Process is - authority over one object. The root holds much, but droppably and enumerably. -- **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; the slow path - copies once, frame to frame. -- **Bolted-on multicore.** Entry state is per-CPU from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`); - single fetch is written for a second core rewriting the buffer; the syscall path takes no global lock. - Two cores on one table stays RFC-0003 §14.2's question. -- **Drivers pulled in; unused device paths.** The Console is the kernel's own diagnostic UART, write-only, - used by every boot-test and scheduled for deletion; it is the only device method reachable from EL0. + authority over one object, and the root's many capabilities are droppable and enumerable. +- **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; one copy. +- **Bolted-on multicore.** Entry is per-CPU from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`); single + fetch is written for a second core; no global lock. Two cores on one table stays RFC-0003 §14.2's. +- **Drivers pulled in; unused device paths.** The Console is the only device method reachable from EL0: + write-only, used by every boot-test, its deletion scheduled. ## 16. Costs — what this makes harder -- **`call` is overloaded:** per-type method decoding and two error layers — the price of interposition. -- **Four physical registers:** past 32 bytes a message touches memory, and a 65–120-word message seL4 - would copy needs a Region here; raising either count later breaks binaries. -- **Handles are 20/44 for good,** and the accepted `capability` crate changes twice. -- **Two thread registers belong to the ABI:** `TPIDRRO_EL0` and the user GS base are not a runtime's. -- **The root task is the most sensitive process:** until it drops authority, a bug there compromises the +- **`call` is overloaded** — per-type method decoding and two error layers, the price of interposition. +- **Four physical registers and 64 words:** past 32 bytes a message touches memory, and a 65–120-word + message seL4 would copy needs a Region here; raising either later breaks binaries. +- **Handles are 20/44 for good,** the accepted `capability` crate changes twice, and `TPIDRRO_EL0` and + the user GS base belong to the ABI rather than a runtime. +- **The root task is the most sensitive process;** until it drops authority a bug there compromises the system, as in seL4. It must stay small. -- **No ambient printing,** so a process without a Console fails silently; **late grants need a willing - child**, and the grant window detects no prior use. -- **Embedding couples builds:** a user change relinks the kernel, the kernel's reproducibility includes the - programs', and `xtask` gains its first, in-workspace dependency. -- **No PAN on the demo CPU,** and a canonical check on every x86_64 `sysretq`. +- **No ambient printing; late grants need a willing child;** the grant window detects no prior use. +- **Embedding couples builds:** a user change relinks the kernel, whose reproducibility now includes the + programs'; `xtask` gains its first, in-workspace dependency. **No PAN on the demo CPU.** ## 17. Open questions 1. **Fault messages** — their layout (this RFC's) and the thread's fault binding (RFC-0006's). -2. **Observing death** — a supervisor must learn a process died (O-17); a notification bound at creation - is the likely shape. +2. **Observing death** — a supervisor must learn a process died (O-17); likely a notification bound at creation. 3. **Badges** — `cap_derive`'s badge word and the badge register are reserved zero until RFC-0003a. 4. **The console after Phase 2** — whether any Console capability stays mintable once a driver owns the UART. -5. **The `unsafe` tree's name** — two of its items are memory, not architecture; the maintainer may prefer - a sibling `src/kernel_mem/`. +5. **The `unsafe` tree's name** — two of its items are memory, not architecture; perhaps `src/kernel_mem/`. 6. **The 64-word budget** — a verdict now, to be revisited with the first measurements. ## 18. Implementation increments Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect -prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo's interims, each -costed above: faults print and stop (§7); the Console is a device path in the kernel (§11); modules come -from the kernel image (§10); RFC-0005's static object pools; the root's boot scheduling constant (§9). The -root doubling as the server is not an interim: it is userspace composition, and yields the two processes. +prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo's interims, costed +above: faults print and stop (§7); the Console is a device path in the kernel (§11); modules come from the +kernel image (§10); RFC-0005's static object pools; the root's boot scheduling constant (§9). The root +doubling as the server is not an interim — it is userspace composition, and yields the two processes. 1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, the slot bound, `ObjectDestroyed`. Host tests: round trips; word 0 and index ≥ 2^20 rejected; a slot retiring at the bound while an object generation passes it; use-after-close and destruction told apart. 2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, `IpcBuffer`, `BootInfo`, bundle and - descriptor types, `const` layout assertions, no `unsafe`. Host tests reject every reserved-bit and - over-length encoding; CI builds it for both targets. + descriptor types, layout assertions, no `unsafe`. Host tests reject every reserved-bit and over-length + encoding; CI builds it for both targets. 3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatcher, restore, `eret`; `yield`, `thread_exit`, `InvalidSyscall`; faults stop the thread, not the core. A feature like - `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off — the softfloat target is - `+strict-align`, so EL0's Device-memory accesses are safe — and runs `svc #0`. Boot-test - `--features syscall-selftest --expect "syscall round trip"`. Lands before RFC-0005's first mapping. + `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off (the softfloat target is + `+strict-align`, so EL0's Device-memory accesses are safe) and runs `svc #0`. Boot-test + `--features syscall-selftest --expect "syscall round trip"`, before RFC-0005's first mapping exists. 4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer-register read — **with the `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first run by increment 7. -5. **[demo] The bundle** — `xtask`'s ELF reader and descriptor writer, host-tested on writable-and-executable, - overlapping, truncated and wrong-machine fixtures; `build.rs`, `.payload`. Boot-tests: a bare build - `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. +5. **[demo] The bundle** — `xtask`'s ELF reader and descriptor writer, host-tested on writable-and- + executable, overlapping, truncated and wrong-machine fixtures; `build.rs`, `.payload`. Boot-tests: a + bare build `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. 6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static interim tables. Boot-test `--features syscall-selftest --expect "[el0] Kaya!"`, printed from EL0 through a Console handle. 7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads: construction, BootInfo, entry state (§9). Boot-test `--expect "[server] up"`. 8. **[demo] The Process object** — grant window and teardown as pure logic in a host-tested `process/` - crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first resume; - `KILL` empties the table. + crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first + resume; `KILL` empties the table. 9. **[demo] The root spawns the client,** granting a `WRITE`-only Console and the endpoint. The client prints `[client] up`, invokes a handle word it was never granted, receives `InvalidHandle` and prints `[client] denied, as expected` — O-27 proved observably, not asserted. @@ -514,5 +465,5 @@ root doubling as the server is not an interim: it is userspace composition, and The first userspace, and with it every Phase-2 paper: the **scheme registry** and the **broker** (built from processes, grants and `call`), the **driver framework** (fault messages, interrupt notifications, the -console moving out), a **relibc port** (a platform layer over `user/abi`, both TLS registers still its -own), and the **x86_64 bring-up**, whose trap path is on paper. RFC-0003a gains an ABI to place badges in. +console moving out), a **relibc port** (a layer over `user/abi`, both TLS registers still its own), and the +**x86_64 bring-up**, whose trap path is on paper. RFC-0003a gains an ABI to place badges in. From 4fc242f7f1a2c5e6f53307f26817499b1e7608c2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:13:33 +0000 Subject: [PATCH 07/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0007-system-calls-and-process-bootstrap.md | 71 +++++++++---------- 1 file changed, 32 insertions(+), 39 deletions(-) diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index cae37bd..53d02d1 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -80,7 +80,9 @@ RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, and the exit path check Every register outside the result set is restored from the frame; the eight result registers carry a result or zero, so no kernel value leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux; the x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and what `syscall` destroys. -seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. +seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. **No argument +is a pointer:** only the IPC buffer is touched, by frame (§5) — on the demo CPU the whole defence, since +the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). **Entry state of every thread,** root included: PC = entry; SP as given; `a0`, `a1` — `x0`/`x1`, `rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — the two start values; every @@ -89,9 +91,6 @@ FP/SIMD trapped (`CPACR_EL1.FPEN` = 0; `CR0.TS` set) until RFC-0006 (proposed) e and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is created so; the spawner writes PC, SP, `a0` and `a1` through RFC-0006 (proposed)'s `write_regs`, and the kernel writes the root's (§9). -**No syscall argument is a pointer.** The syscall path touches user memory only in the IPC buffer, by -frame (§5) — on the demo CPU the whole defence, since the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). - ## 4. The syscall set - **A — one syscall per operation** *(Zircon, Linux)*. Legible, but every object type grows the table, @@ -100,10 +99,9 @@ frame (§5) — on the demo CPU the whole defence, since the Cortex-A72 (Armv8.0 `call` on its capability, the method in the tag's label, answered as a server would answer. - **C — B plus an `invoke` number.** The small table without the point: callers can tell the two apart. -**Verdict sought: B.** The payoff is interposition: a Console and an endpoint to a logging server are -invoked alike, so Phase 2's UART driver takes over the console protocol without a client changing, and -the broker can hand out a forwarder for any kernel object — RFC-0003a option (a)'s precondition. A `call` -on a kernel object runs on the caller's own time: nothing is donated, nothing blocks. +**Verdict sought: B,** for interposition: a Console and a logging server's endpoint are invoked alike, so +Phase 2's UART driver replaces the Console with no client changing, and the broker can hand out a +forwarder for any kernel object (RFC-0003a option (a)'s precondition). Such a `call` donates nothing. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| @@ -385,9 +383,8 @@ memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no externa ## 15. Graves checked (§3) -- **Policy in the kernel.** The kernel runs one image and lists what it granted; what runs, holding what, - is the root's. The rejected manifest and `spawn_from_module` are this grave's two doors; the root's boot - scheduling constant is the one named residue. +- **Policy in the kernel.** The kernel runs one image and lists what it granted; the rest is the root's. + The manifest and `spawn_from_module` were this grave's doors; the root's boot constant is its residue. - **The catch-all right.** No root identity survives `eret`; no right is added; `WRITE` on a Process is authority over one object, and the root's many capabilities are droppable and enumerable. - **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; one copy. @@ -403,9 +400,8 @@ memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no externa message seL4 would copy needs a Region here; raising either later breaks binaries. - **Handles are 20/44 for good,** the accepted `capability` crate changes twice, and `TPIDRRO_EL0` and the user GS base belong to the ABI rather than a runtime. -- **The root task is the most sensitive process;** until it drops authority a bug there compromises the - system, as in seL4. It must stay small. -- **No ambient printing; late grants need a willing child;** the grant window detects no prior use. +- **The root task is the most sensitive process,** as in seL4; it must stay small. **No ambient + printing; late grants need a willing child;** the grant window detects no prior use. - **Embedding couples builds:** a user change relinks the kernel, whose reproducibility now includes the programs'; `xtask` gains its first, in-workspace dependency. **No PAN on the demo CPU.** @@ -421,44 +417,41 @@ memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no externa ## 18. Implementation increments Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect -prefixed strings, since bare `Kaya!` already matches the kernel's greeting. The demo's interims, costed -above: faults print and stop (§7); the Console is a device path in the kernel (§11); modules come from the -kernel image (§10); RFC-0005's static object pools; the root's boot scheduling constant (§9). The root -doubling as the server is not an interim — it is userspace composition, and yields the two processes. +prefixed strings, since bare `Kaya!` matches the kernel's greeting. The demo's interims, costed above: +faults print and stop (§7); the in-kernel Console (§11); modules in the kernel image (§10); RFC-0005's +static object pools; the root's boot scheduling constant (§9). The root doubling as server is composition. 1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, the slot bound, `ObjectDestroyed`. - Host tests: round trips; word 0 and index ≥ 2^20 rejected; a slot retiring at the bound while an - object generation passes it; use-after-close and destruction told apart. -2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, `IpcBuffer`, `BootInfo`, bundle and - descriptor types, layout assertions, no `unsafe`. Host tests reject every reserved-bit and over-length - encoding; CI builds it for both targets. -3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatcher, restore, `eret`; `yield`, + Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot retires at the bound while an object + generation passes it; use-after-close and destruction told apart. +2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, the layouts and their assertions, no + `unsafe`. Host tests refuse every reserved-bit and over-length encoding; CI builds both targets. +3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatch, restore, `eret`; `yield`, `thread_exit`, `InvalidSyscall`; faults stop the thread, not the core. A feature like `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off (the softfloat target is - `+strict-align`, so EL0's Device-memory accesses are safe) and runs `svc #0`. Boot-test - `--features syscall-selftest --expect "syscall round trip"`, before RFC-0005's first mapping exists. + `+strict-align`, so Device-memory accesses are safe) and runs `svc #0`: `--expect "syscall round trip"`. 4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer-register read — **with the `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first run by increment 7. -5. **[demo] The bundle** — `xtask`'s ELF reader and descriptor writer, host-tested on writable-and- - executable, overlapping, truncated and wrong-machine fixtures; `build.rs`, `.payload`. Boot-tests: a - bare build `--expect "boot modules: 0"`; `cargo xtask` `--expect "boot modules: 2"`. -6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static interim tables. Boot-test - `--features syscall-selftest --expect "[el0] Kaya!"`, printed from EL0 through a Console handle. -7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads: construction, - BootInfo, entry state (§9). Boot-test `--expect "[server] up"`. +5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on writable-and-executable, overlapping, + truncated and wrong-machine fixtures; `build.rs`. Boot-tests: bare build `--expect "boot modules: 0"`, + `cargo xtask` `--expect "boot modules: 2"`. +6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static tables; the self-test prints from EL0 + through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. +7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads (§9): + `--expect "[server] up"`. 8. **[demo] The Process object** — grant window and teardown as pure logic in a host-tested `process/` crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first resume; `KILL` empties the table. -9. **[demo] The root spawns the client,** granting a `WRITE`-only Console and the endpoint. The client - prints `[client] up`, invokes a handle word it was never granted, receives `InvalidHandle` and prints +9. **[demo] The root spawns the client** with a `WRITE`-only Console and the endpoint. The client prints + `[client] up`, invokes a handle word it was never granted, gets `InvalidHandle` and prints `[client] denied, as expected` — O-27 proved observably, not asserted. -10. **[demo] IPC syscalls** over RFC-0004 endpoints with RFC-0006's blocking states. Boot-test lines in +10. **[demo] IPC syscalls** over RFC-0004 endpoints and RFC-0006's blocking states. Boot-test lines in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. RFC-0006's increments add the preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. -11. **Fault messages and death notification** (§17.1–2). Boot-test: a deliberately faulting child is - reported to the root, which prints `[server] child faulted`. +11. **Fault messages and death notification** (§17.1–2): a faulting child is reported to the root, + which prints `[server] child faulted`. 12. **x86_64 trap path and `user/abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the - canonical check with `iretq` fallback. Proves §3's second column links; boots with the UEFI stub. + canonical check. Proves §3's second column links; boots with the UEFI stub. 13. **Delete the Console object** when the Phase-2 UART driver lands — scheduled now so it stays interim. ## 19. What this unblocks From f6e05cdf0dfcbb920fe93e20f212c49473165873 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:16:47 +0000 Subject: [PATCH 08/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0007-system-calls-and-process-bootstrap.md | 119 ++++++++---------- 1 file changed, 53 insertions(+), 66 deletions(-) diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index 53d02d1..f1670fc 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -32,10 +32,8 @@ such that every request names a capability, no argument is trusted, and no proce authority it was not explicitly handed?** RFC-0003 §9 fixed the rule and RFC-0004 §10 the IPC semantics; both left the encoding here. This RFC -designs the encoding, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 -named: minting the first capabilities without an ambient grantor. It does not design address spaces -(RFC-0005), threads or scheduling (RFC-0006), or any policy about which programs run holding what — that -is the root task's, in userspace, and later the broker's. +designs it, the trap, the process as a kernel object, and the bootstrap seam RFC-0003 §14.4 named. It does +not design address spaces (RFC-0005), threads (RFC-0006), or any policy on which programs run holding what. ## 2. Which pillar @@ -118,10 +116,9 @@ forwarder for any kernel object (RFC-0003a option (a)'s precondition). Such a `c | 10 | `cap_identify` | object | out: MR0 the object kind (an `abi` enumeration), MR1 the rights bits | | 11 | `thread_exit` | — | halt the calling thread; authority-free | -`reply_recv` fuses two RFC-0004 operations so a server loop is one entry, as direct switch needs. The -`cap_*` calls take and return a zero tag and act only on the caller's own table; an undefined rights bit -answers `InvalidArgument`, and the dispatcher builds `Rights` from the crate's named constants, which -still never accept raw bits. Transfer is a move; a sender keeping a copy derives first (RFC-0003 §6). +`reply_recv` makes a server loop one entry, as direct switch needs. The `cap_*` calls take and return a +zero tag and touch only the caller's own table; the dispatcher builds `Rights` from the crate's named +constants, refusing undefined bits. Transfer is a move; a sender keeping a copy derives first (RFC-0003 §6). **Creation** is a method on a Pool, which RFC-0005 (proposed) provides; per-type parameters are encoded here, and an endpoint's carries **`min_budget`** in MR1 (RFC-0006 (proposed)'s time unit, 0 for none, fixed for the endpoint's life). **Rejected:** a `debug_putc` (§11); anything taking a PID, path or @@ -159,13 +156,12 @@ the discovery register and is never dereferenced. Received capabilities take fre **Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's address into `TPIDRRO_EL0` (readable at EL0, writable only at EL1) and, on x86_64, into the user GS base, with -`CR4.FSGSBASE` off (RFC-0006 (proposed)) and `abi` reading the self-pointer at `gs:[0]` — the x86_64 ELF -TLS convention of a block whose first word is its own address *(Drepper, "ELF Handling For Thread-Local -Storage")*. RFC-0006 (proposed) already restores both on every switch and leaves their use here, so -`_start` has its buffer at its first instruction, in any thread, with no TLS runtime. The kernel trusts -neither: a thread that reloads its own GS selector misleads only itself. **Rejected:** `TPIDR_EL0` and -`FS_BASE`, the ELF TLS thread pointers a relibc port needs untouched; and seL4's current answer, a -thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`) — sound, but it needs TLS before any syscall. +`CR4.FSGSBASE` off (RFC-0006 (proposed)) and `abi` reading the self-pointer at `gs:[0]` — the ELF TLS +convention of a block whose first word is its own address *(Drepper, "ELF Handling For Thread-Local +Storage")*. RFC-0006 (proposed) restores both on every switch, so `_start` has its buffer at its first +instruction with no TLS runtime; the kernel trusts neither, and a thread reloading GS misleads only +itself. **Rejected:** `TPIDR_EL0` and `FS_BASE`, the ELF TLS thread pointers a relibc port needs; and +seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`), which needs TLS first. **Single fetch (O-5).** Once SMP exists another thread can rewrite the buffer mid-syscall, so every buffer word the kernel acts on is read **exactly once** into kernel memory before any check depends on it. @@ -176,18 +172,16 @@ formed into the buffer. *(research/0002 Part 5, virtio's double-fetch lesson.)* ## 6. Handles in one register -RFC-0003's `Handle` is a `u32` index and a 64-bit generation, which fit no register. The ABI word is -**`slot_generation << 20 | index`**: 2^20 slots, Linux's default descriptor ceiling (`fs.nr_open` = -1,048,576); 2^44 slot generations, about two hundred days of one close-and-reopen per microsecond on one -slot before it retires. `Generation::FIRST` is 1, so word `0` is never live and is null. This is the width -RFC-0003's 2026-07-30 amendment asked to be stated. **Only the slot generation crosses the ABI;** object -generations stay in kernel memory. So the accepted crate changes narrowly: `Handle::to_word`/`from_word` -and `Handle::MAX_SLOT_GENERATION` (2^44 − 1); `CapabilityTable::remove` retires a slot whose next -generation would exceed it, through its existing `Retired` state; a const assertion bounds every kernel -table's `N` by 2^20. `Generation` keeps 64 bits for objects, and its doc comment gains the slot bound. -Phase 1 tables have `N` = 64, fixed, failing closed. **Rejected:** two registers per handle (halves the -budget); 32/32 (2^32 reuses is seventy minutes at one per microsecond, and the LIFO free list -concentrates reuse on one slot); a 16-bit index (too few for the server workloads the constitution targets). +RFC-0003's `Handle` — a `u32` index and a 64-bit generation — fits no register. The ABI word is +**`slot_generation << 20 | index`**: 2^20 slots, Linux's default `fs.nr_open` (1,048,576); 2^44 slot +generations, about two hundred days of one close-and-reopen per microsecond on one slot. As +`Generation::FIRST` is 1, word `0` is never live: it is null — the width RFC-0003's amendment asked for. +**Only the slot generation crosses the ABI;** object generations stay in kernel memory and keep 64 bits. +The crate gains `Handle::to_word`/`from_word` and `MAX_SLOT_GENERATION` (2^44 − 1); `remove` retires a +slot whose next generation would exceed it, through its existing `Retired` state; a const assertion bounds +`N` by 2^20; `Generation`'s doc comment gains the slot bound. Phase 1 tables have `N` = 64, failing closed. +**Rejected:** two registers per handle (halves the budget); 32/32 (seventy minutes at the same rate, and +the LIFO free list concentrates reuse on one slot); a 16-bit index (too few for server workloads). ## 7. The error model @@ -215,9 +209,8 @@ and one changed return, and a holder observes destruction without its own bug be the budget statuses arise is RFC-0006 (proposed)'s to define; its draft calls the first `BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. -**Method result** (the reply label) is what the object said — the kernel's for kernel objects, the -server's for endpoints — in one vocabulary, so a stand-in server reports errors the same way *(seL4 returns -invocation errors in the reply's label)*. It reuses status numbers where meanings coincide and adds +**Method result** (the reply label) is what the object said, in one vocabulary for kernel and server, so a +stand-in reports errors alike *(seL4's reply label)*: the status numbers where meanings coincide, plus `InvalidMethod`, RFC-0005 (proposed)'s `PoolExhausted` and `Fragmented`, and §8's `WindowClosed`. **Faults** — EL0 exceptions that are not syscalls — are not return values: the thread is suspended and a @@ -241,16 +234,15 @@ A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, | `GRANT` | `WRITE` on the process; `TRANSFER` on each attached capability | move up to three attached capabilities into the child's table; the reply's MR0–MR2 carry their handle words *in the child* | | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | -`GRANT` is a `call` whose capabilities come from `caps[]` exactly as an endpoint send's do (§5) — fetched -once, all resolved before any moves. - -**The grant window.** `GRANT` succeeds only until the first `resume` of any thread bound to the process — -a one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers -`WindowClosed`, and authority arrives only by IPC, which the process consents to by receiving. No parent -reaches into a running child's table; a stolen Process capability injects nothing into a live one. The -window is **single-use** in research/0002's sense (Part 3, from Vault's response-wrapping) but lacks its -**prior-use detection**: the child is not told who granted what. Here the channel is a Process capability -minted to its creator alone; detection across imperfectly trusted spawn channels is the broker RFC's. +`GRANT` is a `call` whose capabilities come from `caps[]` as an endpoint send's do (§5): fetched once, +all resolved before any moves. **The grant window.** `GRANT` succeeds only until the first `resume` of any +thread bound to the process — a one-way event RFC-0006 (proposed) reports by setting a flag on the +Process; afterwards it answers `WindowClosed`, and authority arrives only by IPC, which the process +consents to by receiving. No parent reaches into a running child's table; a stolen Process capability +injects nothing into a live one. The window is **single-use** in research/0002's sense (Part 3, from +Vault's response-wrapping) but lacks its **prior-use detection**: the child is not told who granted what. +Here the channel is a Process capability minted to its creator alone; detection across imperfectly +trusted spawn channels is the broker RFC's. The spawner passes the child's handle words in `a0` and `a1` (the demo's console and endpoint), or points `a0` at a start block — a runtime convention, not kernel ABI. No right joins RFC-0003's set: `GRANT`, @@ -286,8 +278,7 @@ QEMU's `-kernel` loads one ELF. The candidates for the rest: `hw/arm/boot.c` (read at QEMU 8.2.2), `arm_setup_direct_kernel_boot` loads `-initrd` only for images it treats as Linux, never an ELF; for a non-Linux image `do_cpu_reset` sets the PC and nothing else, so no register names the DTB either, which goes at the base of RAM only if the image leaves room — ours does - not, hence RFC-0005 (proposed)'s relink to `0x4020_0000`. A loader blob lands where only QEMU's command - line knows. + not, hence RFC-0005 (proposed)'s relink to `0x4020_0000`. A loader blob's address is the command line's. - **Link the children into the root's ELF** *(seL4 practice, a CPIO archive)*. The root becomes specific to each image, and the children stop being separate immutable objects it can hand on. @@ -307,8 +298,9 @@ empty bundle**, so a bare `cargo build` or `cargo clippy --package setonix-kerne greets, printing `boot modules: 0`. *(Hubris's `xtask dist`; seL4's elfloader, parsing before the kernel runs.)* The kernel reaches modules through `arch::boot_modules()` — the bundle on AArch64, files the UEFI stub loaded on x86_64 — so embedding is Phase 1's *source*, not the design. It loads **entry 0 -only**; the rest reach the root as **read-only Regions** with descriptors in BootInfo, so the root parses -nothing and the images stay immutable. *(Genode's core hands boot modules to init read-only.)* +only**; the rest reach the root as **read-only Regions** (RFC-0005 (proposed) provides the frames) with +descriptors in BootInfo, so the root parses nothing and the images stay immutable. *(Genode's core hands +boot modules to init read-only.)* **The programs** — `user/demo/server`, `user/demo/client` — are soft-float `no_std` crates on `abi` alone, with no `unsafe`, linked static by `user/link/.ld` in the user half; `x86_64-unknown-none` defaults @@ -339,11 +331,9 @@ port it from. **Verdict sought: the microkernel-core row, write ourselves.** A r "syscall ABI binding (`user/abi`)" to that row — constitution text, and his; if he reads it as the runtime row instead, it is write-ourselves-by-necessity there, and that row's verdict needs a recorded exception. -It holds syscall numbers; `Handle`, `Tag` and `Status`; the `IpcBuffer`, `BootInfo`, bundle and descriptor -layouts with `const` assertions; safe wrappers for the syscalls and each kernel object's methods — no -allocator, no formatting. **One source of truth:** the kernel compiles the data half with -`default-features = false`, keeping the `syscalls` feature that gates `src/arch/**` off, and `xtask` uses it -for the bundle, so kernel, tools and userspace compile the same constants. +It holds the numbers, `Handle`, `Tag`, `Status`, the ABI layouts with `const` assertions, and safe +wrappers — no allocator, no formatting. **One source of truth:** the kernel (with the `syscalls` feature +gating `src/arch/**` off) and `xtask` compile the same data half as userspace. **The third designated `unsafe` tree**, approved in principle on 2026-09-27, is proposed as `user/abi/src/arch/**`: the `svc`/`syscall` assembly; `_start`, which turns `a0` into the root's @@ -407,12 +397,12 @@ memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no externa ## 17. Open questions -1. **Fault messages** — their layout (this RFC's) and the thread's fault binding (RFC-0006's). -2. **Observing death** — a supervisor must learn a process died (O-17); likely a notification bound at creation. -3. **Badges** — `cap_derive`'s badge word and the badge register are reserved zero until RFC-0003a. -4. **The console after Phase 2** — whether any Console capability stays mintable once a driver owns the UART. -5. **The `unsafe` tree's name** — two of its items are memory, not architecture; perhaps `src/kernel_mem/`. -6. **The 64-word budget** — a verdict now, to be revisited with the first measurements. +1. **Fault messages and death** — their layout (this RFC's), the fault binding (RFC-0006's), and how a + supervisor learns a process died (O-17); likely a notification bound at creation. +2. **Badges** — `cap_derive`'s badge word and the badge register are reserved zero until RFC-0003a. +3. **The console after Phase 2** — whether any Console stays mintable once a driver owns the UART. +4. **The `unsafe` tree's name** — two of its items are memory, not architecture: `src/kernel_mem/`? +5. **The 64-word budget** — a verdict now, to be revisited with the first measurements. ## 18. Implementation increments @@ -427,18 +417,16 @@ static object pools; the root's boot scheduling constant (§9). The root doublin 2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, the layouts and their assertions, no `unsafe`. Host tests refuse every reserved-bit and over-length encoding; CI builds both targets. 3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatch, restore, `eret`; `yield`, - `thread_exit`, `InvalidSyscall`; faults stop the thread, not the core. A feature like + `thread_exit`, `InvalidSyscall`; a lower-EL fault stops the thread, not the kernel. A feature like `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off (the softfloat target is `+strict-align`, so Device-memory accesses are safe) and runs `svc #0`: `--expect "syscall round trip"`. 4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer-register read — **with the `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first run by increment 7. -5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on writable-and-executable, overlapping, - truncated and wrong-machine fixtures; `build.rs`. Boot-tests: bare build `--expect "boot modules: 0"`, - `cargo xtask` `--expect "boot modules: 2"`. +5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on W+X, overlapping, truncated and foreign + fixtures; `build.rs`. Boot-tests: bare `--expect "boot modules: 0"`, full `--expect "boot modules: 2"`. 6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static tables; the self-test prints from EL0 through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. -7. **[demo] The root reaches EL0** on RFC-0005's address spaces and RFC-0006's threads (§9): - `--expect "[server] up"`. +7. **[demo] The root reaches EL0** on RFC-0005's spaces and RFC-0006's threads: `--expect "[server] up"`. 8. **[demo] The Process object** — grant window and teardown as pure logic in a host-tested `process/` crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first resume; `KILL` empties the table. @@ -448,7 +436,7 @@ static object pools; the root's boot scheduling constant (§9). The root doublin 10. **[demo] IPC syscalls** over RFC-0004 endpoints and RFC-0006's blocking states. Boot-test lines in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. RFC-0006's increments add the preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. -11. **Fault messages and death notification** (§17.1–2): a faulting child is reported to the root, +11. **Fault messages and death notification** (§17.1): a faulting child is reported to the root, which prints `[server] child faulted`. 12. **x86_64 trap path and `user/abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the canonical check. Proves §3's second column links; boots with the UEFI stub. @@ -456,7 +444,6 @@ static object pools; the root's boot scheduling constant (§9). The root doublin ## 19. What this unblocks -The first userspace, and with it every Phase-2 paper: the **scheme registry** and the **broker** (built -from processes, grants and `call`), the **driver framework** (fault messages, interrupt notifications, the -console moving out), a **relibc port** (a layer over `user/abi`, both TLS registers still its own), and the -**x86_64 bring-up**, whose trap path is on paper. RFC-0003a gains an ABI to place badges in. +The first userspace, and every Phase-2 paper: the **scheme registry** and **broker** (processes, grants, +`call`), the **driver framework** (faults, interrupt notifications, the console moving out), a **relibc +port** over `user/abi` with both TLS registers its own, and the **x86_64 bring-up**. RFC-0003a gains an ABI. From f94bd19fcec5db2b352ecbe5e7afa05c87a9efb1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:30:57 +0000 Subject: [PATCH 09/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 647 ++++++++++-------- 1 file changed, 379 insertions(+), 268 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index b940bf9..836fd82 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -7,24 +7,24 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 (kernel doctrine); RFC-0003 (one new right, three new objects; `capability/src/rights.rs`); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 (one proposed assumption) | +| Affects | Constitution §3 (kernel doctrine) and its "minimal assembly" clause (§18.7); RFC-0003 (a dated amendment adding one right; three new objects; `capability/src/rights.rs` and `object.rs`'s doc comment); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 and O-7's text (proposals via `docs/CHANGELOG.md`) | | Depends on | RFC-0003 (capability table), RFC-0004 (IPC); interlocks with RFC-0006 (threads and scheduling) and RFC-0007 (system calls and process bootstrap), proposed the same day | -| Discharges | O-7 (the memory and object half), O-8, O-13 (the mapping half); O-5 for every path by which the kernel touches user memory; O-3's destruction case extended to memory; answers RFC-0003 §14.3 and RFC-0004 §9.1 | +| Discharges | O-7 for kernel allocation (reclaim across a transfer stays open), O-8, O-13 (the mapping half); O-5 for every path by which the kernel touches user memory; O-3's destruction case extended to memory; answers RFC-0003 §14.3 and RFC-0004 §9.1 (share) | -> **Proposed verdicts.** +> **Proposed verdicts.** **[demo]** marks what tonight's demo stands on; the rest is forward design. > -> 1. **Discovery:** RAM comes from firmware tables into one bounded `MemoryMap`; the image moves to 0x4020_0000, because QEMU gives an ELF at RAM base no DTB (§4). -> 2. **Frame allocator:** a bitmap over 4 KiB frames, colour-capable from day one; every frame zeroed before any process can see it (§4). -> 3. **Who pays (O-7):** typed fixed-capacity slot arrays are the object store; every frame and slot taken after boot is charged to a capability-named `Pool` (§5). -> 4. **Layout:** a higher-half kernel with the *same* layout on both architectures; 47-bit halves; a non-executable physmap; guarded kernel stacks (§6). -> 5. **Objects:** `Region` (eager, zeroed, at most 8 extents) and `AddressSpace` join RFC-0003's objects beside `Pool`; each mapping is a charged record (§7). -> 6. **Operations:** `map` / `unmap` / `protect` are mechanical, capability-checked, bounded and atomic; `protect` only narrows; the kernel never picks an address (§8). -> 7. **Rights:** `EXECUTE` joins RFC-0003's set as bit 5 — map-executable on a Region, mint-executable on a Pool, meaningless elsewhere (§8). -> 8. **W^X (O-13):** unrepresentable in the mapping type, enforced per Region across *all* address spaces, backed by `SCTLR_EL1.WXN` (§9). -> 9. **Sharing (RFC-0004 §9.1):** *share* is derive-and-transfer; *donate* is transfer made exclusive by `reissue`, a destruction-case operation (§10). -> 10. **User memory (O-5):** the kernel never dereferences a user virtual address, so no syscall takes a pointer argument, ever (§11). -> 11. **TLB:** ASIDs/PCIDs are kernel-private tags with generation rollover; maintenance is inner-shareable from the first line (§12). -> 12. **Early boot:** the stub enables the MMU with throw-away tables; Rust runs only at the high alias and installs the final W^X map (§13). +> 1. **[demo] Discovery:** firmware tables into one bounded `MemoryMap`; the image moves to 0x4020_0000, because QEMU places no DTB for an ELF loaded at the base of RAM (§4). +> 2. **[demo] Frames:** a colour-capable bitmap over 4 KiB frames; fresh frames are zeroed, boot data is handed over read-only, so no frame reaches a process holding another's data (§4, §7). +> 3. **[demo] Who pays (O-7):** typed slot arrays reused at once by generation; every post-boot frame and slot charged to a `Pool` presented by capability — `map` included; the Pool is the one accounting domain for counted units (§5). +> 4. **[demo] Layout:** a higher-half kernel with the same layout on both architectures; 47-bit halves; a non-executable physmap in which the image is read-only; per-core stacks in a guarded window, kept by the kernel (§6). +> 5. **[demo] Objects:** `Region` (eager and zeroed, or boot-minted read-only over boot data) and `AddressSpace` join RFC-0003's objects beside `Pool`; each mapping is a charged record (§7). +> 6. **[demo] Operations:** `map` / `unmap` / `protect` are mechanical, capability-checked, bounded and atomic; `protect` only narrows; the kernel never picks an address (§8). +> 7. **[demo] Rights:** `EXECUTE` joins RFC-0003's set as bit 5 — map-executable on a Region, mint-executable on a Pool, refused at mint on every other type — by dated amendment (§8). +> 8. **[demo] W^X (O-13):** unrepresentable in the mapping type, enforced per Region across all address spaces, backed by `SCTLR_EL1.WXN` (§9). +> 9. **Sharing (RFC-0004 §9.1):** *share* is derive-and-transfer; destruction walks the Region's mappings; *donation* is not proposed here but handed to RFC-0003a as input (§10). +> 10. **[demo] User memory (O-5):** the kernel never dereferences a user virtual address; where a syscall carries one, it is a number, validated as a number (§11). +> 11. **TLB:** tags are kernel-private and recycled; this RFC owns the shootdown and rollover protocol for SMP, RFC-0006 provides the IPI; single-core half **[demo]** (§12). +> 12. **[demo] Early boot:** the stub enables the MMU with a stated `SCTLR_EL1`; Rust runs only at the high alias and installs the final W^X map (§13). ## 1. The question @@ -42,9 +42,8 @@ threat model files integrity *in memory* under A1, and W^X is how signed code st ## 3. What the hardware gives — one shape, two encodings Both Tier-1 MMUs, as configured here, are one machine: **four levels of 512 eight-byte entries, 4 KiB pages, -nine index bits per level, and 47-bit user and kernel halves whose top level uses 256 entries each.** -AArch64 gets this from the 4 KiB granule with `TCR_EL1.T0SZ = T1SZ = 17` (a level-0 start); x86_64 from -IA-32e four-level paging without LA57. The differences are in the leaf encoding: +and 47-bit user and kernel halves whose top level uses 256 entries each.** AArch64 gets this from the 4 KiB +granule with `TCR_EL1.T0SZ = T1SZ = 17`; x86_64 from IA-32e four-level paging without LA57. | Property | AArch64 (VMSAv8-64, 4 KiB granule) | x86_64 (IA-32e, 4-level) | |---|---|---| @@ -61,160 +60,201 @@ IA-32e four-level paging without LA57. The differences are in the leaf encoding: this session — and Intel SDM Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, no hardware access-flag update and no PAN: `target/arm/tcg/cpu64.c` sets `ID_AA64MMFR0_EL1 = 0x1124` and leaves `ID_AA64MMFR1_EL1` at 0.)* Hence **one table engine**, generic over a per-architecture `Descriptor` -encoder, host-tested on both against a fake frame store in `memory/`; only turning a frame number into -`&mut [u64; 512]` through the physmap (§6) is `unsafe`, in `kernel/src/mm/`. +encoder and a frame-access trait, host-tested on both against a fake frame store in `memory/`; only turning +a frame number into `&mut [u64; 512]` through a kernel window (§6, §13) is `unsafe`, in `kernel/src/mm/`. ## 4. Physical memory: discovery and the frame allocator **Discovery.** QEMU documents two fixed facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every other address "may vary across QEMU versions" and must be read from the DTB, which a bare-metal ELF finds -"at the start of RAM" with no register pointing at it (`docs/system/arm/virt.rst`). But `hw/arm/boot.c` -puts it there only if the image leaves room below itself, and today's image *is* the base of RAM. Checked -on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB anywhere in RAM** and `x0 = 0`; at -0x4020_0000 it finds one (magic `0xd00dfeed`) at 0x4000_0000. **So the image loads at 0x4020_0000**, 2 MiB -in and block-aligned; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT reader, written for the -purpose (the Borrow Ledger's boot-path row; Devicetree Specification v0.4), reads the cell sizes, `/memory`, -`/memreserve/` and `/reserved-memory`, bounds-checks every offset and fails closed. Threat model §10 names no -firmware assumption; this RFC proposes one — *firmware tables describe memory truthfully* — through -`docs/CHANGELOG.md`, as §10 already assumes the CPU. On x86_64 the UEFI stub (owned by no RFC yet) hands -over `GetMemoryMap`'s output. **Both produce one `MemoryMap`**: at most 64 sorted, disjoint `(PhysRange, -Kind)` entries — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that, -with image, boot stack, DTB and boot tables tagged before the allocator sees it. Nothing above `arch` names -FDT or UEFI. The DTB passes to the root task read-only (RFC-0007 (proposed) provides the handover). +"at the start of RAM" with no register pointing at it (`docs/system/arm/virt.rst`). `hw/arm/boot.c` puts it +there only below the image (`dtb_limit` is the image's lowest address), and today's image *is* the base of +RAM. Checked on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM** and `x0 = 0`; at +0x4020_0000 one is found (magic `0xd00dfeed`) at 0x4000_0000. **So the image loads at 0x4020_0000**, 2 MiB in +and block-aligned — which caps the DTB at 2 MiB; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT +reader, written for the purpose (Borrow Ledger boot-path row; Devicetree Specification v0.4), reads the cell +sizes, `/memory`, `/memreserve/` and `/reserved-memory`, bounds-checks every offset and fails closed. Threat +model §10 names no firmware assumption; this RFC proposes one — *firmware tables describe memory +truthfully* — through `docs/CHANGELOG.md`. On x86_64 the UEFI stub (owned by no RFC yet) hands over +`GetMemoryMap`'s output after `ExitBootServices`, when boot-services ranges become usable. **Both produce +one `MemoryMap`**: adjacent same-kind ranges coalesced, then at most **256** sorted, disjoint `(PhysRange, +Kind)` entries (6 KiB of `.bss`; real UEFI maps run to 100–200 raw descriptors) — `Usable`, `KernelImage`, +`DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that, with image, boot stack, DTB and boot tables +tagged before the allocator sees it. In Phase 1 the `Mmio` entries are the hard-coded interim addresses +(§19). Nothing above `arch` names FDT or UEFI. **The frame allocator is a bitmap**: one bit per 4 KiB frame, its storage taken from the first free frames -(16 KiB for 512 MiB). It allocates frames from a *colour set*, or runs of `n` at an alignment; a double -free fails closed. Colour is `pfn mod C`, `C = 1` until time protection is wanted *(Ge et al., EuroSys '19, -via research/0002 Part 6)*. **Every frame is zeroed before a process can observe it** — a recycled frame -is an O-8 breach that needs no bug. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing fights -colouring, until measurement asks; a **free list**, which cannot find `n` contiguous frames without a scan. +(16 KiB for 512 MiB). It allocates from a *colour set*, or runs of `n` at an alignment; a double free fails +closed. Colour is `pfn mod C`, `C = 1` until time protection is wanted *(Ge et al., EuroSys '19, via +research/0002 Part 6)*. A run search is O(frames / 64) words — bounded by RAM, not by what the caller may +spend; a per-alignment cursor is the named remedy if measurement asks. **Fresh frames are zeroed before a +process can observe them**, and **a page-table frame is zeroed before its parent descriptor is written** +(a `DSB` between) — a recycled table with stale valid entries would translate into another process's +frames, an O-8 breach no process ever "observes". *Rejected:* a **buddy allocator** *(Linux)*, whose +coalescing fights colouring; a **free list**, which cannot find `n` contiguous frames without a scan. ## 5. Who pays: the object store and Pools (O-7) -The kernel has no heap and must never acquire one a process can drain; every kernel object and page-table -frame is memory some process caused it to spend. **A — seL4 Untyped and retype** is the proven strong form, -but reusing an Untyped revokes everything retyped from it through the derivation tree RFC-0003 §4 refused; -it would decide RFC-0003a by the back door and make object layouts ABI. **Rejected.** **B — a kernel heap -with creation policy** *(Zircon)* limits *whether* a process creates objects, not *how much* memory they -take; exhaustion is global and unnamed. **Rejected.** **C — static per-type tables** *(Hubris)* are bounded -but cannot say *who* filled one — research/0002 Part 6's AWS Kinesis outage. **Rejected alone.** **D — C's -structure with counted attribution: verdict sought**, this paper's one deliberate merge *(the requester pays, -as in seL4; a counted quota, as in Fiasco.OC's factory limits and Genode's quota trading)*. +The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped and retype** is +the proven strong form, but reusing an Untyped revokes everything retyped from it through a derivation +tree, which would decide RFC-0003a's B1 by the back door (RFC-0003 §7 and its 2026-07-30 amendment) and +make object layouts ABI. **Rejected.** **B — a kernel heap with creation policy** *(Zircon)* limits +*whether* a process creates objects, not *how much* memory they take; exhaustion is global and unnamed. +**Rejected.** **C — static per-type tables** *(Hubris)* are bounded but cannot say *who* filled one — +research/0002 Part 6's Kinesis outage. **Rejected alone.** **D — C's structure with counted attribution: +verdict sought** *(the requester pays, as in seL4; a counted quota, as in Fiasco.OC's factory limits and +Genode's quota trading)*. **The object store is typed slot arrays.** Each object type has one fixed-capacity array in `.bss`; a slot -holds the object, a generation and a reference count, and **the slot index is the object's identity**. -`ObjectRef` (`capability/src/object.rs`) is `(type, index)` and `current_generation()` is a slot read — no -raw pointer, slab or size class; a process's fixed-capacity `CapabilityTable` lives in its slot. A -destroyed object frees its frames at once; its slot is reused only when the count, stale capabilities' -references included, reaches zero, and is retired, as RFC-0003 requires, if its generation cannot advance. +holds the object, a generation and a count, and the slot index is the object's identity. The kernel's +implementation of `ObjectRef` (`capability/src/object.rs`, a trait) is `(type, index, generation)`, and +`current_generation()` is a slot read — no raw pointer, slab or size class. **Only live references count**: +capabilities minted at the slot's current generation, Mappings and IPC-buffer bindings; a reference whose +generation is not the slot's counts nothing, on clone or drop. So a destroyed object's slot is **reused at +once with its generation advanced**, stale capabilities fail closed on it — exactly what RFC-0003 §8's +generation exists to allow — and no past holder can pin another process's charge. A slot whose generation +cannot advance is retired, its unit debited once from its payer and thereafter kept by the kernel (objects +keep 64-bit generations, RFC-0007 (proposed) §6, so this is a boundary, not a budget). `object.rs`'s +"takes a reference count" comment gains that qualification in increment 7a. + +**Where it lives.** The generic slot array and Pool are safe code in `memory/`, generic over the object type. +The concrete store — one `Kernel` value holding every type's array, RFC-0004/0006/0007 objects included — is +instantiated in the kernel crate and reached through **one** `static` wrapper in `kernel/src/mm/` carrying one +SAFETY argument: one core, the kernel entered with interrupts masked (RFC-0006 (proposed) V1), the big lock +when SMP lands. No other kernel global accretes *(research/0002 Part 6)*. **A Pool is the accounting layer**: an RFC-0003 object holding balances — frames, and slots of each type. At -boot every free frame and slot becomes the balance of one root Pool handed to the root task (RFC-0007 -(proposed) provides the handover). Thereafter, for each resource `r` (frames, or one type's slots): +boot every free frame and slot becomes the balance of one root Pool (RFC-0007 (proposed) hands it over). +Thereafter, for each resource `r`: ```text Σ balance(r) over all Pools == units of r free -- at every instant after boot -every unit of r in use == kept by the kernel at boot, or charged to exactly one Pool +every unit of r in use == kept by the kernel (at boot, or retired), or charged to exactly one Pool ``` -- **Everything is charged**: Region frames; page-table frames, to the AddressSpace's Pool; one `Mapping` - slot per mapping; kernel stacks; every object slot. Each creation presents a Pool with `WRITE`. -- **Charge first, then take.** Conservation guarantees the allocator can then supply; the only later - failure, `TooFragmented` for a multi-frame extent, refunds before returning. -- **Failure is local and attributed.** `PoolExhausted` names the Pool and resource; no array can fill while - a Pool holds its slots, so no unnamed limit exists after boot. Nothing overcommits *(research/0002 Part 6, - "budgets are minima")*. Refund goes to the payer, whose index each object records — no tree. -- **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child with the parent's rights; - `pool_move(from, into, r, n)` moves `n` units of one resource; `pool_merge(from, into)` succeeds only - when `from` has no outstanding charges. Balance is conserved, never minted — O-2's shape for quantity. +- **Charged:** Region frames; every object slot; page-table frames and the `Mapping` slot, to the Pool + `map` names (§8). **Kept, never charged:** the image, bitmap, slot arrays, kernel tables, per-core + stacks and boot-minted Regions (§7). Each charge presents a Pool with `WRITE`; nothing spends ambiently. +- **Charge first, then take.** Conservation guarantees the allocator can supply; the only later failure, + `Fragmented` for a multi-frame extent, refunds before returning. +- **Failure is local and attributed.** `PoolExhausted` names the Pool and resource; nothing overcommits + *(research/0002 Part 6, "budgets are minima")*. Refund goes to the payer, whose index each object records. +- **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child with the parent's rights, + recording the parent as its funder; `pool_move(from, into, r, n)`; `pool_merge(from, into)` when `from` + has no outstanding charges. Balance is conserved, never minted — O-2's shape for quantity. +- **A Pool whose last capability closes** returns its free balance to its funder at once; its slot stays as + a forwarding record until its outstanding charges come home, each refund forwarded — a chain bounded by + the Pool slot count. The root Pool has no funder: closing it hands its balance to the kernel for good. - **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). -This is not option B's heap: nothing grows after boot, and frames come from the bitmap in page units, -charged. The kernel keeps only its image, the map, the bitmap, the slot arrays, its tables and boot stack. +**One accounting domain — decided here.** research/0002 Part 6 asks for one capability-named domain to +which memory, object quotas and scheduling contexts attach. **The Pool is that domain for every counted +unit**: frames and every type's slots, Threads' and SchedContexts' included. **Time is not a unit**: +RFC-0006 (proposed) sells it as per-core bandwidth through `Core`, which moving balance cannot conserve, +and folding it in would make every Pool holder a seller of CPU. That is a deliberate split from the +research; congruence survives by attribution, since every SchedContext is charged to a Pool. RFC-0006 is +told so; O-7's text should then read *"kernel memory is charged to a capability-named Pool; CPU time to a +SchedContext funded through a `Core`"*, proposed through `docs/CHANGELOG.md`. ## 6. The kernel's virtual layout Both architectures get the **same layout**, the cheapest way to keep the HAL honest. The image sits in the -top 2 GiB because `x86_64-unknown-none` uses `code-model: kernel`, which requires it; AArch64 follows. +top 2 GiB, which `x86_64-unknown-none`'s default kernel code model expects; overriding it is possible +(RFC-0007 does so for userspace) but buys nothing. AArch64 follows. | Range (both architectures) | Contents | Attributes | |----------------------------|----------|------------| | `0x0000_0000_0001_0000` – `0x0000_7FFF_FFFF_EFFF` | the current AddressSpace (§7) | per mapping; not-global; never kernel-executable | -| `0xFFFF_8000_0000_0000` – `+64 TiB` | physmap: every discovered RAM frame at `base + pa` | Normal write-back; RW; execute-never; global; largest blocks alignment allows | +| `0xFFFF_8000_0000_0000` – `+64 TiB` | physmap: every discovered RAM frame at `base + pa` | Normal write-back; RW, but the image's `.text`/`.rodata` read-only; execute-never; global | | `0xFFFF_C000_0000_0000` – `+1 TiB` | MMIO window: kernel-owned devices only (console, interrupt controller) | Device-nGnRE / uncached; RW; execute-never | -| `0xFFFF_FFFF_0000_0000` – `0xFFFF_FFFF_7FFF_FFFF` | kernel stacks, an unmapped guard page below each | Normal; RW; execute-never; frames charged like any object | +| `0xFFFF_FFFF_0000_0000` – `0xFFFF_FFFF_7FFF_FFFF` | per-core kernel and emergency stacks, an unmapped guard page below each | Normal; RW; execute-never; kept by the kernel at boot, never charged | | `0xFFFF_FFFF_8000_0000` – top | image at `+0x20_0000` | `.text` RX, `.rodata` R, `.data`/`.bss` RW execute-never | -- **Kernel stacks have their own window** so each has a guard page, which the hole-free physmap cannot give. - RFC-0006 (proposed) decides how many — per core or per thread — and allocates here; the boot stack too. +- **Stacks.** RFC-0006 (proposed) V1 chose one kernel stack per core; they are allocated at boot, and core + 0's boot stack moves from `aarch64.ld`'s unguarded slot into this window in increment 9. A guard page + makes an overflow a fault, but the fault's entry then pushes onto the same stack: **RFC-0006 (proposed) + provides the overflow stack** (x86_64 IST; on AArch64 the EL1-origin vector switching to the emergency + stack, since a kernel fault is fatal). Until then an overflow is a recursive abort — detected, not diagnosed. - **The user floor is 64 KiB**, so a kernel NULL dereference never lands on user data *(Linux - `mmap_min_addr`)* — a fixed mechanism constant where Linux has a sysctl (§17). -- **The user ceiling leaves the last page below the hole unmapped**, closing Intel's `SYSRET` hazard - (CVE-2012-0217) for RFC-0007's x86_64 exit path by layout, as Linux does. The physmap covers RAM, never - MMIO, so no frame is aliased with conflicting attributes. -- **The kernel half never changes after boot.** AArch64 shares `TTBR1_EL1` for free; x86_64 copies PML4 - entries 256–511 into each address space, safe only because they are fixed *(seL4 x86)*. *Rejected:* an - identity-mapped kernel (impossible under the code model); no physmap (an invalidation per IPC-buffer touch). + `mmap_min_addr`)*. **The user ceiling leaves the last page below the hole unmapped**, closing Intel's + `SYSRET` hazard (CVE-2012-0217) for RFC-0007's exit path by layout. The physmap covers RAM, never MMIO. +- **The kernel's top-level entries never change after boot.** AArch64 shares `TTBR1_EL1`; x86_64 copies PML4 + entries 256–511 into each address space *(seL4 x86)*, so every kernel-half PML4 entry that will ever be + used — physmap for discovered RAM, MMIO window, stack window, image — is populated at boot; a PML4E first + filled later would be missing from every earlier copy (Linux's vmalloc-sync bug family). Lower-level kernel + tables may change; such a change is global and invalidates on every core (§12). *Rejected:* an + identity-mapped kernel — possible under an overridden code model, but it puts the kernel where user + addresses go and breaks the symmetric halves; no physmap (an invalidation per IPC-buffer touch). ## 7. The objects: Region, AddressSpace and Mapping **Frame ownership is pKVM's state machine** (research/0002 Part 7): every frame has exactly one owner — the -kernel, or one `Region`. **A Region** is a fixed number of pages backed by **at most eight extents** of -contiguous frames, allocated eagerly, zeroed and charged; a request the bitmap cannot meet in eight fails -`TooFragmented`, and DMA-ready memory is one extent at an alignment. Rights are `READ`, `WRITE`, `EXECUTE`, -with RFC-0003's `DUPLICATE`, `TRANSFER`, `REVOKE`. Mappings and IPC-buffer bindings hold counted references, -so closing the last handle does not pull memory from under a live mapping *(Zircon: a mapping keeps its VMO -alive)*; the forceful paths are destruction and `reissue` (§10). - -**An AddressSpace** is a user root table, an ASID or PCID (§12), the Pool paying for its tables, and its -mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a process is RFC-0007's. -**A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, vaddr, pages, perms}`, -linked by slot index into its AddressSpace's list and its Region's. +kernel, or one `Region`. **A Region** is a fixed number of pages backed by at most eight extents of +contiguous frames. Rights are `READ`, `WRITE`, `EXECUTE`, with RFC-0003's `DUPLICATE`, `TRANSFER`, `REVOKE`. +Mappings and IPC-buffer bindings hold counted references, so closing the last capability does not pull +memory from under a live mapping *(Zircon: a mapping keeps its VMO alive)*. Three kinds, one object: + +- **Ordinary** — `region_create`: allocated eagerly, zeroed, charged; a request the bitmap cannot meet in + eight extents fails `Fragmented`; DMA-ready memory is one extent at an alignment. Zeroing is the long + part: it runs 512 pages per step between RFC-0006 (proposed)'s preemption points, the half-built Region + held on the calling thread, and no capability exists until it is done. +- **Boot-minted, read-only** — once, at boot, over boot data: the DTB and each boot module RFC-0007 + (proposed) embeds. Their `DeviceTree`/`KernelImage` frames **move** into the Region's ownership; they are + never zeroed (public boot data, so O-8 is untouched), never charged, and minted without `WRITE`: the DTB + `READ`, a module `READ | EXECUTE | DUPLICATE | TRANSFER`, so text maps `RX` in place and W^X holds because + no writable use can ever exist. RFC-0007's bundle must page-align every module segment. +- **Device** — minted at boot over `Mmio` ranges the kernel does not keep; never executable, zeroed or + charged; Phase 2. No operation maps a physical address given as a number: the catch-all right in costume. + +Regions the kernel fills for the root before handover (its writable segments, stack, BootInfo) are ordinary: +created and charged to the root Pool, bytes copied in through the physmap, and only then a capability minted. + +**An AddressSpace** is a user root table, a tag (§12), the Pool paying for its tables — the only Pool `map` +may charge — and its mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a +process is RFC-0007's. **A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, +vaddr, pages, perms}`, linked by slot index into its AddressSpace's list and its Region's. **Why a Region must remember its mappings.** RFC-0003's 2026-07-30 amendment makes an invariant -load-bearing: *a resolved capability is never observed or cached outside the kernel table* — why a -generation bump alone makes capabilities inert. **A page-table entry is exactly such a cache**, consulted by -hardware on every access and never re-checked. The MMU makes the exception unavoidable; the per-Region -mapping list is the compensation, and the reason a generation bump alone cannot revoke memory. - -**Device Regions** are minted at boot over `Mmio` ranges the kernel does not keep — never executable, zeroed -or charged; built in Phase 2. No operation maps a physical address given as a number: the catch-all right in -another costume. *Rejected:* **Zircon VMOs and VMARs** — lazy commit, copy-on-write, pagers and nesting are -policy-rich and put faults inside kernel paths; **seL4's capability per frame** — 256 handles per MiB; -**userspace-built page tables** (Xen PV) — research/0002 Part 7's bug farm; O-5 is a rights check, not a walk. +load-bearing: *a resolved capability is never observed or cached outside the kernel table*. **A page-table +entry is exactly such a cache**, consulted by hardware and never re-checked. The per-Region mapping list is +the compensation, and the reason a generation bump alone cannot revoke memory. *Rejected:* **Zircon VMOs +and VMARs** — lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in kernel paths; +**seL4's capability per frame** — 256 handles per MiB; **userspace-built page tables** (Xen PV). ## 8. Map, unmap, protect — and the rights they check -Every operation takes handles (RFC-0003 §9); RFC-0007 (proposed) encodes them. +Every operation takes handles (RFC-0003 §9); RFC-0007 (proposed) encodes them as methods. A second handle — +`map`'s Region and Pool — travels as a **handle word in a message register**, resolved in the caller's +table, never transferred: RFC-0003 §6's move would otherwise take the Region from its mapper. | Operation | Needs | Refused when | |-----------|-------|--------------| -| `region_create(pool, pages, extents, align)` | pool `WRITE`; pool `EXECUTE` to mint `EXECUTE` | `PoolExhausted`; `TooFragmented`; zero pages | -| `as_create(pool)` | pool `WRITE` | `PoolExhausted` (tags roll over, §12) | -| `map(as, region, first_page, vaddr, pages, perms)` | as `WRITE`; region rights ⊇ `perms` | `vaddr` unaligned, below the floor or outside the user half; overlap (never silent replacement); beyond the Region; `pages > 512`; `perms ∉ {R, RW, RX}`; W^X (§9); `PoolExhausted` | +| `region_create(pool, pages, max_extents, align, rights)` | pool `WRITE`; pool `EXECUTE` if `rights` has `EXECUTE` | `PoolExhausted`; `Fragmented`; zero pages; `align` not a power-of-two page count; `rights` ⊄ Region rights | +| `as_create(pool, rights)` | pool `WRITE` | `PoolExhausted` | +| `map(as, region, pool, first_page, vaddr, pages, perms)` | as `WRITE`; region rights ⊇ `perms`; `pool` is the AddressSpace's own, with `WRITE` | `vaddr` unaligned, below the floor or outside the user half; overlap; beyond the Region; `pages > 512`; `perms ∉ {R, RW, RX}`; W^X (§9); `PoolExhausted` | | `unmap(as, vaddr)` | as `WRITE` | no mapping based at `vaddr` — the unit is the whole mapping | | `protect(as, vaddr, perms)` | as `WRITE` | `perms` not a subset of the mapping's current `perms` | -| `reissue(region)` | region `REVOKE` (a proposed reading, §10) | stale handle | User numbers are validated *as numbers*. **The kernel never chooses a virtual address** *(seL4's VSpace -discipline)*: a kernel that picks addresses has an allocation policy in it. **`map` is atomic**: it counts -the table frames the range lacks and charges them, with the Mapping slot, *before* writing anything, so a -failed charge maps nothing; then it links tables and writes leaves. One call is bounded at 512 leaves and -six new tables — an unaligned range spans two leaf tables, each possibly with new parents at two levels. -Nothing splits a mapping. `protect` only narrows; widening is `map` again with the Region capability *(O-2's -shape; seL4's remap-with-capability)*. 4 KiB pages only; blocks later, invisible in the ABI. +discipline)*. **Why `map` names a Pool:** spending is authority (O-4), so a holder of an AddressSpace +capability alone cannot drain its Pool; requiring *the AddressSpace's own* Pool keeps one payer per space, +so refunds need no per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool +(a payer record per table frame); a Pool bound at `as_create` and spent by `WRITE` (spending never granted). +**`map` is atomic**: it counts the table frames the range lacks and charges them, with the Mapping slot, +*before* writing anything; then it zeroes and links tables, writes leaves and ends with `DSB ISHST`, so the +walker sees them before `eret`. One call is bounded at 512 leaves and six new tables. Nothing splits a +mapping. `protect` only narrows; widening is `map` again with the Region capability *(seL4's remap)*. 4 KiB +pages only; blocks later, invisible in the ABI. **Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as RFC-0003 §5 allows "when a subsystem needs one". Folding execution into `READ` makes every readable Region potential code; a Region *kind* fails -the loader, which writes code before running it — a right in disguise. seL4 leaves execution an ungoverned -attribute; Fuchsia's `ZX_RIGHT_EXECUTE` *(Zircon)* is the lineage. The bit has two meanings, deliberately — -*may map executable* on a Region, *may mint executable Regions* on a Pool — one narrow authority over code, -refused on other objects. **`Rights::ALL` does not extend automatically**: `rights.rs` writes it as an -explicit union of five constants, and its doc comment claiming otherwise is misleading. The same PR edits -`ALL` and the comment, extends `Debug` and `rights_from_low_bits`, and widens the exhaustive test to `0..64`. - -The handshake RFC-0003 §14.3 asked for is one rule: **the installed permission is the requested `perms`, -refused unless a subset of the Region capability's rights at map time.** +the loader, which writes code before running it. seL4 leaves execution an ungoverned attribute; Fuchsia's +`ZX_RIGHT_EXECUTE` is the lineage. It means *may map executable* on a Region and *may mint executable +Regions* on a Pool, and is **refused at mint on every other type**: the store's mint path rejects it +(host-tested, increment 7a), and `derive` cannot add it (O-2). `rights.rs`'s `ALL` is an explicit union +despite its comment; increment 4 edits both. **On acceptance, RFC-0003 gains a dated amendment adding +`EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 §14.3's handshake is one rule: **the +installed permission is the requested `perms`, refused unless a subset of the Region capability's rights.** | perms | needs on the Region | AArch64 stage 1 | x86_64 | |-------|---------------------|-----------------|--------| @@ -227,223 +267,294 @@ is refused; execute-only exists on AArch64 but not on x86_64 without protection ## 9. W^X (O-13), in four layers -O-13's intent — a data-only bug cannot become code execution — needs more than its letter: - 1. **Unrepresentable.** `MapPerms` is an enum of `Read`, `ReadWrite`, `ReadExecute`; both encoders are total functions from it, so no writable-executable entry can be built. Host-tested exhaustively. -2. **Per Region, across every address space.** A Region counts its writable uses (an IPC-buffer binding is - one) and its executable mappings; `map` refuses `RX` while any writable use exists anywhere, and `RW` - while any `RX` mapping does. Per-page W^X alone leaves frames writable here and executable there. +2. **Per Region, across every address space.** A Region counts its writable uses (`RW` mappings and + IPC-buffer bindings) and its `RX` mappings. `map RX` is refused while any writable use exists; `map RW` + and IPC-buffer binding are refused while any `RX` mapping exists. `unmap`, `protect` and unbinding + decrement **only when their invalidation has completed on every core** (§12) — before that, a stale + writable entry may still be live somewhere. Per-page W^X alone leaves frames writable here and + executable there. 3. **Over time.** `EXECUTE` is minted only from a Pool carrying `EXECUTE`: the broker hands apps Pools - without it, the loader holds one, a JIT gets one by explicit grant *(Fuchsia moved executable memory from - ambient job policy to the `VmexResource` capability)*. O-13's Phase-3 load-path clause is policy on this. -4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); the - architecture already makes every EL0-writable page privileged-execute-never, and user entries set `PXN` - besides. x86_64 has no `WXN`; it sets `CR0.WP` and `SMEP` where present, and layers 1–3 are the defence. - -`map` with `RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` -on a PIPT instruction cache and `IC IALLUIS` otherwise (on VIPT-aliasing caches the physmap alias may miss -the user one — Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`. QEMU's cortex-a72 reports PIPT (`CTR_EL0 = -0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so errors here pass every test.** The -physmap aliases user code as writable at EL1; writing through it only to zero frames, fill tables and reach -IPC buffers is a convention (§17). - -## 10. Sharing and donation (RFC-0004 §9.1) - -pKVM's *share* (the owner keeps access) and *donate* (the donor loses it), research/0002 Part 7's model, need one operation: + without it, the loader holds one, a JIT gets one by explicit grant *(Fuchsia's `VmexResource`)*. O-13's + Phase-3 load-path clause is policy on this. +4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); user + entries set `PXN`, and the image's text has no writable alias, since the physmap maps it read-only + *(Linux `mark_linear_text_alias_ro`)*. x86_64 has no `WXN`; it sets `CR0.WP` and `SMEP` where present. + +`map RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` on a +PIPT instruction cache and `IC IALLUIS` otherwise (Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`. QEMU's +cortex-a72 reports PIPT (`CTR_EL0 = 0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, +so errors here pass every test.** These, and §12's `TLBI`s and `TTBR` writes, are single-instruction `asm!` +intrinsics inside `kernel/src/arch/**` with no Rust equivalent, as `wfe` is today — a reading of the +constitution's "minimal assembly" the maintainer is asked to confirm (§18.7). The physmap still aliases +*user* code writable at EL1; writing through it only to zero, fill tables and reach IPC buffers is a +convention (§17). + +## 10. Sharing and destruction (RFC-0004 §9.1) - **Share** is `derive` then transfer over IPC: the borrower receives, say, `READ | WRITE` without `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data then travels as RFC-0004 §5's - descriptor. No `share` syscall, and no page-table work during IPC — the long-IPC grave stays shut. -- **Donate** is transferring a capability carrying `REVOKE`, made exclusive by **`reissue(region)`**: tear - down every mapping and IPC-buffer binding of the Region in every address space (§7 says why the walk is - unavoidable), complete the invalidation (§12), advance the generation so every other capability is stale, - and return a fresh capability with the caller's rights over the same frames. **Destruction** is the same - walk without the re-mint; frames return to the payer only after the invalidation completes. - -`reissue` is **RFC-0003 §7's destruction case** — generation bump and re-mint, B2's shape on one object — and -pre-empts nothing in RFC-0003a's B1/B2/B3 choice. Gating it on `REVOKE` is a **proposed reading**: RFC-0003 -§5 defines `REVOKE` over capabilities derived from the holder's, and `reissue` also reaches holders that were -not. Copies the donor derived first die at the bump; a former sharer takes an ordinary fault (RFC-0006 -(proposed) provides delivery) — the "observable and survivable" revocation research/0002 Part 6 asks for. + descriptor. No `share` syscall and no page-table work during IPC — the long-IPC grave stays shut. +- **Destruction** — at the last live reference, or by RFC-0003 §7.1's destruction case (the right is open, + §18.3) — marks the Region *dying* so no new mapping, binding or copy resolves it, tears down every mapping + and IPC-buffer binding in every address space (§7 says why the walk is unavoidable), completes the + invalidation (§12), advances the generation and refunds the payer. Its preemption points fall between + whole mappings. +- **Donation is not proposed.** pKVM's *donate* would be `reissue` — bump and re-mint for the caller over the + same frames, moving the charge to the recipient's Pool — which is RFC-0003 §7's option B2 on one object. + Gating it on `REVOKE` would widen that right ("capabilities derived from this one") to reach peers, and + under the amendment's option **(a)** a peer-transferred Region carrying `REVOKE` could cut the broker off. + So it is **input to RFC-0003a's (a)/(b) choice**, not a verdict here; the demo needs none of it. + +**Concurrency.** Under RFC-0006 (proposed) V1 — non-preemptible, one big lock when SMP lands — no kernel +access to a Region's frames (zeroing, table fill, IPC copy) interleaves with its destruction: each runs +whole within one kernel entry and re-resolves the Region there, and the dying mark covers the gaps between +destruction's steps. If locking ever becomes finer, the order is **Region before AddressSpace**. A former +sharer takes an ordinary fault: RFC-0006 (proposed) binds its delivery and RFC-0007 (proposed) defines the +message; under RFC-0007's interim the thread stops. ## 11. How the kernel touches user memory (O-5) **The kernel never dereferences a user virtual address** — not through a checked copy or a fixup table, not -at all *(seL4: the IPC buffer is a frame reached through the kernel window)*. Everything the kernel reads or -writes for a process is named by capability and reached through the physmap: - -- **The IPC buffer** — RFC-0004's spill area and slow-path copy — is one page of a Region bound to a thread - by capability (RFC-0006 (proposed) records it on the Thread; RFC-0007 (proposed) provides the syscall). - Binding needs `READ | WRITE`, holds a counted reference, is a writable use for §9, and `reissue` severs it. +at all *(seL4: the IPC buffer is a frame reached through the kernel window)*. **Where a syscall carries one +— a `map` destination, an IPC-buffer location — it is a number, validated as a number and used only to +program tables or registers.** Everything the kernel reads or writes for a process is named by capability +and reached through the physmap: + +- **The IPC buffer** is a 1 KiB-aligned slice of one page of a Region (RFC-0007 (proposed) §5 sets its size + and syscall; RFC-0006 (proposed) records it on the Thread). Binding needs `READ | WRITE`, holds a counted + reference, is a writable use for §9, and destruction severs it. - **No kernel path faults on user memory**: frames are eager and pinned, so the slow path copies frame to frame, and the in-kernel fault concurrency that sank L4's long IPC cannot arise. - **Peer-writable memory is never referenced.** No Rust reference is formed into it: one `kernel/src/mm/` - module copies with volatile accesses, and anything the kernel must *interpret* is copied once and validated - as that snapshot *(rust-vmm; virtio's double-fetch lesson, research/0002 Part 5)*. + module copies with volatile accesses, and anything the kernel must interpret is copied once and validated + as that snapshot *(virtio's double-fetch lesson, research/0002 Part 5)*. -**The constraint RFC-0007 inherits: no syscall takes a pointer argument, ever.** Data travels in registers, the -IPC buffer, or as a Region handle plus offset; RFC-0003 §9 forbade naming *resources* by address, and this -extends it to data. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. -`SMAP` and `PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. +RFC-0007 inherits that sentence exactly. RFC-0003 §9 forbade naming *resources* by address; this extends it +to data. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. `SMAP` and +`PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. ## 12. TLBs, ASIDs and PCIDs -- **Tags are kernel-private**: an ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8) or a PCID - (12 bits, where CPUID reports it); tag 0 reserved. Exhaustion bumps a generation, invalidates all and - reassigns lazily *(Linux `arch/arm64/mm/context.c`)*. *Rejected:* seL4's ASID pools — a tag is not authority. -- **Kernel entries are global, user entries never**; without PCID (QEMU's default x86_64 CPU) a `CR3` load - flushes the non-global ones. **`map` needs no invalidation**: neither architecture caches a faulting - entry (the Arm ARM; SDM Vol. 3A §4.10), though x86_64's paging-structure caches can cause one spurious - fault the handler re-checks. `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS` (ASID - and address, leaf only), `DSB ISH`, `ISB`; x86_64 uses `INVLPG`, and `INVPCID` or a stale-on-switch mark. - **Freeing a table frame needs more**: `VALE1IS` leaves walk caches alone, so any path that frees an - intermediate table (§18) uses `TLBI VAE1IS` or `ASIDE1IS`; x86_64's `INVLPG` and `INVPCID` already flush - the PCID's paging-structure caches (SDM Vol. 3A §4.10.4.1). Live remaps use break-before-make. +- **Tags are kernel-private**: an ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8) or a + PCID (12 bits, where CPUID reports it); tag 0 reserved. **Destroying an AddressSpace returns its tag** + after `TLBI ASIDE1IS` (or `INVPCID` single-context), so churn does not force rollover. *Rejected:* seL4's + ASID pools — a tag is not authority. +- **Kernel entries are global, user entries never.** `map` needs no invalidation: neither architecture caches + a faulting entry (the Arm ARM's rule; its section is uncited because the manual was unreachable; SDM + Vol. 3A §4.10), and a fault whose entry is valid on re-walk is retried — x86_64's paging-structure caches + can cause one. `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS` (ASID and address, + leaf only), `DSB ISH`, `ISB`; x86_64 uses `INVLPG`, and `INVPCID` or a stale mark for other tags. Freeing an + intermediate table uses `TLBI VAE1IS` or `ASIDE1IS`, since `VALE1IS` spares walk caches; x86_64's `INVLPG` + and `INVPCID` already flush the PCID's paging-structure caches (SDM Vol. 3A §4.10.4.1). - **O-8's ordering: invalidate, then free.** No frame returns to a Pool while any core can translate to it. -- **Inner-shareable from day one.** The demo is `-smp 1`, but every `TLBI` is the broadcast form. x86_64 has - no broadcast; its shootdown IPIs are RFC-0006 (proposed)'s multi-core shape, walking §7's mapping lists. + +**Multi-core — this RFC owns the protocol; RFC-0006 (proposed) provides the IPI.** Each AddressSpace keeps +the set of cores that have run it since its last full flush *(Linux `mm_cpumask`)*. + +- **Shootdown.** AArch64 needs no IPI: the `…IS` forms reach the inner-shareable domain and `DSB ISH` waits + for completion on every core. x86_64 has no broadcast: the initiator sends RFC-0006's IPI to the cores in + the set, each issues `INVLPG`/`INVPCID` and acknowledges, and the initiator waits for every + acknowledgement before any frame is freed or any W^X count falls. A core spinning on the big lock polls + its per-core flush request while it spins, so the lock holder never waits on a core waiting for it. +- **Rollover.** On exhaustion the generation bumps; every core's *active* tag is reserved into the new + generation; every core gets a pending-flush mark and flushes locally (`TLBI VMALLE1`) at its next + address-space switch. **A tag is reallocated only after every core that ran under it has flushed** + *(Linux `arch/arm64/mm/context.c`: `active_asids`, `reserved_asids`, `tlb_flush_pending`)*. +- **x86_64 PCIDs are per core** (SDM Vol. 3A §4.10.1): each core caches a few AddressSpace→PCID pairs + *(Linux keeps six)*, and an invalidation on a core where the space is not current is a per-core stale mark + consumed at the next switch to it. + +The demo is `-smp 1`, but every `TLBI` is the broadcast form from the first line; the rest is built with +RFC-0006 increment 14. ## 13. Early boot: turning the MMU on -The image is linked at `0xFFFF_FFFF_8020_0000` and loaded at 0x4020_0000 (`AT()` in `aarch64.ld`); QEMU loads -segments at `p_paddr` and translates the entry point to its physical alias (`include/hw/elf_ops.h.inc`; -checked on 8.2.2), and the stub is already position-independent. With the MMU off, the stub: +The image is linked at `0xFFFF_FFFF_8020_0000` and loaded at 0x4020_0000 (`AT()` in `aarch64.ld`). QEMU loads +segments at `p_paddr` and translates the entry point to its physical alias **only if** it lies in an +executable segment whose `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, `load_elf`, at v8.2.2) — which +`AT()` guarantees while `.text.boot` stays first in the first executable segment. With the MMU off, the stub: 1. Parks secondaries, installs the stack and zeroes `.bss` at physical addresses, as today. 2. `MAIR_EL1`: index 0 = `0xFF` (Normal write-back, read/write-allocate), index 1 = `0x04` (Device-nGnRE). -3. `TCR_EL1`: `T0SZ = T1SZ = 17`; `TG0 = 0b00` and `TG1 = 0b10`, both 4 KiB (the trap); `SH = 0b11`; - `IRGN/ORGN = 0b01`; `IPS` from `PARange` (44 bits here); `AS = 1`; `A1 = 0`; `TBI0 = TBI1 = 0`; `HA = HD = 0`. -4. Four page-aligned boot tables in `.bss`: `TTBR0_EL1` identity-maps 0x4000_0000 as a 1 GiB Normal block and - 0x0000_0000 as a 1 GiB Device block, so the UART survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to - 0x4000_0000 as a 1 GiB Normal block. Five descriptors, all `AF = 1`. -5. `DC IVAC` over the tables, written non-cacheable, so no stale line shadows them once walks are cacheable - (Linux `head.S` does the same); `TLBI VMALLE1`; `DSB ISH`; both `TTBR`s; `ISB`; `SCTLR_EL1` written whole - with `M`, `C`, `I` (bits 0, 2, 12) and `nTWI = nTWE = 0`, as RFC-0006 (proposed) asks; `ISB`. +3. `TCR_EL1`: `T0SZ = T1SZ = 17`; `TG0 = 0b00`, `TG1 = 0b10` (both 4 KiB); `SH = 0b11`; `IRGN/ORGN = 0b01`; + `IPS` from `PARange`; `AS` from `ASIDBits`; `A1 = 0`; `TBI0 = TBI1 = 0`; `HA = HD = 0`. +4. Four page-aligned boot tables in `.bss`, `DC IVAC` before and after they are written, so no dirty + firmware line is evicted over them and no stale line shadows them *(Linux `head.S`)*: `TTBR0_EL1` + identity-maps 0x4000_0000 as a 1 GiB Normal block and 0x0000_0000 as a 1 GiB Device block, `UXN` and + `PXN` set, so the UART survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to 0x4000_0000 as a 1 GiB + Normal block. All `AF = 1`. The Normal blocks span 512 MiB with no RAM behind it — a QEMU-only interim. +5. `TLBI VMALLE1`; `IC IALLU`; `DSB NSH`; `ISB`; both `TTBR`s; `ISB`; `SCTLR_EL1` = **`0x30D4581D`**; `ISB`. 6. Branches to the high alias by absolute address, moves `SP` and `VBAR_EL1` there, and calls `rust_entry`. **Rust never executes at the low alias.** -Rust then, in an order that never leaves the exception reporter without a console: (1) reads the DTB through -the high block and builds the `MemoryMap`, bitmap and slot arrays; (2) builds §6's final tables; (3) swaps -`TTBR1_EL1` by an assembly routine run from the identity map — empty table, `TLBI VMALLE1`, final table — -since turning blocks into pages under a live translation needs break-before-make *(Linux -`idmap_cpu_replace_ttbr1`)*; (4) **repoints the console at the UART's MMIO-window address** — the physical -`UART0_BASE` in `uart.rs` dies silently once the identity map goes; (5) only then sets `TCR_EL1.EPD0 = 1`, -so nothing walks `TTBR0_EL1` until the first AddressSpace loads, sets `WXN`, and issues `TLBI VMALLE1`, -`DSB ISH`, `ISB` (`WXN` may be cached in the TLB); the boot tables' frames are freed. - -**From MMU-on to step 5 the kernel is one RWX gigabyte** — before any process exists, but a stray write in -early Rust goes uncaught. The reporter gains the decode this needs: aborts print `FAR_EL1` and the fault -level from `ESR_EL1.ISS.DFSC`; on x86_64, `#PF` prints `CR2` and its error code. **x86_64**, once the UEFI -stub exists, is already paging; Rust builds §6's PML4, sets `IA32_EFER.NXE` (bit 11 — without it `XD` is a -reserved-bit fault), `CR0.WP` (bit 16, so ring 0 honours read-only pages), `CR4.PGE` (bit 7) and -`CR4.PCIDE`/`SMEP`/`SMAP` (bits 17, 20, 21) where CPUID permits, and loads `CR3`. *Rejected:* a separate +| `SCTLR_EL1` field (bit) | Value | Why | +|-------------------------|-------|-----| +| `M` (0), `C` (2), `I` (12) | 1 | MMU and caches on | +| `A` (1) | 0 | alignment is the compiler's; both targets are `+strict-align` | +| `SA` (3), `SA0` (4) | 1 | stack-alignment checks at EL1 and EL0 | +| `CP15BEN` (5), `ITD` (7), `SED` (8) | 0 | AArch32 controls; no thread runs AArch32 (RFC-0007 builds every `SPSR`) | +| `UMA` (9) | 0 | EL0 cannot mask interrupts, so it cannot defeat preemption | +| `DZE` (14) | 1 | EL0 `DC ZVA` is a store its mapping already permits | +| `UCT` (15), `UCI` (26) | 0 | EL0 does no cache maintenance; the kernel does it at `map RX` (§9) | +| `nTWI` (16), `nTWE` (18) | 0, 1 | `wfi` traps and `wfe` does not — RFC-0006 (proposed) §8 | +| `WXN` (19) | 0, then 1 | set by Rust step (5): `0x30DC581D` | +| `E0E` (24), `EE` (25) | 0 | little-endian | +| 11, 20, 22, 23, 28, 29 | 1 | RES1 on Armv8.0: Linux v4.19's `SCTLR_EL1_RES1`, plus bit 23 (`SPAN`, RES1 before Armv8.1), which Linux always sets; not re-checked against the Arm ARM | + +Every other bit is 0: RES0 on Armv8.0, or a later extension's control left off. Rust then, in an order that +never leaves the reporter without a console: (1) reads the DTB and builds the `MemoryMap`, bitmap and slot +arrays through the **boot window** — the frame-access trait takes a window offset, here `pa − 0x4000_0000 + +0xFFFF_FFFF_8000_0000`, so all RAM touched this early lies in the first boot GiB; (2) builds §6's final +tables, stack window included; (3) swaps `TTBR1_EL1` by an assembly routine run from the identity map — empty +table, `TLBI VMALLE1`, final table — since turning blocks into pages under a live translation needs +break-before-make *(Linux `idmap_cpu_replace_ttbr1`)*, then re-references the bitmap and switches the trait +to the physmap offset; (4) **repoints every kernel MMIO user at the MMIO window** — the UART, and the GIC +if RFC-0006 has brought it up; (5) sets `TCR_EL1.EPD0 = 1` (cleared by the first AddressSpace install), sets +`WXN`, and issues `TLBI VMALLE1`, `DSB ISH`, `ISB`. **The boot tables are never freed**: they are 16 KiB of +`.bss`, and freeing them would give image frames a second owner. + +The reporter gains a DFSC and fault-level decode (it prints `FAR` already); on x86_64 `#PF` prints `CR2` and +its error code. **x86_64**, once the UEFI stub exists, is already paging; Rust builds §6's PML4 with a +temporary identity entry for a trampoline page, sets `IA32_EFER.NXE` (bit 11), `CR0.WP` (bit 16), `CR4.PGE` +(bit 7) and `CR4.PCIDE`/`SMEP`/`SMAP` (bits 17, 20, 21) where CPUID permits; assembly loads `CR3`, jumps to +the high alias and drops the identity entry — the same "Rust runs only high" rule. *Rejected:* a separate loader *(seL4's elfloader)*; Rust building tables with the MMU off, sound only without absolute addresses. ## 14. Lineage | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4** | frames as capability objects; the kernel window; never touching user addresses; all remaining memory to the first task; user-chosen layout | Untyped/retype and the CDT; execute as an ungoverned attribute; ASID pools as capabilities | -| **pKVM** *(research/0002 Part 7)* | one owner per frame; share and donate as the only transitions | stage-2 mechanics; a separate page-state table | -| **Zircon / Fuchsia** | a region mapped into many spaces, kept alive by mappings; `EXECUTE` as a right; `VmexResource` | the kernel heap; lazily committed VMOs; VMARs | -| **Hubris** | build-time-sized kernel tables | a fully static task set | -| **Fiasco.OC, Genode** | kernel memory as a counted, tradeable quota | quotas on a general heap | -| **Linux** | the linear map; ASID rollover; arm64 descriptor headers as a checked reference; software PAN as an option | the buddy allocator; `copy_from_user` and its fixups | +| **seL4** | frames as capability objects; the kernel window; never touching user addresses; all memory to the first task; caller-paid tables | Untyped/retype and the CDT; execute as an ungoverned attribute; ASID pools | +| **pKVM** *(research/0002 Part 7)* | one owner per frame; *share* | *donate*, until RFC-0003a; stage-2 mechanics | +| **Zircon / Fuchsia** | a region mapped into many spaces, kept alive by mappings; `EXECUTE` as a right; `VmexResource` | the kernel heap; lazy VMOs; VMARs | +| **Hubris; Fiasco.OC, Genode** | build-time-sized tables; kernel memory as a counted, tradeable quota | a static task set; quotas on a heap | +| **Linux** | the linear map, its text alias read-only; ASID rollover and `mm_cpumask`; per-core PCIDs; arm64 headers as a checked reference | the buddy allocator; `copy_from_user` and its fixups | | **Ge et al., EuroSys '19** | the colour-capable allocator | time protection itself, for later | ## 15. Obligations | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| -| O-3 revocability | destruction and `reissue` walk the Region's mapping list, invalidate, then bump; a page-table entry is a cached resolution RFC-0003's invariant cannot cover, so the generation bump alone is not enough for memory (§7, §10) | **destruction case extended to memory**; selective revocation stays RFC-0003a's | -| O-5 argument validation | no user virtual address ever dereferenced; memory reached by capability through the physmap; snapshot-validate peer-writable data (§8, §11) | **discharged for the memory surface**; RFC-0007 inherits "no pointer arguments" | -| O-6 kernel memory safety | new `unsafe` confined to `mm/**` (physmap, volatile copy) and `arch/**` (tables, TLB, boot); engine, allocator and store are safe host-tested code | **stays Built** — every new block listed per PR | -| O-7 no unprivileged exhaustion | typed slot arrays; every post-boot frame and slot charged to a named Pool; charge-first atomic `map`; bounded work per call (§5, §8) | **discharged for memory and object slots**; CPU time is RFC-0006's half | -| O-8 address-space disjointness | per-process tables; Regions only by capability; zero before exposure; invalidate before free; privileged-only kernel half (§4, §7, §12) | **discharged** | +| O-3 revocability | destruction walks the Region's mapping list, invalidates, then bumps; a page-table entry is a cached resolution (§7, §10) | **destruction case extended to memory**; selective revocation and donation are RFC-0003a's | +| O-5 argument validation | no user virtual address dereferenced; carried ones validated as numbers; memory by capability through the physmap; snapshot-validate peer-writable data (§11) | **discharged for the memory surface**; RFC-0007 inherits §11's sentence | +| O-6 kernel memory safety | new `unsafe` confined to `mm/**` (window access, volatile copy, the store's one static) and `arch/**` (tables, TLB, boot) | **stays Built** — every new block listed per PR | +| O-7 no unprivileged exhaustion | typed arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map` included; charge-first; `map` ≤ 512 leaves; zeroing in 512-page steps; run search bounded by RAM (§4, §5, §8) | **discharged for kernel allocation**; a transferred object stays charged to its payer until destroyed — **open** with RFC-0003a (§18.2); CPU time is RFC-0006's | +| O-8 address-space disjointness | per-process tables; Regions only by capability; zero before exposure, table frames too; invalidate before free (§4, §7, §12) | **discharged** | | O-9 no ambient side channel | no shared mapping a capability did not establish (§10) | **contributes** | -| O-13 W^X | four layers, including per Region across all address spaces (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | -| O-2, O-4 | `perms` ⊆ capability rights; `protect` narrows; no `brk`, anonymous `mmap` or physical-address map | **maintained** | +| O-13 W^X | four layers, per Region across all address spaces, counts falling only on completed invalidation (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | +| O-2, O-4 | `perms` ⊆ capability rights; `protect` narrows; spending needs a Pool capability; no physical-address map | **maintained** | | O-18 confined DMA | single-extent Regions; Device Regions minted at boot | **groundwork only** — Phase 2 | ## 16. Graves checked (§3) - **Policy in the kernel.** No address choice, demand paging, copy-on-write, overcommit or OOM response. -- **Baroque capability hierarchies.** No Untyped tree; Pools are flat and balance moves rather than derives. +- **Baroque capability hierarchies.** No Untyped tree; Pools are flat, with one funder index for closure. - **Multi-copy IPC.** The slow path copies once, frame to frame; bulk data is shared, never copied (§10). -- **Bolted-on multicore.** Broadcast invalidation from the first line; the tag allocator is the SMP design. -- **The catch-all right.** `EXECUTE` is narrow; there is deliberately no "map any physical address" right. +- **Bolted-on multicore.** Broadcast invalidation from the first line; shootdown, rollover and per-core tags + designed now (§12); only the IPI is RFC-0006's. +- **The catch-all right.** `EXECUTE` is narrow and refused off-type; no "map any physical address" right. - **Compiled-in unused device paths (VENOM).** The kernel maps only the devices it drives itself. ## 17. Costs — what this makes harder - **Capacities are ceilings**: raising a slot count is a rebuild, and `pool_move` is one call per resource. -- **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `TooFragmented`. -- **47-bit halves on AArch64** give up half of what `T0SZ = 16` allows, for one layout and one user ABI; - **the 64 KiB floor is a fixed constant**, where Linux lets the administrator tune it. -- **No pointer arguments** (§11): an ergonomic cost on every syscall RFC-0007 designs, for a bug class gone. +- **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `Fragmented`. +- **`map` needs the AddressSpace's own Pool**: a loader mapping into a child holds the child's Pool. +- **47-bit halves on AArch64** give up half of what `T0SZ = 16` allows; **the 64 KiB floor is fixed**. +- **User addresses are numbers only** (§11): RFC-0007 designs every syscall around registers, the IPC + buffer and Region handles, for a bug class gone. - **The physmap is a large target**: one kernel write bug reaches all RAM, user code included (§9) — a software invariant, as are W^X on x86_64 and O-5 without PAN on cortex-a72. -- **Cache maintenance is untestable under QEMU** (§9, §13); and **the load address is a QEMU `virt` fact**. -- **Reclaim needs cooperation or revocation**: stale capabilities pin slots and transferred ones keep a Pool - charged, so `pool_merge` waits — RFC-0003a's problem made concrete. +- **At every boot the kernel runs RWX** from MMU-on to Rust step (5), before any process exists; a stray + early write goes uncaught. +- **Cache maintenance is untestable under QEMU** (§9, §13); **the load address is a QEMU `virt` fact**. +- **Reclaim across a transfer needs cooperation**: a transferred object keeps its payer charged until it is + destroyed, so `pool_merge` waits on other processes — RFC-0003a's problem made concrete. ## 18. Open questions 1. **Page-table frames on unmap.** Freed only with the AddressSpace; eager freeing needs §12's walk-cache rule. -2. **One accounting domain?** research/0002 Part 6 wants memory, object quotas and scheduling contexts under - one capability-named domain; the Pool can take time as another resource. RFC-0006 decides. -3. **Which right authorises `reissue` and destruction.** Proposed `REVOKE` (§10); RFC-0003a and RFC-0007 settle it. +2. **Donation and reclaim across a transfer** — `reissue` with a charge move, offered to RFC-0003a (§10). +3. **Which right authorises explicit destruction** — `REVOKE` would widen it as `reissue` would; RFC-0003a. 4. **Hardware backing for two software rules:** software PAN on Armv8.0 (a `TTBR0_EL1` change per kernel entry), and unmapping executable user frames from the physmap. -5. **Preemptible teardown.** The O(mappings) walk must be restartable to use RFC-0006 (proposed)'s preemption points. -6. **The firmware assumption** (§4) for threat model §10, via `docs/CHANGELOG.md`. +5. **Preemptible teardown** at RFC-0006 (proposed)'s preemption points: the O(mappings) walk made restartable. +6. **The firmware assumption** (§4) for threat model §10, and O-7's new text (§5), via `docs/CHANGELOG.md`. +7. **"Minimal assembly"** (Constitution §11.1): are runtime single-instruction `TLBI`/`DC`/`IC`/`TTBR` + intrinsics in `arch/**` within it (§9), or does the clause need amending? +8. **One programme order** across RFC-0005/0006/0007 — §19 proposes one for the maintainer to fix. ## 19. Implementation increments One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate — `no_std` in the kernel, `std` under `cargo test`, built for both targets in CI, no `unsafe`, as `capability/`. - -1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU still off. Boot-test `--expect "DTB 0xd00dfeed at 0x40000000"`. -2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs; - `CLAUDE.md` § Layout gains `memory/`. Boot-test `mm: 512 MiB usable at 0x40000000, N frames, M reserved`. -3. **[demo] Frame bitmap.** Host churn tests against a shadow model: no frame twice, none reserved. -4. **[demo] `EXECUTE` in `capability/`.** Bit 5, `ALL` edited by hand and its comment corrected, `Debug`, - the test widened to `0..64`. Host-only; demo-needed because the root maps the client's text. -5. **[demo] `MapPerms` and both encoders.** Exhaustive host tests: nothing writable and executable; every - user entry `PXN`, `AF`, not-global; every row of §8's table. -6. **[demo] The table engine** over a frame-access trait. Host tests on both formats: overlap, kernel-half - refusal, the 512-leaf and six-table bound, nothing mapped when a table cannot be paid for. -7. **[demo] Object store and Pools (§5).** Host tests: conservation under random create, destroy, split, - move and merge; `PoolExhausted` names the payer. -8. **[demo] MMU on (§13, stub).** Higher-half relink, boot tables, `VBAR_EL1` high, `FAR_EL1`/DFSC decoding. - Boot-test: `Kaya!` from the high alias, and the exception self-test still passes. -9. **[demo] Final kernel map (§13, Rust)**, the console repointed in the same PR; `EPD0`, `WXN`. Boot-test - `--features provoke-wx` writes to `.text`, expecting `data abort, same EL` and `FAR = 0xffffffff8…`. -10. **[demo] Region, AddressSpace, Mapping** (§7–§9, §12) charged to the root Pool, with per-Region W^X - counts, the tag allocator (rollover host-tested) and `TTBR0_EL1` switching. Boot-test: `AT S1E0R` under - two ASIDs resolves one shared Region to the same frame and private ones apart: `mm: shared region agrees`. -11. **User-memory module (§11).** Boot-test with a message beyond the register budget. Not demo. -12. **Destruction and `reissue`.** Boot-test that a reissued sharer faults. Not demo. -13. **PAN, SMAP, SMEP** where reported, with a `provoke-user-deref` self-test under `-cpu max`. Not demo. -14. **x86_64** tables, `CR3` and PCID, after the UEFI stub. Not demo. -15. **Device Regions** — Phase 2, with the first userspace driver. +Each new `arch` function (memory map, window access, TLB, address-space switch) gets a documented x86_64 +stub: the memory-map source returns an empty `MemoryMap`, and memory initialisation prints `mm: no memory +map (UEFI stub pending)` and halts, so the second Tier-1 build compiles the whole path from increment 2. + +1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU off. `--expect "DTB 0xd00dfeed at 0x40000000"`. +2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs, and + on a recorded OVMF `GetMemoryMap` dump once the stub exists; touches the workspace `members`, + `.github/workflows` and `CLAUDE.md` § Layout. `--expect "mm: 512 MiB RAM at 0x40000000"`. +3. **[demo] Frame bitmap.** Host churn tests against a shadow model: no frame twice, none reserved, no image + frame ever free. +4. **[demo] `EXECUTE` in `capability/`.** Bit 5, `ALL` and its comment, `Debug`, the test widened to `0..64`; + with RFC-0003's dated amendment. Host-only; demo-needed because the root maps the client's text. +5. **[demo] `MapPerms` and both encoders.** Exhaustive: nothing writable and executable; every user entry + `PXN`, `AF`, not-global; every row of §8's table. +6. **[demo] The table engine** over the frame-access trait. Host tests on both formats: overlap, the 512-leaf + and six-table bound, nothing mapped when unpaid, tables zeroed before linking, a kernel-half map never + writing the root. +7. **[demo] 7a — slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale + references count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. The one static. +8. **7b — `pool_split`/`move`/`merge` and closure forwarding.** Host tests under random churn. Not demo. +9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode. + Prints the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. +10. **[demo] Final kernel map (§13, Rust)**: physmap with the image read-only, MMIO window with every kernel + MMIO user repointed, stack window with core 0's stack moved behind a guard; `EPD0`, `WXN`. Three + self-tests, one `--expect` each: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the + image and the physmap, expecting `data abort, same EL`; `provoke-wxn` branches into `.data`, expecting + `instruction abort, same EL`. +11. **[demo] 11a — Region and Mapping records** with the W^X counts; host-only in `memory/`. +12. **[demo] 11b — the tag allocator**: rollover, per-core reservation, return on destruction; host-only. +13. **[demo] 11c — AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first + install. Boot-test: `AT S1E0R` under two ASIDs resolves one shared Region to one frame and private ones + apart: `mm: shared region agrees`. +14. **[demo] 11d — Pool, Region and AddressSpace methods through RFC-0007's `call`**, handle words in message + registers, with RFC-0007 increment 6: an EL0 self-test maps a fresh Region and prints `[el0] map ok`. +15. **`unmap` and `protect`** with invalidation and count decrements (§9, §12). Not demo. +16. **User-memory module (§11).** A message beyond the register budget. Not demo. +17. **Destruction** (§10), after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches its + handler. Not demo. +18. **Stack overflow**, with RFC-0006's overflow stack: `provoke-stack-overflow` reaches the reporter. +19. **PAN, SMAP, SMEP** where reported, with a `provoke-user-deref` self-test under `-cpu max`. +20. **x86_64** tables, `CR3` and PCID, after the UEFI stub; **Device Regions** in Phase 2. + +**Programme order proposed** (§18.8): 1–7 beside RFC-0006 1–2 and RFC-0007 1–5; RFC-0007's MMU-off EL0 +self-test (its increment 3) lands before 9, which retires it; 9–10 before RFC-0006 3 — or 10 repoints the GIC +too — and before RFC-0006 4, which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7. **Interims the demo carries, with their cost:** -- **One root Pool.** `pool_split`/`move`/`merge` are host-tested, not yet syscalls: server and client - charges are indistinguishable. -- **No destruction or `reissue`**: a killed process's frames and slots stay charged; O-3's memory case is +- **One root Pool.** 7b is not a syscall yet: server and client charges are indistinguishable. +- **No destruction or `unmap`**: a killed process's frames and slots stay charged; O-3's memory case is designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. - **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. -- **One RWX gigabyte** between MMU-on and increment 9, before any process exists; cache maintenance unverified. -- **Not an interim:** the kernel maps the root's image at boot — RFC-0007 (proposed)'s one bootstrap act — - through increment 10's `map` with kernel-minted Region capabilities, so §8's rights check runs even there. +- **Until increment 10, the kernel runs RWX throughout**; cache maintenance stays unverified. +- **1 GiB boot blocks over unbacked space** (§13): real hardware needs 2 MiB blocks sized from the DTB. +- **Zeroing in one step** until RFC-0006's preemption points; the demo's Regions are a few pages. +- **An overflow of the guarded stack is a recursive abort** until increment 18. +- **Not an interim:** the root's image reaches it through boot-minted and kernel-filled Regions and + increment 13's `map`, so §8's rights check runs even at boot. ## 20. What this unblocks -Increments 1–10 give RFC-0006 (proposed) address spaces, guarded kernel stacks and a Pool to charge threads -to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a constraint -it inherits rather than discovers: **no syscall takes a pointer argument**. The root task receives all memory -as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor; increment 12 closes RFC-0004 -§9.1 in code. Increments 3–7 are host-only and 1–2 change only where the kernel loads and what it prints: all can land now. +Increments 1–14 give RFC-0006 (proposed) address spaces, a guarded per-core stack and a Pool to charge +threads to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a +constraint it inherits rather than discovers: **the kernel never dereferences a user virtual address; where +a syscall carries one, it is a number, validated as a number and used only to program tables or registers.** +The root task receives all memory as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient +grantor. Increments 3–8 are host-only and 1–2 change only where the kernel loads and what it prints: all can +land now. From b5c0154f5515f9fa79a2dd9083b70a81b1cce4d1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:32:32 +0000 Subject: [PATCH 10/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index 836fd82..d4dc803 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -115,7 +115,7 @@ once with its generation advanced**, stale capabilities fail closed on it — ex generation exists to allow — and no past holder can pin another process's charge. A slot whose generation cannot advance is retired, its unit debited once from its payer and thereafter kept by the kernel (objects keep 64-bit generations, RFC-0007 (proposed) §6, so this is a boundary, not a budget). `object.rs`'s -"takes a reference count" comment gains that qualification in increment 7a. +"takes a reference count" comment gains that qualification in increment 7. **Where it lives.** The generic slot array and Pool are safe code in `memory/`, generic over the object type. The concrete store — one `Kernel` value holding every type's array, RFC-0004/0006/0007 objects included — is @@ -171,7 +171,7 @@ top 2 GiB, which `x86_64-unknown-none`'s default kernel code model expects; over | `0xFFFF_FFFF_8000_0000` – top | image at `+0x20_0000` | `.text` RX, `.rodata` R, `.data`/`.bss` RW execute-never | - **Stacks.** RFC-0006 (proposed) V1 chose one kernel stack per core; they are allocated at boot, and core - 0's boot stack moves from `aarch64.ld`'s unguarded slot into this window in increment 9. A guard page + 0's boot stack moves from `aarch64.ld`'s unguarded slot into this window in increment 10. A guard page makes an overflow a fault, but the fault's entry then pushes onto the same stack: **RFC-0006 (proposed) provides the overflow stack** (x86_64 IST; on AArch64 the EL1-origin vector switching to the emergency stack, since a kernel fault is fatal). Until then an overflow is a recursive abort — detected, not diagnosed. @@ -251,7 +251,7 @@ needs one". Folding execution into `READ` makes every readable Region potential the loader, which writes code before running it. seL4 leaves execution an ungoverned attribute; Fuchsia's `ZX_RIGHT_EXECUTE` is the lineage. It means *may map executable* on a Region and *may mint executable Regions* on a Pool, and is **refused at mint on every other type**: the store's mint path rejects it -(host-tested, increment 7a), and `derive` cannot add it (O-2). `rights.rs`'s `ALL` is an explicit union +(host-tested, increment 7), and `derive` cannot add it (O-2). `rights.rs`'s `ALL` is an explicit union despite its comment; increment 4 edits both. **On acceptance, RFC-0003 gains a dated amendment adding `EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 §14.3's handshake is one rule: **the installed permission is the requested `perms`, refused unless a subset of the Region capability's rights.** @@ -507,22 +507,22 @@ map (UEFI stub pending)` and halts, so the second Tier-1 build compiles the whol 6. **[demo] The table engine** over the frame-access trait. Host tests on both formats: overlap, the 512-leaf and six-table bound, nothing mapped when unpaid, tables zeroed before linking, a kernel-half map never writing the root. -7. **[demo] 7a — slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale - references count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. The one static. -8. **7b — `pool_split`/`move`/`merge` and closure forwarding.** Host tests under random churn. Not demo. +7. **[demo] Slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale + references count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. Host-only. +8. **`pool_split`/`move`/`merge` and closure forwarding.** Host tests under random churn. Not demo. 9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode. Prints the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. 10. **[demo] Final kernel map (§13, Rust)**: physmap with the image read-only, MMIO window with every kernel MMIO user repointed, stack window with core 0's stack moved behind a guard; `EPD0`, `WXN`. Three - self-tests, one `--expect` each: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the + self-tests, one `--expect` each, and the store behind its one static: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the image and the physmap, expecting `data abort, same EL`; `provoke-wxn` branches into `.data`, expecting `instruction abort, same EL`. -11. **[demo] 11a — Region and Mapping records** with the W^X counts; host-only in `memory/`. -12. **[demo] 11b — the tag allocator**: rollover, per-core reservation, return on destruction; host-only. -13. **[demo] 11c — AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first +11. **[demo] Region and Mapping records** with the W^X counts; host-only in `memory/`. +12. **[demo] The tag allocator**: rollover, per-core reservation, return on destruction; host-only. +13. **[demo] AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first install. Boot-test: `AT S1E0R` under two ASIDs resolves one shared Region to one frame and private ones apart: `mm: shared region agrees`. -14. **[demo] 11d — Pool, Region and AddressSpace methods through RFC-0007's `call`**, handle words in message +14. **[demo] Pool, Region and AddressSpace methods through RFC-0007's `call`**, handle words in message registers, with RFC-0007 increment 6: an EL0 self-test maps a fresh Region and prints `[el0] map ok`. 15. **`unmap` and `protect`** with invalidation and count decrements (§9, §12). Not demo. 16. **User-memory module (§11).** A message beyond the register budget. Not demo. @@ -538,7 +538,7 @@ too — and before RFC-0006 4, which needs the stack window; 11–13 before RFC- **Interims the demo carries, with their cost:** -- **One root Pool.** 7b is not a syscall yet: server and client charges are indistinguishable. +- **One root Pool.** Increment 8 is not a syscall yet: server and client charges are indistinguishable. - **No destruction or `unmap`**: a killed process's frames and slots stay charged; O-3's memory case is designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. - **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. From 6ba18c6388ce939b1df42896f0e8b0c48597343a Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:33:42 +0000 Subject: [PATCH 11/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Carries the reviser's rewrap of RFC-0005's verdict box, which clears MD013. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 634 +++++++++--------- 1 file changed, 301 insertions(+), 333 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index d4dc803..095fb25 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -7,24 +7,25 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 (kernel doctrine) and its "minimal assembly" clause (§18.7); RFC-0003 (a dated amendment adding one right; three new objects; `capability/src/rights.rs` and `object.rs`'s doc comment); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 and O-7's text (proposals via `docs/CHANGELOG.md`) | +| Affects | Constitution §3 (kernel doctrine) and §11.1's "minimal assembly" (§18.7); RFC-0003 (a dated amendment adding one right; three new objects; `capability/src/rights.rs` and `object.rs`'s doc comment); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 and O-7's text (proposals via `docs/CHANGELOG.md`) | | Depends on | RFC-0003 (capability table), RFC-0004 (IPC); interlocks with RFC-0006 (threads and scheduling) and RFC-0007 (system calls and process bootstrap), proposed the same day | | Discharges | O-7 for kernel allocation (reclaim across a transfer stays open), O-8, O-13 (the mapping half); O-5 for every path by which the kernel touches user memory; O-3's destruction case extended to memory; answers RFC-0003 §14.3 and RFC-0004 §9.1 (share) | -> **Proposed verdicts.** **[demo]** marks what tonight's demo stands on; the rest is forward design. +> **Proposed verdicts.** **[demo]** marks what the demo stands on; the rest is forward design. > -> 1. **[demo] Discovery:** firmware tables into one bounded `MemoryMap`; the image moves to 0x4020_0000, because QEMU places no DTB for an ELF loaded at the base of RAM (§4). -> 2. **[demo] Frames:** a colour-capable bitmap over 4 KiB frames; fresh frames are zeroed, boot data is handed over read-only, so no frame reaches a process holding another's data (§4, §7). -> 3. **[demo] Who pays (O-7):** typed slot arrays reused at once by generation; every post-boot frame and slot charged to a `Pool` presented by capability — `map` included; the Pool is the one accounting domain for counted units (§5). -> 4. **[demo] Layout:** a higher-half kernel with the same layout on both architectures; 47-bit halves; a non-executable physmap in which the image is read-only; per-core stacks in a guarded window, kept by the kernel (§6). -> 5. **[demo] Objects:** `Region` (eager and zeroed, or boot-minted read-only over boot data) and `AddressSpace` join RFC-0003's objects beside `Pool`; each mapping is a charged record (§7). -> 6. **[demo] Operations:** `map` / `unmap` / `protect` are mechanical, capability-checked, bounded and atomic; `protect` only narrows; the kernel never picks an address (§8). -> 7. **[demo] Rights:** `EXECUTE` joins RFC-0003's set as bit 5 — map-executable on a Region, mint-executable on a Pool, refused at mint on every other type — by dated amendment (§8). -> 8. **[demo] W^X (O-13):** unrepresentable in the mapping type, enforced per Region across all address spaces, backed by `SCTLR_EL1.WXN` (§9). -> 9. **Sharing (RFC-0004 §9.1):** *share* is derive-and-transfer; destruction walks the Region's mappings; *donation* is not proposed here but handed to RFC-0003a as input (§10). -> 10. **[demo] User memory (O-5):** the kernel never dereferences a user virtual address; where a syscall carries one, it is a number, validated as a number (§11). -> 11. **TLB:** tags are kernel-private and recycled; this RFC owns the shootdown and rollover protocol for SMP, RFC-0006 provides the IPI; single-core half **[demo]** (§12). -> 12. **[demo] Early boot:** the stub enables the MMU with a stated `SCTLR_EL1`; Rust runs only at the high alias and installs the final W^X map (§13). +> 1. **[demo] Discovery:** firmware tables into one bounded `MemoryMap`; the image moves to 0x4020_0000, where QEMU leaves room for the DTB (§4). +> 2. **[demo] Frames:** a colour-capable bitmap; fresh frames and table frames zeroed, boot data handed over read-only (§4, §7). +> 3. **[demo] Who pays (O-7):** typed slot arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map`'s too (§5). +> 4. **[demo] One accounting domain:** the Pool, for every counted unit; time stays with RFC-0006's `Core`, a stated split from research/0002 (§5). +> 5. **[demo] Layout:** the same higher-half layout on both; the image read-only in the physmap; per-core stacks guarded, kept by the kernel (§6). +> 6. **[demo] Objects:** `Region` (eager and zeroed, or boot-minted read-only) and `AddressSpace` join RFC-0003's objects beside `Pool` (§7). +> 7. **[demo] Operations:** `map` / `unmap` / `protect` are capability-checked, bounded and atomic; the kernel never picks an address (§8). +> 8. **[demo] Rights:** `EXECUTE` joins RFC-0003's set as bit 5 — on Regions and Pools only, refused at mint elsewhere — by dated amendment (§8). +> 9. **[demo] W^X (O-13):** unrepresentable, enforced per Region across all address spaces, backed by `SCTLR_EL1.WXN` (§9). +> 10. **Sharing (RFC-0004 §9.1):** *share* is derive-and-transfer; destruction walks mappings; *donation* goes to RFC-0003a as input (§10). +> 11. **[demo] User memory (O-5):** the kernel never dereferences a user virtual address; a carried one is a number, validated as one (§11). +> 12. **TLB:** kernel-private, recycled tags; this RFC owns SMP shootdown and rollover, RFC-0006 provides the IPI; single-core half [demo] (§12). +> 13. **[demo] Early boot:** the stub enables the MMU with a stated `SCTLR_EL1`; Rust runs only high and installs the final W^X map (§13). ## 1. The question @@ -42,8 +43,8 @@ threat model files integrity *in memory* under A1, and W^X is how signed code st ## 3. What the hardware gives — one shape, two encodings Both Tier-1 MMUs, as configured here, are one machine: **four levels of 512 eight-byte entries, 4 KiB pages, -and 47-bit user and kernel halves whose top level uses 256 entries each.** AArch64 gets this from the 4 KiB -granule with `TCR_EL1.T0SZ = T1SZ = 17`; x86_64 from IA-32e four-level paging without LA57. +and 47-bit user and kernel halves whose top level uses 256 entries each** — AArch64's 4 KiB granule with +`TCR_EL1.T0SZ = T1SZ = 17`, x86_64's four-level paging without LA57. The leaves differ: | Property | AArch64 (VMSAv8-64, 4 KiB granule) | x86_64 (IA-32e, 4-level) | |---|---|---| @@ -56,205 +57,188 @@ granule with `TCR_EL1.T0SZ = T1SZ = 17`; x86_64 from IA-32e four-level paging wi | Accessed | `AF` (bit 10), set by software on this CPU | bit 5 (A), set by hardware | | Address-space tag | 16-bit ASID in `TTBR0_EL1[63:48]` | 12-bit PCID in `CR3[11:0]` | -*(Bits checked against Linux's `arch/arm64/include/asm/pgtable-hwdef.h` — Arm's site was unreachable from -this session — and Intel SDM Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, -no hardware access-flag update and no PAN: `target/arm/tcg/cpu64.c` sets `ID_AA64MMFR0_EL1 = 0x1124` and -leaves `ID_AA64MMFR1_EL1` at 0.)* Hence **one table engine**, generic over a per-architecture `Descriptor` -encoder and a frame-access trait, host-tested on both against a fake frame store in `memory/`; only turning -a frame number into `&mut [u64; 512]` through a kernel window (§6, §13) is `unsafe`, in `kernel/src/mm/`. +*(Bits checked against Linux's `arch/arm64/include/asm/pgtable-hwdef.h` — Arm's site was unreachable — and +Intel SDM Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, no hardware +access-flag update and no PAN: `target/arm/tcg/cpu64.c` sets `ID_AA64MMFR0_EL1 = 0x1124`, `MMFR1` 0.)* +Hence **one table engine**, generic over a per-architecture `Descriptor` encoder and a frame-access trait, +host-tested on both in `memory/`; only turning a frame number into `&mut [u64; 512]` through a kernel window +(§6, §13) is `unsafe`, in `kernel/src/mm/`. ## 4. Physical memory: discovery and the frame allocator -**Discovery.** QEMU documents two fixed facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every -other address "may vary across QEMU versions" and must be read from the DTB, which a bare-metal ELF finds -"at the start of RAM" with no register pointing at it (`docs/system/arm/virt.rst`). `hw/arm/boot.c` puts it -there only below the image (`dtb_limit` is the image's lowest address), and today's image *is* the base of -RAM. Checked on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM** and `x0 = 0`; at -0x4020_0000 one is found (magic `0xd00dfeed`) at 0x4000_0000. **So the image loads at 0x4020_0000**, 2 MiB in -and block-aligned — which caps the DTB at 2 MiB; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT -reader, written for the purpose (Borrow Ledger boot-path row; Devicetree Specification v0.4), reads the cell -sizes, `/memory`, `/memreserve/` and `/reserved-memory`, bounds-checks every offset and fails closed. Threat -model §10 names no firmware assumption; this RFC proposes one — *firmware tables describe memory -truthfully* — through `docs/CHANGELOG.md`. On x86_64 the UEFI stub (owned by no RFC yet) hands over -`GetMemoryMap`'s output after `ExitBootServices`, when boot-services ranges become usable. **Both produce -one `MemoryMap`**: adjacent same-kind ranges coalesced, then at most **256** sorted, disjoint `(PhysRange, -Kind)` entries (6 KiB of `.bss`; real UEFI maps run to 100–200 raw descriptors) — `Usable`, `KernelImage`, -`DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that, with image, boot stack, DTB and boot tables -tagged before the allocator sees it. In Phase 1 the `Mmio` entries are the hard-coded interim addresses -(§19). Nothing above `arch` names FDT or UEFI. - -**The frame allocator is a bitmap**: one bit per 4 KiB frame, its storage taken from the first free frames -(16 KiB for 512 MiB). It allocates from a *colour set*, or runs of `n` at an alignment; a double free fails -closed. Colour is `pfn mod C`, `C = 1` until time protection is wanted *(Ge et al., EuroSys '19, via -research/0002 Part 6)*. A run search is O(frames / 64) words — bounded by RAM, not by what the caller may -spend; a per-alignment cursor is the named remedy if measurement asks. **Fresh frames are zeroed before a -process can observe them**, and **a page-table frame is zeroed before its parent descriptor is written** -(a `DSB` between) — a recycled table with stale valid entries would translate into another process's -frames, an O-8 breach no process ever "observes". *Rejected:* a **buddy allocator** *(Linux)*, whose +**Discovery.** QEMU fixes two facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every other +address "may vary" and must be read from the DTB, found "at the start of RAM" with no register pointing at it +(`docs/system/arm/virt.rst`). `hw/arm/boot.c` places it only *below* the image (`dtb_limit` is the image's +lowest address). Checked on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM**; +at 0x4020_0000 one is found at 0x4000_0000. **So the image loads at 0x4020_0000**, block-aligned, capping +the DTB at 2 MiB; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT reader (Borrow Ledger boot-path +row; Devicetree Specification v0.4) reads the cell sizes, `/memory`, `/memreserve/` and `/reserved-memory`, +bounds-checks every offset and fails closed. This RFC proposes threat model §10's first firmware assumption — +*firmware tables describe memory truthfully* — via `docs/CHANGELOG.md`. On x86_64 the UEFI stub (no RFC yet) +hands over `GetMemoryMap` after `ExitBootServices`. **Both produce one `MemoryMap`**: same-kind neighbours +coalesced, then at most **256** sorted, disjoint `(PhysRange, Kind)` entries (6 KiB; raw UEFI maps run to +100–200) — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that. Image, +boot stack, DTB and boot tables are tagged first; Phase 1's `Mmio` entries are the interim hard-coded +addresses (§19). Nothing above `arch` names FDT or UEFI. + +**The frame allocator is a bitmap** — one bit per 4 KiB frame, stored in the first free frames (16 KiB per +512 MiB) — allocating from a *colour set*, or runs of `n` at an alignment; a double free fails closed. +Colour is `pfn mod C`, `C = 1` until time protection *(Ge et al., EuroSys '19, via research/0002 Part 6)*. +A run search is O(frames / 64) words, bounded by RAM rather than by the caller; a per-alignment cursor is +the remedy if measurement asks. **Fresh frames are zeroed before a process can observe them, and a table +frame before its parent descriptor is written** (a `DSB` between): a recycled table with stale valid +entries would translate into another process's frames. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing fights colouring; a **free list**, which cannot find `n` contiguous frames without a scan. ## 5. Who pays: the object store and Pools (O-7) -The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped and retype** is -the proven strong form, but reusing an Untyped revokes everything retyped from it through a derivation -tree, which would decide RFC-0003a's B1 by the back door (RFC-0003 §7 and its 2026-07-30 amendment) and -make object layouts ABI. **Rejected.** **B — a kernel heap with creation policy** *(Zircon)* limits -*whether* a process creates objects, not *how much* memory they take; exhaustion is global and unnamed. -**Rejected.** **C — static per-type tables** *(Hubris)* are bounded but cannot say *who* filled one — -research/0002 Part 6's Kinesis outage. **Rejected alone.** **D — C's structure with counted attribution: -verdict sought** *(the requester pays, as in seL4; a counted quota, as in Fiasco.OC's factory limits and -Genode's quota trading)*. - -**The object store is typed slot arrays.** Each object type has one fixed-capacity array in `.bss`; a slot -holds the object, a generation and a count, and the slot index is the object's identity. The kernel's -implementation of `ObjectRef` (`capability/src/object.rs`, a trait) is `(type, index, generation)`, and -`current_generation()` is a slot read — no raw pointer, slab or size class. **Only live references count**: -capabilities minted at the slot's current generation, Mappings and IPC-buffer bindings; a reference whose -generation is not the slot's counts nothing, on clone or drop. So a destroyed object's slot is **reused at -once with its generation advanced**, stale capabilities fail closed on it — exactly what RFC-0003 §8's -generation exists to allow — and no past holder can pin another process's charge. A slot whose generation -cannot advance is retired, its unit debited once from its payer and thereafter kept by the kernel (objects -keep 64-bit generations, RFC-0007 (proposed) §6, so this is a boundary, not a budget). `object.rs`'s -"takes a reference count" comment gains that qualification in increment 7. - -**Where it lives.** The generic slot array and Pool are safe code in `memory/`, generic over the object type. -The concrete store — one `Kernel` value holding every type's array, RFC-0004/0006/0007 objects included — is -instantiated in the kernel crate and reached through **one** `static` wrapper in `kernel/src/mm/` carrying one -SAFETY argument: one core, the kernel entered with interrupts masked (RFC-0006 (proposed) V1), the big lock -when SMP lands. No other kernel global accretes *(research/0002 Part 6)*. - -**A Pool is the accounting layer**: an RFC-0003 object holding balances — frames, and slots of each type. At -boot every free frame and slot becomes the balance of one root Pool (RFC-0007 (proposed) hands it over). -Thereafter, for each resource `r`: +The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped and retype**: +reusing an Untyped revokes everything retyped from it through a derivation tree, which would decide +RFC-0003a's B1 by the back door (RFC-0003 §7 and its 2026-07-30 amendment) and make object layouts ABI. +**Rejected.** **B — a kernel heap with creation policy** *(Zircon)* limits *whether* a process creates +objects, not how much memory they take. **Rejected.** **C — static per-type tables** *(Hubris)* cannot say +*who* filled one (research/0002 Part 6's Kinesis outage). **Rejected alone.** **D — C with counted +attribution: verdict sought** *(the requester pays, as in seL4; a counted quota, as in Fiasco.OC's factory +limits and Genode's quota trading)*. + +**The store is typed slot arrays**, one fixed-capacity array per type in `.bss`; a slot holds the object, a +generation and a count. The kernel's implementation of `ObjectRef` (`capability/src/object.rs`, a trait) is +`(type, index, generation)`; `current_generation()` is a slot read. **Only live references count** — +capabilities at the slot's current generation, Mappings, IPC-buffer bindings; a reference at any other +generation counts nothing on clone or drop. A destroyed object's slot is therefore **reused at once, its +generation advanced**, and stale capabilities fail closed — what RFC-0003 §8's generation exists for — so no +past holder pins another's charge. A slot whose generation cannot advance retires, debited once from its +payer and kept by the kernel thereafter (objects keep 64-bit generations, RFC-0007 (proposed) §6). +`object.rs`'s "takes a reference count" gains that qualification in increment 7. + +**Where it lives.** The generic slot array and Pool are safe code in `memory/`. The concrete store — one +`Kernel` value with every type's array, RFC-0004/0006/0007 objects included — lives in the kernel crate +behind **one** `static` wrapper in `kernel/src/mm/` with one SAFETY argument: one core, kernel entered with +interrupts masked (RFC-0006 (proposed) V1), the big lock once SMP lands. No other kernel global accretes. + +**A Pool is the accounting layer**: an RFC-0003 object holding balances of frames and of each type's slots. +At boot every free unit becomes the balance of one root Pool (RFC-0007 (proposed) hands it over). Then: ```text Σ balance(r) over all Pools == units of r free -- at every instant after boot every unit of r in use == kept by the kernel (at boot, or retired), or charged to exactly one Pool ``` -- **Charged:** Region frames; every object slot; page-table frames and the `Mapping` slot, to the Pool - `map` names (§8). **Kept, never charged:** the image, bitmap, slot arrays, kernel tables, per-core - stacks and boot-minted Regions (§7). Each charge presents a Pool with `WRITE`; nothing spends ambiently. -- **Charge first, then take.** Conservation guarantees the allocator can supply; the only later failure, - `Fragmented` for a multi-frame extent, refunds before returning. -- **Failure is local and attributed.** `PoolExhausted` names the Pool and resource; nothing overcommits - *(research/0002 Part 6, "budgets are minima")*. Refund goes to the payer, whose index each object records. -- **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child with the parent's rights, - recording the parent as its funder; `pool_move(from, into, r, n)`; `pool_merge(from, into)` when `from` - has no outstanding charges. Balance is conserved, never minted — O-2's shape for quantity. -- **A Pool whose last capability closes** returns its free balance to its funder at once; its slot stays as - a forwarding record until its outstanding charges come home, each refund forwarded — a chain bounded by - the Pool slot count. The root Pool has no funder: closing it hands its balance to the kernel for good. +- **Charged:** Region frames; every object slot; table frames and the `Mapping` slot, to the Pool `map` + names (§8). **Kept, never charged:** image, bitmap, slot arrays, kernel tables, per-core stacks, + boot-minted Regions (§7). Every charge presents a Pool with `WRITE`. +- **Charge first, then take**, so the only later failure, `Fragmented`, refunds before returning. + **`PoolExhausted` names the Pool and resource**; nothing overcommits *(research/0002 Part 6, "budgets are + minima")*. Refund goes to the payer, whose index each object records. +- **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child recording its funder; + `pool_move(from, into, r, n)`; `pool_merge(from, into)` once `from` has no outstanding charges. +- **A Pool whose last capability closes** returns its free balance to its funder at once and stays as a + forwarding slot until its charges come home — a chain bounded by the Pool slot count. The root Pool has + no funder; closing it gives its balance to the kernel. - **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). -**One accounting domain — decided here.** research/0002 Part 6 asks for one capability-named domain to -which memory, object quotas and scheduling contexts attach. **The Pool is that domain for every counted -unit**: frames and every type's slots, Threads' and SchedContexts' included. **Time is not a unit**: -RFC-0006 (proposed) sells it as per-core bandwidth through `Core`, which moving balance cannot conserve, -and folding it in would make every Pool holder a seller of CPU. That is a deliberate split from the -research; congruence survives by attribution, since every SchedContext is charged to a Pool. RFC-0006 is -told so; O-7's text should then read *"kernel memory is charged to a capability-named Pool; CPU time to a -SchedContext funded through a `Core`"*, proposed through `docs/CHANGELOG.md`. +**One accounting domain — decided here, as the programme assigns O-7's model to this RFC.** research/0002 +Part 6 asks for one capability-named domain for memory, object quotas and scheduling contexts. **The Pool is +that domain for every counted unit**, Threads' and SchedContexts' slots included. **Time is not a unit**: +RFC-0006 (proposed) sells per-core bandwidth through `Core`, which moving balance cannot conserve, and +folding it in would make every Pool holder a seller of CPU — a deliberate split from the research, with +congruence kept by attribution, since every SchedContext is charged to a Pool. O-7's text would add +*"kernel memory is charged to a capability-named Pool; CPU time to a SchedContext funded through a +`Core`"*, proposed through `docs/CHANGELOG.md`. ## 6. The kernel's virtual layout Both architectures get the **same layout**, the cheapest way to keep the HAL honest. The image sits in the -top 2 GiB, which `x86_64-unknown-none`'s default kernel code model expects; overriding it is possible -(RFC-0007 does so for userspace) but buys nothing. AArch64 follows. +top 2 GiB, which `x86_64-unknown-none`'s default kernel code model expects (overridable, as RFC-0007 does +for userspace, but that buys nothing); AArch64 follows. | Range (both architectures) | Contents | Attributes | |----------------------------|----------|------------| | `0x0000_0000_0001_0000` – `0x0000_7FFF_FFFF_EFFF` | the current AddressSpace (§7) | per mapping; not-global; never kernel-executable | -| `0xFFFF_8000_0000_0000` – `+64 TiB` | physmap: every discovered RAM frame at `base + pa` | Normal write-back; RW, but the image's `.text`/`.rodata` read-only; execute-never; global | +| `0xFFFF_8000_0000_0000` – `+64 TiB` | physmap: every discovered RAM frame at `base + pa` | Normal write-back; execute-never; global; RW, but the image's `.text`/`.rodata` read-only | | `0xFFFF_C000_0000_0000` – `+1 TiB` | MMIO window: kernel-owned devices only (console, interrupt controller) | Device-nGnRE / uncached; RW; execute-never | -| `0xFFFF_FFFF_0000_0000` – `0xFFFF_FFFF_7FFF_FFFF` | per-core kernel and emergency stacks, an unmapped guard page below each | Normal; RW; execute-never; kept by the kernel at boot, never charged | +| `0xFFFF_FFFF_0000_0000` – `0xFFFF_FFFF_7FFF_FFFF` | per-core kernel and emergency stacks, an unmapped guard page below each | Normal; RW; execute-never; kept by the kernel at boot | | `0xFFFF_FFFF_8000_0000` – top | image at `+0x20_0000` | `.text` RX, `.rodata` R, `.data`/`.bss` RW execute-never | -- **Stacks.** RFC-0006 (proposed) V1 chose one kernel stack per core; they are allocated at boot, and core - 0's boot stack moves from `aarch64.ld`'s unguarded slot into this window in increment 10. A guard page - makes an overflow a fault, but the fault's entry then pushes onto the same stack: **RFC-0006 (proposed) - provides the overflow stack** (x86_64 IST; on AArch64 the EL1-origin vector switching to the emergency - stack, since a kernel fault is fatal). Until then an overflow is a recursive abort — detected, not diagnosed. -- **The user floor is 64 KiB**, so a kernel NULL dereference never lands on user data *(Linux - `mmap_min_addr`)*. **The user ceiling leaves the last page below the hole unmapped**, closing Intel's - `SYSRET` hazard (CVE-2012-0217) for RFC-0007's exit path by layout. The physmap covers RAM, never MMIO. +- **Stacks.** RFC-0006 (proposed) V1 chose one kernel stack per core, allocated here at boot; core 0's boot + stack leaves `aarch64.ld`'s unguarded slot in increment 10. A fault's entry pushes onto the stack that + overflowed, so **RFC-0006 (proposed) provides the overflow stack** (x86_64 IST; on AArch64 the EL1-origin + vector moving to the emergency stack, a kernel fault being fatal). Until then an overflow is a recursive abort. +- **The user floor is 64 KiB** *(Linux `mmap_min_addr`)*; **the ceiling leaves the last page below the hole + unmapped**, closing Intel's `SYSRET` hazard (CVE-2012-0217) for RFC-0007 by layout. The physmap is RAM only. - **The kernel's top-level entries never change after boot.** AArch64 shares `TTBR1_EL1`; x86_64 copies PML4 - entries 256–511 into each address space *(seL4 x86)*, so every kernel-half PML4 entry that will ever be - used — physmap for discovered RAM, MMIO window, stack window, image — is populated at boot; a PML4E first - filled later would be missing from every earlier copy (Linux's vmalloc-sync bug family). Lower-level kernel - tables may change; such a change is global and invalidates on every core (§12). *Rejected:* an - identity-mapped kernel — possible under an overridden code model, but it puts the kernel where user - addresses go and breaks the symmetric halves; no physmap (an invalidation per IPC-buffer touch). + entries 256–511 into each address space *(seL4 x86)*, so every kernel-half PML4E ever used is populated at + boot — one filled later would be missing from earlier copies (Linux's vmalloc-sync bug family). Lower + kernel tables may change; that is a global change, invalidated on every core (§12). *Rejected:* an + identity-mapped kernel — it puts the kernel where user addresses go and breaks the symmetric halves; no + physmap (an invalidation per IPC-buffer touch). ## 7. The objects: Region, AddressSpace and Mapping **Frame ownership is pKVM's state machine** (research/0002 Part 7): every frame has exactly one owner — the -kernel, or one `Region`. **A Region** is a fixed number of pages backed by at most eight extents of -contiguous frames. Rights are `READ`, `WRITE`, `EXECUTE`, with RFC-0003's `DUPLICATE`, `TRANSFER`, `REVOKE`. -Mappings and IPC-buffer bindings hold counted references, so closing the last capability does not pull -memory from under a live mapping *(Zircon: a mapping keeps its VMO alive)*. Three kinds, one object: - -- **Ordinary** — `region_create`: allocated eagerly, zeroed, charged; a request the bitmap cannot meet in - eight extents fails `Fragmented`; DMA-ready memory is one extent at an alignment. Zeroing is the long - part: it runs 512 pages per step between RFC-0006 (proposed)'s preemption points, the half-built Region - held on the calling thread, and no capability exists until it is done. -- **Boot-minted, read-only** — once, at boot, over boot data: the DTB and each boot module RFC-0007 - (proposed) embeds. Their `DeviceTree`/`KernelImage` frames **move** into the Region's ownership; they are - never zeroed (public boot data, so O-8 is untouched), never charged, and minted without `WRITE`: the DTB - `READ`, a module `READ | EXECUTE | DUPLICATE | TRANSFER`, so text maps `RX` in place and W^X holds because - no writable use can ever exist. RFC-0007's bundle must page-align every module segment. -- **Device** — minted at boot over `Mmio` ranges the kernel does not keep; never executable, zeroed or - charged; Phase 2. No operation maps a physical address given as a number: the catch-all right in costume. - -Regions the kernel fills for the root before handover (its writable segments, stack, BootInfo) are ordinary: -created and charged to the root Pool, bytes copied in through the physmap, and only then a capability minted. +kernel, or one `Region`. **A Region** is a fixed number of pages over at most eight extents of contiguous +frames, with rights `READ`, `WRITE`, `EXECUTE` and RFC-0003's `DUPLICATE`, `TRANSFER`, `REVOKE`. Mappings +and IPC-buffer bindings hold counted references, so closing the last capability does not pull memory from +under a live mapping *(Zircon: a mapping keeps its VMO alive)*. One object, three kinds: + +- **Ordinary** (`region_create`): eager, zeroed, charged; a request the bitmap cannot meet in eight extents + fails `Fragmented`; DMA memory is one extent at an alignment. Zeroing runs 512 pages per step between + RFC-0006 (proposed)'s preemption points, the half-built Region held on the caller's thread, no capability + existing until it is done. +- **Boot-minted, read-only**, once, over the DTB and each boot module RFC-0007 (proposed) embeds. Their + `DeviceTree`/`KernelImage` frames **move** into the Region; never zeroed (public boot data: O-8 is + untouched), never charged, never `WRITE`: the DTB gets `READ`, a module `READ | EXECUTE | DUPLICATE | + TRANSFER`, so text maps `RX` in place and W^X holds because no writable use can exist. RFC-0007's bundle + must page-align every module segment. +- **Device**, minted at boot over `Mmio` the kernel does not keep; never executable, zeroed or charged; + Phase 2. No operation maps a physical address given as a number: the catch-all right in costume. + +What the kernel fills for the root before handover (writable segments, stack, BootInfo) is ordinary Regions: +charged to the root Pool, bytes copied in through the physmap, only then a capability minted. **An AddressSpace** is a user root table, a tag (§12), the Pool paying for its tables — the only Pool `map` may charge — and its mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a process is RFC-0007's. **A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, vaddr, pages, perms}`, linked by slot index into its AddressSpace's list and its Region's. -**Why a Region must remember its mappings.** RFC-0003's 2026-07-30 amendment makes an invariant -load-bearing: *a resolved capability is never observed or cached outside the kernel table*. **A page-table -entry is exactly such a cache**, consulted by hardware and never re-checked. The per-Region mapping list is -the compensation, and the reason a generation bump alone cannot revoke memory. *Rejected:* **Zircon VMOs -and VMARs** — lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in kernel paths; -**seL4's capability per frame** — 256 handles per MiB; **userspace-built page tables** (Xen PV). +**Why a Region remembers its mappings.** RFC-0003's 2026-07-30 amendment makes load-bearing that *a resolved +capability is never cached outside the kernel table*. **A page-table entry is exactly such a cache**, which +hardware never re-checks; the per-Region list is the compensation. *Rejected:* **Zircon VMOs and VMARs** — +lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in kernel paths; **seL4's +capability per frame** — 256 handles per MiB; **userspace-built page tables** (Xen PV). ## 8. Map, unmap, protect — and the rights they check Every operation takes handles (RFC-0003 §9); RFC-0007 (proposed) encodes them as methods. A second handle — `map`'s Region and Pool — travels as a **handle word in a message register**, resolved in the caller's -table, never transferred: RFC-0003 §6's move would otherwise take the Region from its mapper. +table, never transferred: RFC-0003 §6's move would take the Region from its mapper. | Operation | Needs | Refused when | |-----------|-------|--------------| -| `region_create(pool, pages, max_extents, align, rights)` | pool `WRITE`; pool `EXECUTE` if `rights` has `EXECUTE` | `PoolExhausted`; `Fragmented`; zero pages; `align` not a power-of-two page count; `rights` ⊄ Region rights | +| `region_create(pool, pages, max_extents, align, rights)` | pool `WRITE`; pool `EXECUTE` if `rights` has it | `PoolExhausted`; `Fragmented`; zero pages; `align` not a power-of-two page count; `rights` not Region rights | | `as_create(pool, rights)` | pool `WRITE` | `PoolExhausted` | -| `map(as, region, pool, first_page, vaddr, pages, perms)` | as `WRITE`; region rights ⊇ `perms`; `pool` is the AddressSpace's own, with `WRITE` | `vaddr` unaligned, below the floor or outside the user half; overlap; beyond the Region; `pages > 512`; `perms ∉ {R, RW, RX}`; W^X (§9); `PoolExhausted` | +| `map(as, region, pool, first_page, vaddr, pages, perms)` | as `WRITE`; region rights ⊇ `perms`; `pool` the AddressSpace's own, `WRITE` | `vaddr` unaligned, below the floor or outside the user half; overlap; beyond the Region; `pages > 512`; `perms ∉ {R, RW, RX}`; W^X (§9); `PoolExhausted` | | `unmap(as, vaddr)` | as `WRITE` | no mapping based at `vaddr` — the unit is the whole mapping | | `protect(as, vaddr, perms)` | as `WRITE` | `perms` not a subset of the mapping's current `perms` | -User numbers are validated *as numbers*. **The kernel never chooses a virtual address** *(seL4's VSpace -discipline)*. **Why `map` names a Pool:** spending is authority (O-4), so a holder of an AddressSpace -capability alone cannot drain its Pool; requiring *the AddressSpace's own* Pool keeps one payer per space, -so refunds need no per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool -(a payer record per table frame); a Pool bound at `as_create` and spent by `WRITE` (spending never granted). -**`map` is atomic**: it counts the table frames the range lacks and charges them, with the Mapping slot, -*before* writing anything; then it zeroes and links tables, writes leaves and ends with `DSB ISHST`, so the -walker sees them before `eret`. One call is bounded at 512 leaves and six new tables. Nothing splits a -mapping. `protect` only narrows; widening is `map` again with the Region capability *(seL4's remap)*. 4 KiB -pages only; blocks later, invisible in the ABI. +User numbers are validated *as numbers*; **the kernel never chooses a virtual address** *(seL4's VSpace +discipline)*. **`map` names a Pool because spending is authority** (O-4): an AddressSpace capability alone +cannot drain one, and requiring the space's *own* Pool keeps one payer per space, so refunds need no +per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool (a payer record +per table frame); a Pool bound at `as_create` and spent through `WRITE` (spending never granted). **`map` is +atomic**: it charges the missing tables and the Mapping slot before writing anything, then zeroes and links +tables, writes leaves and ends with `DSB ISHST`, so the walker sees them before `eret`. One call: at most 512 +leaves and six new tables. `protect` only narrows; widening is `map` again *(seL4's remap)*. 4 KiB pages +only; blocks later, invisible in the ABI. **Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as RFC-0003 §5 allows "when a subsystem -needs one". Folding execution into `READ` makes every readable Region potential code; a Region *kind* fails -the loader, which writes code before running it. seL4 leaves execution an ungoverned attribute; Fuchsia's -`ZX_RIGHT_EXECUTE` is the lineage. It means *may map executable* on a Region and *may mint executable -Regions* on a Pool, and is **refused at mint on every other type**: the store's mint path rejects it -(host-tested, increment 7), and `derive` cannot add it (O-2). `rights.rs`'s `ALL` is an explicit union -despite its comment; increment 4 edits both. **On acceptance, RFC-0003 gains a dated amendment adding -`EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 §14.3's handshake is one rule: **the -installed permission is the requested `perms`, refused unless a subset of the Region capability's rights.** +needs one". Folding it into `READ` makes every readable Region potential code; a Region *kind* fails the +loader, which writes code before running it. seL4 leaves execution ungoverned; Fuchsia's `ZX_RIGHT_EXECUTE` +is the lineage. It means *may map executable* on a Region and *may mint executable Regions* on a Pool, and +is **refused at mint on every other type** by the store (host-tested, increment 7); `derive` cannot add it. +`rights.rs`'s `ALL` is an explicit union despite its comment; increment 4 edits both. **On acceptance, +RFC-0003 gains a dated amendment adding `EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 +§14.3's handshake: **the installed permission is `perms`, refused unless a subset of the Region's rights.** | perms | needs on the Region | AArch64 stage 1 | x86_64 | |-------|---------------------|-----------------|--------| @@ -262,131 +246,122 @@ installed permission is the requested `perms`, refused unless a subset of the Re | RW | `READ`, `WRITE` | `AP[2:1] = 0b01`, `UXN = 1`, `PXN = 1` | `P`, `U/S`, `R/W = 1`, `XD = 1` | | RX | `READ`, `EXECUTE` | `AP[2:1] = 0b11`, `UXN = 0`, `PXN = 1` | `P`, `U/S`, `R/W = 0`, `XD = 0` | -Every user entry also sets `AF = 1`, `nG = 1`, inner-shareable, `AttrIndx` → Normal write-back. Write-only -is refused; execute-only exists on AArch64 but not on x86_64 without protection keys, so neither offers it. +Every user entry also sets `AF = 1`, `nG = 1`, inner-shareable, Normal write-back. Write-only is refused; +execute-only exists on AArch64 but not on x86_64 without protection keys, so neither offers it. ## 9. W^X (O-13), in four layers -1. **Unrepresentable.** `MapPerms` is an enum of `Read`, `ReadWrite`, `ReadExecute`; both encoders are total - functions from it, so no writable-executable entry can be built. Host-tested exhaustively. -2. **Per Region, across every address space.** A Region counts its writable uses (`RW` mappings and - IPC-buffer bindings) and its `RX` mappings. `map RX` is refused while any writable use exists; `map RW` - and IPC-buffer binding are refused while any `RX` mapping exists. `unmap`, `protect` and unbinding - decrement **only when their invalidation has completed on every core** (§12) — before that, a stale - writable entry may still be live somewhere. Per-page W^X alone leaves frames writable here and - executable there. -3. **Over time.** `EXECUTE` is minted only from a Pool carrying `EXECUTE`: the broker hands apps Pools - without it, the loader holds one, a JIT gets one by explicit grant *(Fuchsia's `VmexResource`)*. O-13's - Phase-3 load-path clause is policy on this. +1. **Unrepresentable.** `MapPerms` is `Read`, `ReadWrite` or `ReadExecute`; both encoders are total functions + from it, so no writable-executable entry can be built. Host-tested exhaustively. +2. **Per Region, across every address space.** A Region counts writable uses (`RW` mappings, IPC-buffer + bindings) and `RX` mappings: `map RX` is refused while any writable use exists, `map RW` and binding while + any `RX` mapping does. `unmap`, `protect` and unbinding decrement **only once their invalidation has + completed on every core** (§12) — until then a stale writable entry may be live somewhere. +3. **Over time.** `EXECUTE` is minted only from a Pool carrying it: the broker hands apps Pools without it, + the loader holds one, a JIT gets one by explicit grant *(Fuchsia's `VmexResource`)*. O-13's Phase-3 + load-path clause is policy on this. 4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); user - entries set `PXN`, and the image's text has no writable alias, since the physmap maps it read-only - *(Linux `mark_linear_text_alias_ro`)*. x86_64 has no `WXN`; it sets `CR0.WP` and `SMEP` where present. + entries set `PXN`; the image's text has no writable alias, the physmap mapping it read-only *(Linux + `mark_linear_text_alias_ro`)*. x86_64 has no `WXN`: `CR0.WP` and `SMEP` where present. `map RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` on a -PIPT instruction cache and `IC IALLUIS` otherwise (Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`. QEMU's -cortex-a72 reports PIPT (`CTR_EL0 = 0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, -so errors here pass every test.** These, and §12's `TLBI`s and `TTBR` writes, are single-instruction `asm!` -intrinsics inside `kernel/src/arch/**` with no Rust equivalent, as `wfe` is today — a reading of the -constitution's "minimal assembly" the maintainer is asked to confirm (§18.7). The physmap still aliases -*user* code writable at EL1; writing through it only to zero, fill tables and reach IPC buffers is a -convention (§17). +PIPT instruction cache and `IC IALLUIS` otherwise (Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`; QEMU's +cortex-a72 reports PIPT (`CTR_EL0 = 0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so +errors here pass every test.** These and §12's `TLBI`s and `TTBR` writes are single-instruction `asm!` +intrinsics in `kernel/src/arch/**` with no Rust equivalent, as `wfe` is today — a reading of "minimal +assembly" the maintainer is asked to confirm (§18.7). The physmap still aliases *user* code writable at EL1; +writing through it only to zero, fill tables and reach IPC buffers is a convention (§17). ## 10. Sharing and destruction (RFC-0004 §9.1) - **Share** is `derive` then transfer over IPC: the borrower receives, say, `READ | WRITE` without - `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data then travels as RFC-0004 §5's + `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data travels as RFC-0004 §5's descriptor. No `share` syscall and no page-table work during IPC — the long-IPC grave stays shut. -- **Destruction** — at the last live reference, or by RFC-0003 §7.1's destruction case (the right is open, - §18.3) — marks the Region *dying* so no new mapping, binding or copy resolves it, tears down every mapping - and IPC-buffer binding in every address space (§7 says why the walk is unavoidable), completes the - invalidation (§12), advances the generation and refunds the payer. Its preemption points fall between - whole mappings. -- **Donation is not proposed.** pKVM's *donate* would be `reissue` — bump and re-mint for the caller over the - same frames, moving the charge to the recipient's Pool — which is RFC-0003 §7's option B2 on one object. - Gating it on `REVOKE` would widen that right ("capabilities derived from this one") to reach peers, and - under the amendment's option **(a)** a peer-transferred Region carrying `REVOKE` could cut the broker off. - So it is **input to RFC-0003a's (a)/(b) choice**, not a verdict here; the demo needs none of it. - -**Concurrency.** Under RFC-0006 (proposed) V1 — non-preemptible, one big lock when SMP lands — no kernel +- **Destruction** — at the last live reference, or through RFC-0003 §7.1's destruction case (its right is + open, §18.3) — marks the Region *dying*, so no new mapping, binding or copy resolves it; tears down every + mapping and binding in every address space (§7 says why); completes the invalidation (§12); advances the + generation and refunds the payer. Its preemption points fall between whole mappings. +- **Donation is not proposed.** pKVM's *donate* would be `reissue` — bump, re-mint for the caller over the same + frames, move the charge to the recipient's Pool — which is RFC-0003 §7's option B2 on one object. Gating + it on `REVOKE` would widen that right beyond "capabilities derived from this one", and under the + amendment's option **(a)** a peer-transferred Region carrying `REVOKE` could cut the broker off. It is + **input to RFC-0003a's (a)/(b) choice**; the demo needs none of it. + +**Concurrency.** Under RFC-0006 (proposed) V1 — non-preemptible, one big lock once SMP lands — no kernel access to a Region's frames (zeroing, table fill, IPC copy) interleaves with its destruction: each runs -whole within one kernel entry and re-resolves the Region there, and the dying mark covers the gaps between +whole in one kernel entry and re-resolves the Region, and the dying mark covers the gaps between destruction's steps. If locking ever becomes finer, the order is **Region before AddressSpace**. A former -sharer takes an ordinary fault: RFC-0006 (proposed) binds its delivery and RFC-0007 (proposed) defines the -message; under RFC-0007's interim the thread stops. +sharer takes an ordinary fault: RFC-0006 (proposed) binds delivery, RFC-0007 (proposed) defines the message, +and under RFC-0007's interim the thread stops. ## 11. How the kernel touches user memory (O-5) **The kernel never dereferences a user virtual address** — not through a checked copy or a fixup table, not at all *(seL4: the IPC buffer is a frame reached through the kernel window)*. **Where a syscall carries one — a `map` destination, an IPC-buffer location — it is a number, validated as a number and used only to -program tables or registers.** Everything the kernel reads or writes for a process is named by capability -and reached through the physmap: +program tables or registers.** RFC-0007 inherits that sentence exactly. Everything the kernel reads or +writes for a process is named by capability and reached through the physmap: -- **The IPC buffer** is a 1 KiB-aligned slice of one page of a Region (RFC-0007 (proposed) §5 sets its size - and syscall; RFC-0006 (proposed) records it on the Thread). Binding needs `READ | WRITE`, holds a counted +- **The IPC buffer** is a 1 KiB-aligned slice of one page of a Region (RFC-0007 (proposed) §5 sets size and + syscall; RFC-0006 (proposed) records it on the Thread). Binding needs `READ | WRITE`, holds a counted reference, is a writable use for §9, and destruction severs it. - **No kernel path faults on user memory**: frames are eager and pinned, so the slow path copies frame to frame, and the in-kernel fault concurrency that sank L4's long IPC cannot arise. -- **Peer-writable memory is never referenced.** No Rust reference is formed into it: one `kernel/src/mm/` - module copies with volatile accesses, and anything the kernel must interpret is copied once and validated - as that snapshot *(virtio's double-fetch lesson, research/0002 Part 5)*. +- **Peer-writable memory is never referenced**: one `kernel/src/mm/` module copies with volatile accesses, + and anything the kernel interprets is copied once and validated as that snapshot *(virtio's double-fetch + lesson, research/0002 Part 5)*. -RFC-0007 inherits that sentence exactly. RFC-0003 §9 forbade naming *resources* by address; this extends it -to data. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. `SMAP` and +*Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. `SMAP` and `PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. ## 12. TLBs, ASIDs and PCIDs -- **Tags are kernel-private**: an ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8) or a - PCID (12 bits, where CPUID reports it); tag 0 reserved. **Destroying an AddressSpace returns its tag** - after `TLBI ASIDE1IS` (or `INVPCID` single-context), so churn does not force rollover. *Rejected:* seL4's +- **Tags are kernel-private**: an ASID (16 bits when `ID_AA64MMFR0_EL1.ASIDBits = 0b0010`, else 8) or a PCID + (12 bits, where CPUID reports it); tag 0 reserved. **Destroying an AddressSpace returns its tag** after + `TLBI ASIDE1IS` (x86_64: `INVPCID` single-context), so churn does not force rollover. *Rejected:* seL4's ASID pools — a tag is not authority. - **Kernel entries are global, user entries never.** `map` needs no invalidation: neither architecture caches - a faulting entry (the Arm ARM's rule; its section is uncited because the manual was unreachable; SDM - Vol. 3A §4.10), and a fault whose entry is valid on re-walk is retried — x86_64's paging-structure caches - can cause one. `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS` (ASID and address, - leaf only), `DSB ISH`, `ISB`; x86_64 uses `INVLPG`, and `INVPCID` or a stale mark for other tags. Freeing an - intermediate table uses `TLBI VAE1IS` or `ASIDE1IS`, since `VALE1IS` spares walk caches; x86_64's `INVLPG` - and `INVPCID` already flush the PCID's paging-structure caches (SDM Vol. 3A §4.10.4.1). + a faulting entry (the Arm ARM's rule, section uncited as the manual was unreachable; SDM Vol. 3A §4.10), + and a fault whose entry is valid on re-walk is retried, as x86_64's paging-structure caches can cause. + `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS`, `DSB ISH`, `ISB`; x86_64 uses + `INVLPG`. Freeing an intermediate table uses `TLBI VAE1IS` or `ASIDE1IS`, since `VALE1IS` spares walk + caches; `INVLPG` and `INVPCID` already flush paging-structure caches (SDM Vol. 3A §4.10.4.1). - **O-8's ordering: invalidate, then free.** No frame returns to a Pool while any core can translate to it. -**Multi-core — this RFC owns the protocol; RFC-0006 (proposed) provides the IPI.** Each AddressSpace keeps +**Multi-core: this RFC owns the protocol; RFC-0006 (proposed) provides the IPI.** Each AddressSpace keeps the set of cores that have run it since its last full flush *(Linux `mm_cpumask`)*. - **Shootdown.** AArch64 needs no IPI: the `…IS` forms reach the inner-shareable domain and `DSB ISH` waits - for completion on every core. x86_64 has no broadcast: the initiator sends RFC-0006's IPI to the cores in - the set, each issues `INVLPG`/`INVPCID` and acknowledges, and the initiator waits for every - acknowledgement before any frame is freed or any W^X count falls. A core spinning on the big lock polls - its per-core flush request while it spins, so the lock holder never waits on a core waiting for it. -- **Rollover.** On exhaustion the generation bumps; every core's *active* tag is reserved into the new - generation; every core gets a pending-flush mark and flushes locally (`TLBI VMALLE1`) at its next - address-space switch. **A tag is reallocated only after every core that ran under it has flushed** - *(Linux `arch/arm64/mm/context.c`: `active_asids`, `reserved_asids`, `tlb_flush_pending`)*. -- **x86_64 PCIDs are per core** (SDM Vol. 3A §4.10.1): each core caches a few AddressSpace→PCID pairs - *(Linux keeps six)*, and an invalidation on a core where the space is not current is a per-core stale mark - consumed at the next switch to it. - -The demo is `-smp 1`, but every `TLBI` is the broadcast form from the first line; the rest is built with -RFC-0006 increment 14. + for completion everywhere. x86_64 has no broadcast: the initiator IPIs the cores in the set, each + invalidates and acknowledges, and **no frame is freed nor W^X count lowered before every + acknowledgement**. A core spinning on the big lock polls its flush request, so the holder never waits on + a core waiting for it. +- **Rollover.** On exhaustion the generation bumps, every core's *active* tag is reserved into the new one, + and every core gets a pending-flush mark, consumed by a local `TLBI VMALLE1` at its next switch. **A tag is + reallocated only after every core that ran under it has flushed** *(Linux `arch/arm64/mm/context.c`: + `active_asids`, `reserved_asids`, `tlb_flush_pending`)*. +- **x86_64 PCIDs are per core**: each core caches a few AddressSpace→PCID pairs *(Linux keeps six)*; an + invalidation where the space is not current is a per-core stale mark consumed at the next switch to it. + +The demo is `-smp 1`, but every `TLBI` is the broadcast form; the rest lands with RFC-0006's SMP increment. ## 13. Early boot: turning the MMU on The image is linked at `0xFFFF_FFFF_8020_0000` and loaded at 0x4020_0000 (`AT()` in `aarch64.ld`). QEMU loads -segments at `p_paddr` and translates the entry point to its physical alias **only if** it lies in an -executable segment whose `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, `load_elf`, at v8.2.2) — which -`AT()` guarantees while `.text.boot` stays first in the first executable segment. With the MMU off, the stub: +segments at `p_paddr` and translates the entry to its physical alias **only if** it lies in an executable +segment with `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, `load_elf`, v8.2.2) — which `AT()` guarantees +while `.text.boot` leads the first executable segment. With the MMU off, the stub: 1. Parks secondaries, installs the stack and zeroes `.bss` at physical addresses, as today. 2. `MAIR_EL1`: index 0 = `0xFF` (Normal write-back, read/write-allocate), index 1 = `0x04` (Device-nGnRE). 3. `TCR_EL1`: `T0SZ = T1SZ = 17`; `TG0 = 0b00`, `TG1 = 0b10` (both 4 KiB); `SH = 0b11`; `IRGN/ORGN = 0b01`; `IPS` from `PARange`; `AS` from `ASIDBits`; `A1 = 0`; `TBI0 = TBI1 = 0`; `HA = HD = 0`. -4. Four page-aligned boot tables in `.bss`, `DC IVAC` before and after they are written, so no dirty - firmware line is evicted over them and no stale line shadows them *(Linux `head.S`)*: `TTBR0_EL1` - identity-maps 0x4000_0000 as a 1 GiB Normal block and 0x0000_0000 as a 1 GiB Device block, `UXN` and - `PXN` set, so the UART survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to 0x4000_0000 as a 1 GiB - Normal block. All `AF = 1`. The Normal blocks span 512 MiB with no RAM behind it — a QEMU-only interim. +4. Four page-aligned boot tables in `.bss`, with `DC IVAC` before and after writing them, so no dirty firmware + line is evicted over them and none shadows them *(Linux `head.S`)*: `TTBR0_EL1` identity-maps 0x4000_0000 + as a 1 GiB Normal block and 0x0000_0000 as a 1 GiB Device block with `UXN` and `PXN`, so the UART + survives; `TTBR1_EL1` maps `0xFFFF_FFFF_8000_0000` to 0x4000_0000 as a 1 GiB Normal block; all `AF = 1`. 5. `TLBI VMALLE1`; `IC IALLU`; `DSB NSH`; `ISB`; both `TTBR`s; `ISB`; `SCTLR_EL1` = **`0x30D4581D`**; `ISB`. -6. Branches to the high alias by absolute address, moves `SP` and `VBAR_EL1` there, and calls `rust_entry`. +6. Branches to the high alias by absolute address, moves `SP` and `VBAR_EL1` there, calls `rust_entry`. **Rust never executes at the low alias.** | `SCTLR_EL1` field (bit) | Value | Why | @@ -395,59 +370,47 @@ executable segment whose `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, `load_elf | `A` (1) | 0 | alignment is the compiler's; both targets are `+strict-align` | | `SA` (3), `SA0` (4) | 1 | stack-alignment checks at EL1 and EL0 | | `CP15BEN` (5), `ITD` (7), `SED` (8) | 0 | AArch32 controls; no thread runs AArch32 (RFC-0007 builds every `SPSR`) | -| `UMA` (9) | 0 | EL0 cannot mask interrupts, so it cannot defeat preemption | +| `UMA` (9) | 0 | EL0 cannot mask interrupts, so cannot defeat preemption | | `DZE` (14) | 1 | EL0 `DC ZVA` is a store its mapping already permits | | `UCT` (15), `UCI` (26) | 0 | EL0 does no cache maintenance; the kernel does it at `map RX` (§9) | | `nTWI` (16), `nTWE` (18) | 0, 1 | `wfi` traps and `wfe` does not — RFC-0006 (proposed) §8 | -| `WXN` (19) | 0, then 1 | set by Rust step (5): `0x30DC581D` | +| `WXN` (19) | 0, then 1 | set by Rust step (5), giving `0x30DC581D` | | `E0E` (24), `EE` (25) | 0 | little-endian | | 11, 20, 22, 23, 28, 29 | 1 | RES1 on Armv8.0: Linux v4.19's `SCTLR_EL1_RES1`, plus bit 23 (`SPAN`, RES1 before Armv8.1), which Linux always sets; not re-checked against the Arm ARM | -Every other bit is 0: RES0 on Armv8.0, or a later extension's control left off. Rust then, in an order that -never leaves the reporter without a console: (1) reads the DTB and builds the `MemoryMap`, bitmap and slot -arrays through the **boot window** — the frame-access trait takes a window offset, here `pa − 0x4000_0000 + -0xFFFF_FFFF_8000_0000`, so all RAM touched this early lies in the first boot GiB; (2) builds §6's final -tables, stack window included; (3) swaps `TTBR1_EL1` by an assembly routine run from the identity map — empty -table, `TLBI VMALLE1`, final table — since turning blocks into pages under a live translation needs -break-before-make *(Linux `idmap_cpu_replace_ttbr1`)*, then re-references the bitmap and switches the trait -to the physmap offset; (4) **repoints every kernel MMIO user at the MMIO window** — the UART, and the GIC -if RFC-0006 has brought it up; (5) sets `TCR_EL1.EPD0 = 1` (cleared by the first AddressSpace install), sets -`WXN`, and issues `TLBI VMALLE1`, `DSB ISH`, `ISB`. **The boot tables are never freed**: they are 16 KiB of -`.bss`, and freeing them would give image frames a second owner. - -The reporter gains a DFSC and fault-level decode (it prints `FAR` already); on x86_64 `#PF` prints `CR2` and -its error code. **x86_64**, once the UEFI stub exists, is already paging; Rust builds §6's PML4 with a -temporary identity entry for a trampoline page, sets `IA32_EFER.NXE` (bit 11), `CR0.WP` (bit 16), `CR4.PGE` -(bit 7) and `CR4.PCIDE`/`SMEP`/`SMAP` (bits 17, 20, 21) where CPUID permits; assembly loads `CR3`, jumps to -the high alias and drops the identity entry — the same "Rust runs only high" rule. *Rejected:* a separate -loader *(seL4's elfloader)*; Rust building tables with the MMU off, sound only without absolute addresses. - -## 14. Lineage - -| Source | What is taken | What is left | -|--------|---------------|--------------| -| **seL4** | frames as capability objects; the kernel window; never touching user addresses; all memory to the first task; caller-paid tables | Untyped/retype and the CDT; execute as an ungoverned attribute; ASID pools | -| **pKVM** *(research/0002 Part 7)* | one owner per frame; *share* | *donate*, until RFC-0003a; stage-2 mechanics | -| **Zircon / Fuchsia** | a region mapped into many spaces, kept alive by mappings; `EXECUTE` as a right; `VmexResource` | the kernel heap; lazy VMOs; VMARs | -| **Hubris; Fiasco.OC, Genode** | build-time-sized tables; kernel memory as a counted, tradeable quota | a static task set; quotas on a heap | -| **Linux** | the linear map, its text alias read-only; ASID rollover and `mm_cpumask`; per-core PCIDs; arm64 headers as a checked reference | the buddy allocator; `copy_from_user` and its fixups | -| **Ge et al., EuroSys '19** | the colour-capable allocator | time protection itself, for later | - -## 15. Obligations +Every other bit is 0: RES0 on Armv8.0, or a later extension left off. Rust then, never leaving the reporter +without a console: (1) builds the `MemoryMap`, bitmap and slot arrays through the **boot window** — the +frame-access trait takes an offset, here `pa − 0x4000_0000 + 0xFFFF_FFFF_8000_0000`, so everything touched +this early lies in the first boot GiB; (2) builds §6's final tables, stack window included; (3) swaps +`TTBR1_EL1` by an assembly routine run from the identity map — empty table, `TLBI VMALLE1`, final table — +as break-before-make requires *(Linux `idmap_cpu_replace_ttbr1`)*, then re-references the bitmap and +switches the trait to the physmap offset; (4) **repoints every kernel MMIO user** — the UART, and the GIC if +RFC-0006 has brought it up; (5) sets `TCR_EL1.EPD0 = 1` (cleared by the first AddressSpace install) and +`WXN`, then `TLBI VMALLE1`, `DSB ISH`, `ISB`. **The boot tables are never freed**: 16 KiB of `.bss`, and +freeing them would give image frames a second owner. + +The reporter gains a DFSC and fault-level decode (it prints `FAR` already); x86_64's `#PF` prints `CR2` and +its error code. **x86_64**, once the UEFI stub exists, is already paging: Rust builds §6's PML4 with a +temporary identity entry for a trampoline page and sets `IA32_EFER.NXE` (bit 11), `CR0.WP` (bit 16), +`CR4.PGE` (bit 7) and `CR4.PCIDE`/`SMEP`/`SMAP` (bits 17, 20, 21) where CPUID permits; assembly loads `CR3`, +jumps high and drops the identity entry — the same "Rust runs only high" rule. *Rejected:* a separate loader +*(seL4's elfloader)*; Rust building tables with the MMU off, sound only without absolute addresses. + +## 14. Obligations | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| -| O-3 revocability | destruction walks the Region's mapping list, invalidates, then bumps; a page-table entry is a cached resolution (§7, §10) | **destruction case extended to memory**; selective revocation and donation are RFC-0003a's | -| O-5 argument validation | no user virtual address dereferenced; carried ones validated as numbers; memory by capability through the physmap; snapshot-validate peer-writable data (§11) | **discharged for the memory surface**; RFC-0007 inherits §11's sentence | -| O-6 kernel memory safety | new `unsafe` confined to `mm/**` (window access, volatile copy, the store's one static) and `arch/**` (tables, TLB, boot) | **stays Built** — every new block listed per PR | -| O-7 no unprivileged exhaustion | typed arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map` included; charge-first; `map` ≤ 512 leaves; zeroing in 512-page steps; run search bounded by RAM (§4, §5, §8) | **discharged for kernel allocation**; a transferred object stays charged to its payer until destroyed — **open** with RFC-0003a (§18.2); CPU time is RFC-0006's | -| O-8 address-space disjointness | per-process tables; Regions only by capability; zero before exposure, table frames too; invalidate before free (§4, §7, §12) | **discharged** | +| O-3 revocability | destruction walks the Region's mappings, invalidates, then bumps; a page-table entry is a cached resolution (§7, §10) | **destruction case extended to memory**; selective revocation and donation are RFC-0003a's | +| O-5 argument validation | no user virtual address dereferenced; carried ones are numbers; memory by capability through the physmap; snapshots (§11) | **discharged for the memory surface**; RFC-0007 inherits §11's sentence | +| O-6 kernel memory safety | new `unsafe` only in `mm/**` (window access, volatile copy, the store's static) and `arch/**` (tables, TLB, boot) | **stays Built** — every new block listed per PR | +| O-7 no unprivileged exhaustion | arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map`'s too; charge-first; `map` ≤ 512 leaves; zeroing in 512-page steps; run search bounded by RAM (§4, §5, §8) | **discharged for kernel allocation**; a transferred object stays charged to its payer until destroyed — **open** with RFC-0003a (§18.2); CPU time is RFC-0006's | +| O-8 address-space disjointness | per-process tables; Regions only by capability; frames and tables zeroed before use; invalidate before free (§4, §7, §12) | **discharged** | | O-9 no ambient side channel | no shared mapping a capability did not establish (§10) | **contributes** | -| O-13 W^X | four layers, per Region across all address spaces, counts falling only on completed invalidation (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | -| O-2, O-4 | `perms` ⊆ capability rights; `protect` narrows; spending needs a Pool capability; no physical-address map | **maintained** | +| O-13 W^X | four layers, per Region across all spaces, counts falling only on completed invalidation (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | +| O-2, O-4 | `perms` ⊆ rights; `protect` narrows; spending needs a Pool capability; no physical-address map | **maintained** | | O-18 confined DMA | single-extent Regions; Device Regions minted at boot | **groundwork only** — Phase 2 | -## 16. Graves checked (§3) +## 15. Graves checked (§3) - **Policy in the kernel.** No address choice, demand paging, copy-on-write, overcommit or OOM response. - **Baroque capability hierarchies.** No Untyped tree; Pools are flat, with one funder index for closure. @@ -457,31 +420,38 @@ loader *(seL4's elfloader)*; Rust building tables with the MMU off, sound only w - **The catch-all right.** `EXECUTE` is narrow and refused off-type; no "map any physical address" right. - **Compiled-in unused device paths (VENOM).** The kernel maps only the devices it drives itself. +## 16. Lineage in one place + +seL4 (frames as capability objects, the kernel window, caller-paid tables, all memory to the first task); +pKVM (one owner per frame, *share*); Zircon/Fuchsia (mappings keep a region alive, `EXECUTE`, `VmexResource`); +Hubris, Fiasco.OC and Genode (sized tables, counted quotas); Linux (the linear map and its read-only text +alias, ASID rollover, `mm_cpumask`, per-core PCIDs); Ge et al. (colouring). Left: Untyped and the CDT, lazy +VMOs, the kernel heap, the buddy allocator, `copy_from_user`. + ## 17. Costs — what this makes harder -- **Capacities are ceilings**: raising a slot count is a rebuild, and `pool_move` is one call per resource. +- **Capacities are ceilings**: a slot count is a rebuild; `pool_move` is one call per resource. - **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `Fragmented`. - **`map` needs the AddressSpace's own Pool**: a loader mapping into a child holds the child's Pool. - **47-bit halves on AArch64** give up half of what `T0SZ = 16` allows; **the 64 KiB floor is fixed**. -- **User addresses are numbers only** (§11): RFC-0007 designs every syscall around registers, the IPC - buffer and Region handles, for a bug class gone. +- **User addresses are numbers only** (§11): every RFC-0007 syscall works in registers, the IPC buffer and + Region handles, for a bug class gone. - **The physmap is a large target**: one kernel write bug reaches all RAM, user code included (§9) — a software invariant, as are W^X on x86_64 and O-5 without PAN on cortex-a72. -- **At every boot the kernel runs RWX** from MMU-on to Rust step (5), before any process exists; a stray - early write goes uncaught. -- **Cache maintenance is untestable under QEMU** (§9, §13); **the load address is a QEMU `virt` fact**. -- **Reclaim across a transfer needs cooperation**: a transferred object keeps its payer charged until it is +- **Every boot runs RWX** from MMU-on to Rust step (5), before any process exists; a stray early write goes + uncaught. **Cache maintenance is untestable under QEMU**; **the load address is a QEMU `virt` fact**. +- **Reclaim across a transfer needs cooperation**: a transferred object keeps its payer charged until destroyed, so `pool_merge` waits on other processes — RFC-0003a's problem made concrete. ## 18. Open questions 1. **Page-table frames on unmap.** Freed only with the AddressSpace; eager freeing needs §12's walk-cache rule. 2. **Donation and reclaim across a transfer** — `reissue` with a charge move, offered to RFC-0003a (§10). -3. **Which right authorises explicit destruction** — `REVOKE` would widen it as `reissue` would; RFC-0003a. +3. **Which right authorises explicit destruction** — `REVOKE` would widen as `reissue` would; RFC-0003a. 4. **Hardware backing for two software rules:** software PAN on Armv8.0 (a `TTBR0_EL1` change per kernel entry), and unmapping executable user frames from the physmap. -5. **Preemptible teardown** at RFC-0006 (proposed)'s preemption points: the O(mappings) walk made restartable. -6. **The firmware assumption** (§4) for threat model §10, and O-7's new text (§5), via `docs/CHANGELOG.md`. +5. **Preemptible teardown**: the O(mappings) walk made restartable at RFC-0006 (proposed)'s preemption points. +6. **The firmware assumption** (§4) and O-7's new text (§5), via `docs/CHANGELOG.md`. 7. **"Minimal assembly"** (Constitution §11.1): are runtime single-instruction `TLBI`/`DC`/`IC`/`TTBR` intrinsics in `arch/**` within it (§9), or does the clause need amending? 8. **One programme order** across RFC-0005/0006/0007 — §19 proposes one for the maintainer to fix. @@ -490,62 +460,60 @@ loader *(seL4's elfloader)*; Rust building tables with the MMU off, sound only w One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate — `no_std` in the kernel, `std` under `cargo test`, built for both targets in CI, no `unsafe`, as `capability/`. -Each new `arch` function (memory map, window access, TLB, address-space switch) gets a documented x86_64 -stub: the memory-map source returns an empty `MemoryMap`, and memory initialisation prints `mm: no memory -map (UEFI stub pending)` and halts, so the second Tier-1 build compiles the whole path from increment 2. +Each new `arch` function (memory map, window, TLB, address-space switch) gets a documented x86_64 stub: an +empty `MemoryMap`, on which memory initialisation prints `mm: no memory map (UEFI stub pending)` and halts, +so the second Tier-1 build compiles the whole path from increment 2. 1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU off. `--expect "DTB 0xd00dfeed at 0x40000000"`. -2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs, and - on a recorded OVMF `GetMemoryMap` dump once the stub exists; touches the workspace `members`, - `.github/workflows` and `CLAUDE.md` § Layout. `--expect "mm: 512 MiB RAM at 0x40000000"`. -3. **[demo] Frame bitmap.** Host churn tests against a shadow model: no frame twice, none reserved, no image - frame ever free. -4. **[demo] `EXECUTE` in `capability/`.** Bit 5, `ALL` and its comment, `Debug`, the test widened to `0..64`; - with RFC-0003's dated amendment. Host-only; demo-needed because the root maps the client's text. +2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs (an + OVMF `GetMemoryMap` dump once the stub exists); workspace `members`, `.github/workflows` and `CLAUDE.md` + § Layout gain `memory/`. `--expect "mm: 512 MiB RAM at 0x40000000"`. +3. **[demo] Frame bitmap.** Churn tests against a shadow model: no frame twice, none reserved, no image frame free. +4. **[demo] `EXECUTE` in `capability/`** with RFC-0003's dated amendment: bit 5, `ALL` and its comment, `Debug`, + the test widened to `0..64`. Demo-needed because the root maps the client's text. 5. **[demo] `MapPerms` and both encoders.** Exhaustive: nothing writable and executable; every user entry `PXN`, `AF`, not-global; every row of §8's table. -6. **[demo] The table engine** over the frame-access trait. Host tests on both formats: overlap, the 512-leaf - and six-table bound, nothing mapped when unpaid, tables zeroed before linking, a kernel-half map never +6. **[demo] The table engine** over the frame-access trait, on both formats: overlap, the 512-leaf and + six-table bound, nothing mapped when unpaid, tables zeroed before linking, a kernel-half map never writing the root. 7. **[demo] Slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale references count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. Host-only. -8. **`pool_split`/`move`/`merge` and closure forwarding.** Host tests under random churn. Not demo. -9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode. - Prints the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. -10. **[demo] Final kernel map (§13, Rust)**: physmap with the image read-only, MMIO window with every kernel - MMIO user repointed, stack window with core 0's stack moved behind a guard; `EPD0`, `WXN`. Three - self-tests, one `--expect` each, and the store behind its one static: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the - image and the physmap, expecting `data abort, same EL`; `provoke-wxn` branches into `.data`, expecting - `instruction abort, same EL`. -11. **[demo] Region and Mapping records** with the W^X counts; host-only in `memory/`. -12. **[demo] The tag allocator**: rollover, per-core reservation, return on destruction; host-only. -13. **[demo] AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first - install. Boot-test: `AT S1E0R` under two ASIDs resolves one shared Region to one frame and private ones - apart: `mm: shared region agrees`. -14. **[demo] Pool, Region and AddressSpace methods through RFC-0007's `call`**, handle words in message - registers, with RFC-0007 increment 6: an EL0 self-test maps a fresh Region and prints `[el0] map ok`. +8. **`pool_split`/`move`/`merge` and closure forwarding**, under random churn. Host-only; not demo. +9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode; prints + the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. +10. **[demo] Final kernel map (§13, Rust)**, the store behind its static: physmap with the image read-only, + MMIO window with every kernel MMIO user repointed, core 0's stack behind a guard; `EPD0`, `WXN`. One + `--expect` per self-test: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the image and + the physmap (`data abort, same EL`); `provoke-wxn` branches into `.data` (`instruction abort, same EL`). +11. **[demo] Region and Mapping records** with the W^X counts. Host-only. +12. **[demo] The tag allocator**: rollover, per-core reservation, return on destruction. Host-only. +13. **[demo] AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first install. + `AT S1E0R` under two ASIDs resolves a shared Region to one frame, private ones apart: `mm: shared region agrees`. +14. **[demo] Pool, Region and AddressSpace methods through RFC-0007's `call`**, with its increment 6: an EL0 + self-test maps a fresh Region and prints `[el0] map ok`. 15. **`unmap` and `protect`** with invalidation and count decrements (§9, §12). Not demo. -16. **User-memory module (§11).** A message beyond the register budget. Not demo. -17. **Destruction** (§10), after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches its - handler. Not demo. +16. **User-memory module (§11)**: a message beyond the register budget. Not demo. +17. **Destruction (§10)**, after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches + its handler. Not demo. 18. **Stack overflow**, with RFC-0006's overflow stack: `provoke-stack-overflow` reaches the reporter. 19. **PAN, SMAP, SMEP** where reported, with a `provoke-user-deref` self-test under `-cpu max`. 20. **x86_64** tables, `CR3` and PCID, after the UEFI stub; **Device Regions** in Phase 2. -**Programme order proposed** (§18.8): 1–7 beside RFC-0006 1–2 and RFC-0007 1–5; RFC-0007's MMU-off EL0 -self-test (its increment 3) lands before 9, which retires it; 9–10 before RFC-0006 3 — or 10 repoints the GIC -too — and before RFC-0006 4, which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7. +**Programme order proposed** (§18.8): 1–8 beside RFC-0006 1–2 and RFC-0007 1–5; RFC-0007's MMU-off EL0 +self-test (its 3) before 9, which retires it; 9–10 before RFC-0006 3 (or 10 repoints the GIC too) and 4, +which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7; 14 with RFC-0007 6. **Interims the demo carries, with their cost:** -- **One root Pool.** Increment 8 is not a syscall yet: server and client charges are indistinguishable. +- **One root Pool** (increment 8 is not a syscall yet): server and client charges are indistinguishable. - **No destruction or `unmap`**: a killed process's frames and slots stay charged; O-3's memory case is designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. - **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. -- **Until increment 10, the kernel runs RWX throughout**; cache maintenance stays unverified. -- **1 GiB boot blocks over unbacked space** (§13): real hardware needs 2 MiB blocks sized from the DTB. +- **Until increment 10 the kernel runs RWX throughout**; cache maintenance stays unverified. +- **1 GiB boot blocks** cover 512 MiB with no RAM behind them, which speculation may touch on real + hardware: there, 2 MiB blocks sized from the DTB. - **Zeroing in one step** until RFC-0006's preemption points; the demo's Regions are a few pages. -- **An overflow of the guarded stack is a recursive abort** until increment 18. +- **A guarded-stack overflow is a recursive abort** until increment 18. - **Not an interim:** the root's image reaches it through boot-minted and kernel-filled Regions and increment 13's `map`, so §8's rights check runs even at boot. @@ -555,6 +523,6 @@ Increments 1–14 give RFC-0006 (proposed) address spaces, a guarded per-core st threads to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a constraint it inherits rather than discovers: **the kernel never dereferences a user virtual address; where a syscall carries one, it is a number, validated as a number and used only to program tables or registers.** -The root task receives all memory as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient +The root receives all memory as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor. Increments 3–8 are host-only and 1–2 change only where the kernel loads and what it prints: all can land now. From d2029e921346e62bda1d498717889e558de73e8a Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:37:53 +0000 Subject: [PATCH 12/20] wip(docs): snapshot RFC revisions in progress Backup only; the revision workflows are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 268 +++---- docs/rfcs/0006-threads-and-scheduling.md | 705 +++++++++++------- 2 files changed, 539 insertions(+), 434 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index 095fb25..54b11d6 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -7,7 +7,7 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 (kernel doctrine) and §11.1's "minimal assembly" (§18.7); RFC-0003 (a dated amendment adding one right; three new objects; `capability/src/rights.rs` and `object.rs`'s doc comment); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 and O-7's text (proposals via `docs/CHANGELOG.md`) | +| Affects | Constitution §3 (kernel doctrine) and §11.1's "minimal assembly" (§17.7); RFC-0003 (a dated amendment adding one right; three new objects; `capability/src/rights.rs` and `object.rs`'s doc comment); RFC-0004 §9.1; the boot stub, `aarch64.ld` and the exception reporter; `kernel/src/mm/**`; a new host-tested `memory/` crate; threat model §10 and O-7's text (proposals via `docs/CHANGELOG.md`) | | Depends on | RFC-0003 (capability table), RFC-0004 (IPC); interlocks with RFC-0006 (threads and scheduling) and RFC-0007 (system calls and process bootstrap), proposed the same day | | Discharges | O-7 for kernel allocation (reclaim across a transfer stays open), O-8, O-13 (the mapping half); O-5 for every path by which the kernel touches user memory; O-3's destruction case extended to memory; answers RFC-0003 §14.3 and RFC-0004 §9.1 (share) | @@ -49,7 +49,6 @@ and 47-bit user and kernel halves whose top level uses 256 entries each** — AA | Property | AArch64 (VMSAv8-64, 4 KiB granule) | x86_64 (IA-32e, 4-level) | |---|---|---| | User / kernel split | two roots: `TTBR0_EL1` (low), `TTBR1_EL1` (high) | one root, `CR3`; PML4 entries 0–255 user, 256–511 kernel | -| Present | bits 1:0 = `0b11` (valid page) at level 3 | bit 0 (P) | | Writable | `AP[2]` (bit 7) clear | bit 1 (R/W) set | | User-accessible | `AP[1]` (bit 6) set | bit 2 (U/S) set | | No-execute | `PXN` (bit 53) and `UXN` (bit 54), separately | `XD` (bit 63), only with `IA32_EFER.NXE` | @@ -67,54 +66,51 @@ host-tested on both in `memory/`; only turning a frame number into `&mut [u64; 5 ## 4. Physical memory: discovery and the frame allocator **Discovery.** QEMU fixes two facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every other -address "may vary" and must be read from the DTB, found "at the start of RAM" with no register pointing at it -(`docs/system/arm/virt.rst`). `hw/arm/boot.c` places it only *below* the image (`dtb_limit` is the image's -lowest address). Checked on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM**; -at 0x4020_0000 one is found at 0x4000_0000. **So the image loads at 0x4020_0000**, block-aligned, capping -the DTB at 2 MiB; increment 1 re-checks the pinned QEMU 11.0.3. A minimal FDT reader (Borrow Ledger boot-path -row; Devicetree Specification v0.4) reads the cell sizes, `/memory`, `/memreserve/` and `/reserved-memory`, -bounds-checks every offset and fails closed. This RFC proposes threat model §10's first firmware assumption — -*firmware tables describe memory truthfully* — via `docs/CHANGELOG.md`. On x86_64 the UEFI stub (no RFC yet) -hands over `GetMemoryMap` after `ExitBootServices`. **Both produce one `MemoryMap`**: same-kind neighbours +address "may vary" and must be read from the DTB, found "at the start of RAM" with no register pointing at +it (`docs/system/arm/virt.rst`); `hw/arm/boot.c` places it only *below* the image (`dtb_limit`). Checked on +QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM**; at 0x4020_0000 one is found at +0x4000_0000. **So the image loads at 0x4020_0000**, block-aligned, capping the DTB at 2 MiB; increment 1 +re-checks the pinned QEMU 11.0.3. A minimal FDT reader (Borrow Ledger boot-path row; Devicetree +Specification v0.4) reads the cell sizes, `/memory`, `/memreserve/` and `/reserved-memory`, bounds-checks +every offset and fails closed. Threat model §10 gains a proposed first firmware assumption — *firmware +tables describe memory truthfully* — via `docs/CHANGELOG.md`. On x86_64 the UEFI stub (no RFC yet) hands +over `GetMemoryMap` after `ExitBootServices`. **Both produce one `MemoryMap`**: same-kind neighbours coalesced, then at most **256** sorted, disjoint `(PhysRange, Kind)` entries (6 KiB; raw UEFI maps run to -100–200) — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that. Image, -boot stack, DTB and boot tables are tagged first; Phase 1's `Mmio` entries are the interim hard-coded -addresses (§19). Nothing above `arch` names FDT or UEFI. +100–200) — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that; image, +boot stack, DTB and boot tables tagged first. Phase 1's `Mmio` entries are the interim hard-coded +addresses (§18). Nothing above `arch` names FDT or UEFI. **The frame allocator is a bitmap** — one bit per 4 KiB frame, stored in the first free frames (16 KiB per 512 MiB) — allocating from a *colour set*, or runs of `n` at an alignment; a double free fails closed. -Colour is `pfn mod C`, `C = 1` until time protection *(Ge et al., EuroSys '19, via research/0002 Part 6)*. -A run search is O(frames / 64) words, bounded by RAM rather than by the caller; a per-alignment cursor is -the remedy if measurement asks. **Fresh frames are zeroed before a process can observe them, and a table -frame before its parent descriptor is written** (a `DSB` between): a recycled table with stale valid -entries would translate into another process's frames. *Rejected:* a **buddy allocator** *(Linux)*, whose -coalescing fights colouring; a **free list**, which cannot find `n` contiguous frames without a scan. +Colour is `pfn mod C`, `C = 1` until time protection *(Ge et al., EuroSys '19, via research/0002 Part 6)*. A +run search is O(frames / 64) words, bounded by RAM rather than the caller; a per-alignment cursor is the +remedy if measurement asks. **Fresh frames are zeroed before a process can observe them, and a table frame +before its parent descriptor is written** (a `DSB` between): a recycled table's stale entries would +translate into another process's frames. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing +fights colouring; a **free list**, which cannot find `n` contiguous frames without a scan. ## 5. Who pays: the object store and Pools (O-7) The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped and retype**: -reusing an Untyped revokes everything retyped from it through a derivation tree, which would decide -RFC-0003a's B1 by the back door (RFC-0003 §7 and its 2026-07-30 amendment) and make object layouts ABI. -**Rejected.** **B — a kernel heap with creation policy** *(Zircon)* limits *whether* a process creates -objects, not how much memory they take. **Rejected.** **C — static per-type tables** *(Hubris)* cannot say -*who* filled one (research/0002 Part 6's Kinesis outage). **Rejected alone.** **D — C with counted -attribution: verdict sought** *(the requester pays, as in seL4; a counted quota, as in Fiasco.OC's factory -limits and Genode's quota trading)*. +reusing an Untyped revokes what was retyped from it through a derivation tree, deciding RFC-0003a's B1 by +the back door (RFC-0003 §7 and its 2026-07-30 amendment) and making object layouts ABI. **Rejected.** **B — a +kernel heap with creation policy** *(Zircon)* limits *whether* a process creates objects, not how much memory +they take. **Rejected.** **C — static per-type tables** *(Hubris)* cannot say *who* filled one (research/0002 +Part 6's Kinesis outage). **Rejected alone.** **D — C with counted attribution: verdict sought** *(the +requester pays, as in seL4; counted quotas, as in Fiasco.OC's factory limits and Genode's quota trading)*. **The store is typed slot arrays**, one fixed-capacity array per type in `.bss`; a slot holds the object, a generation and a count. The kernel's implementation of `ObjectRef` (`capability/src/object.rs`, a trait) is `(type, index, generation)`; `current_generation()` is a slot read. **Only live references count** — -capabilities at the slot's current generation, Mappings, IPC-buffer bindings; a reference at any other -generation counts nothing on clone or drop. A destroyed object's slot is therefore **reused at once, its -generation advanced**, and stale capabilities fail closed — what RFC-0003 §8's generation exists for — so no -past holder pins another's charge. A slot whose generation cannot advance retires, debited once from its -payer and kept by the kernel thereafter (objects keep 64-bit generations, RFC-0007 (proposed) §6). -`object.rs`'s "takes a reference count" gains that qualification in increment 7. - -**Where it lives.** The generic slot array and Pool are safe code in `memory/`. The concrete store — one -`Kernel` value with every type's array, RFC-0004/0006/0007 objects included — lives in the kernel crate -behind **one** `static` wrapper in `kernel/src/mm/` with one SAFETY argument: one core, kernel entered with -interrupts masked (RFC-0006 (proposed) V1), the big lock once SMP lands. No other kernel global accretes. +capabilities at the slot's current generation, Mappings, IPC-buffer bindings; one at any other generation +counts nothing on clone or drop. A destroyed object's slot is therefore **reused at once, generation +advanced**, stale capabilities failing closed — what RFC-0003 §8's generation exists for — and no past holder +pins another's charge. A slot whose generation cannot advance retires, debited once from its payer (objects +keep 64-bit generations, RFC-0007 (proposed) §6). `object.rs`'s "takes a reference count" is qualified to +match in increment 7. **Where it lives:** the generic array and Pool are safe code in `memory/`; the concrete +store — one `Kernel` value with every type's array, RFC-0004/0006/0007 objects included — sits behind **one** +`static` wrapper in `kernel/src/mm/` with one SAFETY argument: one core, kernel entered with interrupts +masked (RFC-0006 (proposed) V1), the big lock once SMP lands. No other kernel global accretes. **A Pool is the accounting layer**: an RFC-0003 object holding balances of frames and of each type's slots. At boot every free unit becomes the balance of one root Pool (RFC-0007 (proposed) hands it over). Then: @@ -125,26 +121,24 @@ every unit of r in use == kept by the kernel (at boot, or retired), or ch ``` - **Charged:** Region frames; every object slot; table frames and the `Mapping` slot, to the Pool `map` - names (§8). **Kept, never charged:** image, bitmap, slot arrays, kernel tables, per-core stacks, - boot-minted Regions (§7). Every charge presents a Pool with `WRITE`. + names (§8). **Kept, never charged:** image, bitmap, arrays, kernel tables, per-core stacks, boot-minted + Regions (§7). Every charge presents a Pool with `WRITE`. - **Charge first, then take**, so the only later failure, `Fragmented`, refunds before returning. - **`PoolExhausted` names the Pool and resource**; nothing overcommits *(research/0002 Part 6, "budgets are - minima")*. Refund goes to the payer, whose index each object records. + `PoolExhausted` names the Pool and resource; nothing overcommits *(research/0002 Part 6, "budgets are + minima")*; refund goes to the payer, whose index each object records. - **Pools subdivide by moving balance.** `pool_split(pool)` makes an empty child recording its funder; - `pool_move(from, into, r, n)`; `pool_merge(from, into)` once `from` has no outstanding charges. -- **A Pool whose last capability closes** returns its free balance to its funder at once and stays as a - forwarding slot until its charges come home — a chain bounded by the Pool slot count. The root Pool has - no funder; closing it gives its balance to the kernel. + `pool_move(from, into, r, n)`; `pool_merge(from, into)` once `from` has no outstanding charges. **A Pool + whose last capability closes** returns its free balance to its funder and stays as a forwarding slot until + its charges come home — a chain bounded by the Pool slot count; the root's balance goes to the kernel. - **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). -**One accounting domain — decided here, as the programme assigns O-7's model to this RFC.** research/0002 +**One accounting domain — decided here, since the programme gives this RFC O-7's model.** research/0002 Part 6 asks for one capability-named domain for memory, object quotas and scheduling contexts. **The Pool is that domain for every counted unit**, Threads' and SchedContexts' slots included. **Time is not a unit**: RFC-0006 (proposed) sells per-core bandwidth through `Core`, which moving balance cannot conserve, and folding it in would make every Pool holder a seller of CPU — a deliberate split from the research, with -congruence kept by attribution, since every SchedContext is charged to a Pool. O-7's text would add -*"kernel memory is charged to a capability-named Pool; CPU time to a SchedContext funded through a -`Core`"*, proposed through `docs/CHANGELOG.md`. +congruence kept by attribution. O-7's text would add *"kernel memory is charged to a capability-named Pool; +CPU time to a SchedContext funded through a `Core`"*, proposed through `docs/CHANGELOG.md`. ## 6. The kernel's virtual layout @@ -161,17 +155,17 @@ for userspace, but that buys nothing); AArch64 follows. | `0xFFFF_FFFF_8000_0000` – top | image at `+0x20_0000` | `.text` RX, `.rodata` R, `.data`/`.bss` RW execute-never | - **Stacks.** RFC-0006 (proposed) V1 chose one kernel stack per core, allocated here at boot; core 0's boot - stack leaves `aarch64.ld`'s unguarded slot in increment 10. A fault's entry pushes onto the stack that - overflowed, so **RFC-0006 (proposed) provides the overflow stack** (x86_64 IST; on AArch64 the EL1-origin - vector moving to the emergency stack, a kernel fault being fatal). Until then an overflow is a recursive abort. + stack leaves `aarch64.ld`'s unguarded slot in increment 10. A fault's entry pushes onto the overflowed + stack, so **RFC-0006 (proposed) provides the overflow stack** (x86_64 IST; AArch64's EL1-origin vector + moving to the emergency stack, a kernel fault being fatal). Until then an overflow is a recursive abort. - **The user floor is 64 KiB** *(Linux `mmap_min_addr`)*; **the ceiling leaves the last page below the hole unmapped**, closing Intel's `SYSRET` hazard (CVE-2012-0217) for RFC-0007 by layout. The physmap is RAM only. - **The kernel's top-level entries never change after boot.** AArch64 shares `TTBR1_EL1`; x86_64 copies PML4 entries 256–511 into each address space *(seL4 x86)*, so every kernel-half PML4E ever used is populated at - boot — one filled later would be missing from earlier copies (Linux's vmalloc-sync bug family). Lower - kernel tables may change; that is a global change, invalidated on every core (§12). *Rejected:* an - identity-mapped kernel — it puts the kernel where user addresses go and breaks the symmetric halves; no - physmap (an invalidation per IPC-buffer touch). + boot — one filled later would be missing from earlier copies (Linux's vmalloc-sync bugs). Lower kernel + tables may change, as global changes invalidated on every core (§12). *Rejected:* an identity-mapped + kernel, which sits where user addresses go and breaks the symmetric halves; no physmap, which costs an + invalidation per IPC-buffer touch. ## 7. The objects: Region, AddressSpace and Mapping @@ -181,31 +175,28 @@ frames, with rights `READ`, `WRITE`, `EXECUTE` and RFC-0003's `DUPLICATE`, `TRAN and IPC-buffer bindings hold counted references, so closing the last capability does not pull memory from under a live mapping *(Zircon: a mapping keeps its VMO alive)*. One object, three kinds: -- **Ordinary** (`region_create`): eager, zeroed, charged; a request the bitmap cannot meet in eight extents - fails `Fragmented`; DMA memory is one extent at an alignment. Zeroing runs 512 pages per step between - RFC-0006 (proposed)'s preemption points, the half-built Region held on the caller's thread, no capability - existing until it is done. -- **Boot-minted, read-only**, once, over the DTB and each boot module RFC-0007 (proposed) embeds. Their - `DeviceTree`/`KernelImage` frames **move** into the Region; never zeroed (public boot data: O-8 is - untouched), never charged, never `WRITE`: the DTB gets `READ`, a module `READ | EXECUTE | DUPLICATE | +- **Ordinary** (`region_create`): eager, zeroed, charged; unmet in eight extents it fails `Fragmented`; DMA + memory is one extent at an alignment. Zeroing runs 512 pages per step between RFC-0006 (proposed)'s + preemption points, the half-built Region held on the caller's thread; no capability exists until done. +- **Boot-minted, read-only**, once, over the DTB and each boot module RFC-0007 (proposed) embeds: their + `DeviceTree`/`KernelImage` frames **move** into the Region, never zeroed (public boot data: O-8 is + untouched), never charged, never `WRITE`. The DTB gets `READ`; a module `READ | EXECUTE | DUPLICATE | TRANSFER`, so text maps `RX` in place and W^X holds because no writable use can exist. RFC-0007's bundle must page-align every module segment. - **Device**, minted at boot over `Mmio` the kernel does not keep; never executable, zeroed or charged; Phase 2. No operation maps a physical address given as a number: the catch-all right in costume. -What the kernel fills for the root before handover (writable segments, stack, BootInfo) is ordinary Regions: -charged to the root Pool, bytes copied in through the physmap, only then a capability minted. +What the kernel fills for the root before handover (writable segments, stack, BootInfo) is ordinary Regions, +charged to the root Pool, bytes copied in through the physmap, and only then a capability minted. **An AddressSpace** is a user root table, a tag (§12), the Pool paying for its tables — the only Pool `map` may charge — and its mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a process is RFC-0007's. **A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, -vaddr, pages, perms}`, linked by slot index into its AddressSpace's list and its Region's. - -**Why a Region remembers its mappings.** RFC-0003's 2026-07-30 amendment makes load-bearing that *a resolved -capability is never cached outside the kernel table*. **A page-table entry is exactly such a cache**, which -hardware never re-checks; the per-Region list is the compensation. *Rejected:* **Zircon VMOs and VMARs** — -lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in kernel paths; **seL4's -capability per frame** — 256 handles per MiB; **userspace-built page tables** (Xen PV). +vaddr, pages, perms}`, linked by slot index into its AddressSpace's list and its Region's — because RFC-0003's +2026-07-30 amendment makes load-bearing that *a resolved capability is never cached outside the kernel +table*, and **a page-table entry is exactly such a cache**, which hardware never re-checks. *Rejected:* +**Zircon VMOs and VMARs** — lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in +kernel paths; **seL4's capability per frame** — 256 handles per MiB; **userspace-built tables** (Xen PV). ## 8. Map, unmap, protect — and the rights they check @@ -224,18 +215,18 @@ table, never transferred: RFC-0003 §6's move would take the Region from its map User numbers are validated *as numbers*; **the kernel never chooses a virtual address** *(seL4's VSpace discipline)*. **`map` names a Pool because spending is authority** (O-4): an AddressSpace capability alone cannot drain one, and requiring the space's *own* Pool keeps one payer per space, so refunds need no -per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool (a payer record -per table frame); a Pool bound at `as_create` and spent through `WRITE` (spending never granted). **`map` is -atomic**: it charges the missing tables and the Mapping slot before writing anything, then zeroes and links -tables, writes leaves and ends with `DSB ISHST`, so the walker sees them before `eret`. One call: at most 512 -leaves and six new tables. `protect` only narrows; widening is `map` again *(seL4's remap)*. 4 KiB pages +per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool (a payer record per +table frame); a Pool bound at `as_create` and spent through `WRITE` (spending never granted). **`map` is +atomic**: it charges missing tables and the Mapping slot before writing anything, then zeroes and links +tables, writes leaves and ends with `DSB ISHST`, so the walker sees them before `eret`; at most 512 leaves +and six new tables per call. `protect` only narrows; widening is `map` again *(seL4's remap)*. 4 KiB pages only; blocks later, invisible in the ABI. **Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as RFC-0003 §5 allows "when a subsystem needs one". Folding it into `READ` makes every readable Region potential code; a Region *kind* fails the loader, which writes code before running it. seL4 leaves execution ungoverned; Fuchsia's `ZX_RIGHT_EXECUTE` -is the lineage. It means *may map executable* on a Region and *may mint executable Regions* on a Pool, and -is **refused at mint on every other type** by the store (host-tested, increment 7); `derive` cannot add it. +is the lineage. It means *may map executable* on a Region, *may mint executable Regions* on a Pool, and is +**refused at mint on every other type** by the store (host-tested, increment 7); `derive` cannot add it. `rights.rs`'s `ALL` is an explicit union despite its comment; increment 4 edits both. **On acceptance, RFC-0003 gains a dated amendment adding `EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 §14.3's handshake: **the installed permission is `perms`, refused unless a subset of the Region's rights.** @@ -261,16 +252,16 @@ execute-only exists on AArch64 but not on x86_64 without protection keys, so nei the loader holds one, a JIT gets one by explicit grant *(Fuchsia's `VmexResource`)*. O-13's Phase-3 load-path clause is policy on this. 4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); user - entries set `PXN`; the image's text has no writable alias, the physmap mapping it read-only *(Linux + entries set `PXN`; kernel text has no writable alias, the physmap mapping it read-only *(Linux `mark_linear_text_alias_ro`)*. x86_64 has no `WXN`: `CR0.WP` and `SMEP` where present. `map RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` on a -PIPT instruction cache and `IC IALLUIS` otherwise (Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`; QEMU's +PIPT instruction cache (`IC IALLUIS` otherwise; Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`; QEMU's cortex-a72 reports PIPT (`CTR_EL0 = 0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so errors here pass every test.** These and §12's `TLBI`s and `TTBR` writes are single-instruction `asm!` intrinsics in `kernel/src/arch/**` with no Rust equivalent, as `wfe` is today — a reading of "minimal -assembly" the maintainer is asked to confirm (§18.7). The physmap still aliases *user* code writable at EL1; -writing through it only to zero, fill tables and reach IPC buffers is a convention (§17). +assembly" the maintainer is asked to confirm (§17.7). The physmap still aliases *user* code writable at EL1; +writing through it only to zero, fill tables and reach IPC buffers is a convention (§16). ## 10. Sharing and destruction (RFC-0004 §9.1) @@ -278,21 +269,20 @@ writing through it only to zero, fill tables and reach IPC buffers is a conventi `EXECUTE`, `DUPLICATE` or `REVOKE`, and maps it where it chooses; bulk data travels as RFC-0004 §5's descriptor. No `share` syscall and no page-table work during IPC — the long-IPC grave stays shut. - **Destruction** — at the last live reference, or through RFC-0003 §7.1's destruction case (its right is - open, §18.3) — marks the Region *dying*, so no new mapping, binding or copy resolves it; tears down every - mapping and binding in every address space (§7 says why); completes the invalidation (§12); advances the - generation and refunds the payer. Its preemption points fall between whole mappings. -- **Donation is not proposed.** pKVM's *donate* would be `reissue` — bump, re-mint for the caller over the same - frames, move the charge to the recipient's Pool — which is RFC-0003 §7's option B2 on one object. Gating - it on `REVOKE` would widen that right beyond "capabilities derived from this one", and under the - amendment's option **(a)** a peer-transferred Region carrying `REVOKE` could cut the broker off. It is - **input to RFC-0003a's (a)/(b) choice**; the demo needs none of it. + open, §17.3) — marks the Region *dying*, so no new mapping, binding or copy resolves it; tears down every + mapping and binding in every address space; completes the invalidation (§12); advances the generation and + refunds the payer. Its preemption points fall between whole mappings. +- **Donation is not proposed.** pKVM's *donate* would be `reissue` — bump, re-mint for the caller over the + same frames, move the charge to the recipient's Pool — RFC-0003 §7's option B2 on one object. Gating it on + `REVOKE` would widen that right beyond "capabilities derived from this one", and under the amendment's + option **(a)** a peer-transferred Region carrying `REVOKE` could cut the broker off. It is **input to + RFC-0003a's (a)/(b) choice**; the demo needs none of it. **Concurrency.** Under RFC-0006 (proposed) V1 — non-preemptible, one big lock once SMP lands — no kernel -access to a Region's frames (zeroing, table fill, IPC copy) interleaves with its destruction: each runs -whole in one kernel entry and re-resolves the Region, and the dying mark covers the gaps between -destruction's steps. If locking ever becomes finer, the order is **Region before AddressSpace**. A former -sharer takes an ordinary fault: RFC-0006 (proposed) binds delivery, RFC-0007 (proposed) defines the message, -and under RFC-0007's interim the thread stops. +access to a Region's frames (zeroing, table fill, IPC copy) interleaves with its destruction: each runs whole +in one kernel entry and re-resolves the Region, and the dying mark covers the gaps between destruction's +steps. Finer locking, if ever, takes **Region before AddressSpace**. A former sharer takes an ordinary fault: +RFC-0006 (proposed) binds delivery, RFC-0007 (proposed) defines the message; under its interim the thread stops. ## 11. How the kernel touches user memory (O-5) @@ -307,12 +297,11 @@ writes for a process is named by capability and reached through the physmap: reference, is a writable use for §9, and destruction severs it. - **No kernel path faults on user memory**: frames are eager and pinned, so the slow path copies frame to frame, and the in-kernel fault concurrency that sank L4's long IPC cannot arise. -- **Peer-writable memory is never referenced**: one `kernel/src/mm/` module copies with volatile accesses, - and anything the kernel interprets is copied once and validated as that snapshot *(virtio's double-fetch - lesson, research/0002 Part 5)*. +- **Peer-writable memory is never referenced**: one `kernel/src/mm/` module copies with volatile accesses; + anything interpreted is copied once and validated as that snapshot *(virtio's double fetch, research/0002 + Part 5)*. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. -*Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. `SMAP` and -`PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. +`SMAP` and `PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. ## 12. TLBs, ASIDs and PCIDs @@ -333,13 +322,12 @@ the set of cores that have run it since its last full flush *(Linux `mm_cpumask` - **Shootdown.** AArch64 needs no IPI: the `…IS` forms reach the inner-shareable domain and `DSB ISH` waits for completion everywhere. x86_64 has no broadcast: the initiator IPIs the cores in the set, each - invalidates and acknowledges, and **no frame is freed nor W^X count lowered before every - acknowledgement**. A core spinning on the big lock polls its flush request, so the holder never waits on - a core waiting for it. -- **Rollover.** On exhaustion the generation bumps, every core's *active* tag is reserved into the new one, - and every core gets a pending-flush mark, consumed by a local `TLBI VMALLE1` at its next switch. **A tag is - reallocated only after every core that ran under it has flushed** *(Linux `arch/arm64/mm/context.c`: - `active_asids`, `reserved_asids`, `tlb_flush_pending`)*. + invalidates and acknowledges, and **no frame is freed nor W^X count lowered before every acknowledgement**. + A core spinning on the big lock polls its flush request, so the holder never waits on a core waiting for it. +- **Rollover.** The generation bumps, every core's *active* tag is reserved into the new one, and every core + gets a pending-flush mark, consumed by a local `TLBI VMALLE1` at its next switch. **A tag is reallocated + only after every core that ran under it has flushed** *(Linux `arch/arm64/mm/context.c`: `active_asids`, + `reserved_asids`, `tlb_flush_pending`)*. - **x86_64 PCIDs are per core**: each core caches a few AddressSpace→PCID pairs *(Linux keeps six)*; an invalidation where the space is not current is a per-core stale mark consumed at the next switch to it. @@ -366,16 +354,12 @@ while `.text.boot` leads the first executable segment. With the MMU off, the stu | `SCTLR_EL1` field (bit) | Value | Why | |-------------------------|-------|-----| -| `M` (0), `C` (2), `I` (12) | 1 | MMU and caches on | -| `A` (1) | 0 | alignment is the compiler's; both targets are `+strict-align` | -| `SA` (3), `SA0` (4) | 1 | stack-alignment checks at EL1 and EL0 | -| `CP15BEN` (5), `ITD` (7), `SED` (8) | 0 | AArch32 controls; no thread runs AArch32 (RFC-0007 builds every `SPSR`) | -| `UMA` (9) | 0 | EL0 cannot mask interrupts, so cannot defeat preemption | +| `M` (0), `C` (2), `SA` (3), `SA0` (4), `I` (12) | 1 | MMU and caches on; stack-alignment checks at EL1 and EL0 | +| `A` (1), `CP15BEN` (5), `ITD` (7), `SED` (8), `E0E` (24), `EE` (25) | 0 | alignment is the compiler's (`+strict-align`); AArch32 controls, and no thread runs AArch32; little-endian | +| `UMA` (9), `UCT` (15), `UCI` (26) | 0 | EL0 may neither mask interrupts (defeating preemption) nor maintain caches — the kernel does that at `map RX` (§9) | | `DZE` (14) | 1 | EL0 `DC ZVA` is a store its mapping already permits | -| `UCT` (15), `UCI` (26) | 0 | EL0 does no cache maintenance; the kernel does it at `map RX` (§9) | | `nTWI` (16), `nTWE` (18) | 0, 1 | `wfi` traps and `wfe` does not — RFC-0006 (proposed) §8 | | `WXN` (19) | 0, then 1 | set by Rust step (5), giving `0x30DC581D` | -| `E0E` (24), `EE` (25) | 0 | little-endian | | 11, 20, 22, 23, 28, 29 | 1 | RES1 on Armv8.0: Linux v4.19's `SCTLR_EL1_RES1`, plus bit 23 (`SPAN`, RES1 before Armv8.1), which Linux always sets; not re-checked against the Arm ARM | Every other bit is 0: RES0 on Armv8.0, or a later extension left off. Rust then, never leaving the reporter @@ -403,7 +387,7 @@ jumps high and drops the identity entry — the same "Rust runs only high" rule. | O-3 revocability | destruction walks the Region's mappings, invalidates, then bumps; a page-table entry is a cached resolution (§7, §10) | **destruction case extended to memory**; selective revocation and donation are RFC-0003a's | | O-5 argument validation | no user virtual address dereferenced; carried ones are numbers; memory by capability through the physmap; snapshots (§11) | **discharged for the memory surface**; RFC-0007 inherits §11's sentence | | O-6 kernel memory safety | new `unsafe` only in `mm/**` (window access, volatile copy, the store's static) and `arch/**` (tables, TLB, boot) | **stays Built** — every new block listed per PR | -| O-7 no unprivileged exhaustion | arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map`'s too; charge-first; `map` ≤ 512 leaves; zeroing in 512-page steps; run search bounded by RAM (§4, §5, §8) | **discharged for kernel allocation**; a transferred object stays charged to its payer until destroyed — **open** with RFC-0003a (§18.2); CPU time is RFC-0006's | +| O-7 no unprivileged exhaustion | arrays reused by generation; every post-boot unit charged to a Pool named by capability, `map`'s too; charge-first; `map` ≤ 512 leaves; zeroing in 512-page steps; run search bounded by RAM (§4, §5, §8) | **discharged for kernel allocation**; a transferred object stays charged to its payer until destroyed — **open** with RFC-0003a (§17.2); CPU time is RFC-0006's | | O-8 address-space disjointness | per-process tables; Regions only by capability; frames and tables zeroed before use; invalidate before free (§4, §7, §12) | **discharged** | | O-9 no ambient side channel | no shared mapping a capability did not establish (§10) | **contributes** | | O-13 W^X | four layers, per Region across all spaces, counts falling only on completed invalidation (§9) | **discharged at mapping**; the load-path clause is Phase-3 policy over layer 3 | @@ -420,15 +404,7 @@ jumps high and drops the identity entry — the same "Rust runs only high" rule. - **The catch-all right.** `EXECUTE` is narrow and refused off-type; no "map any physical address" right. - **Compiled-in unused device paths (VENOM).** The kernel maps only the devices it drives itself. -## 16. Lineage in one place - -seL4 (frames as capability objects, the kernel window, caller-paid tables, all memory to the first task); -pKVM (one owner per frame, *share*); Zircon/Fuchsia (mappings keep a region alive, `EXECUTE`, `VmexResource`); -Hubris, Fiasco.OC and Genode (sized tables, counted quotas); Linux (the linear map and its read-only text -alias, ASID rollover, `mm_cpumask`, per-core PCIDs); Ge et al. (colouring). Left: Untyped and the CDT, lazy -VMOs, the kernel heap, the buddy allocator, `copy_from_user`. - -## 17. Costs — what this makes harder +## 16. Costs — what this makes harder - **Capacities are ceilings**: a slot count is a rebuild; `pool_move` is one call per resource. - **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `Fragmented`. @@ -443,7 +419,7 @@ VMOs, the kernel heap, the buddy allocator, `copy_from_user`. - **Reclaim across a transfer needs cooperation**: a transferred object keeps its payer charged until destroyed, so `pool_merge` waits on other processes — RFC-0003a's problem made concrete. -## 18. Open questions +## 17. Open questions 1. **Page-table frames on unmap.** Freed only with the AddressSpace; eager freeing needs §12's walk-cache rule. 2. **Donation and reclaim across a transfer** — `reissue` with a charge move, offered to RFC-0003a (§10). @@ -454,9 +430,9 @@ VMOs, the kernel heap, the buddy allocator, `copy_from_user`. 6. **The firmware assumption** (§4) and O-7's new text (§5), via `docs/CHANGELOG.md`. 7. **"Minimal assembly"** (Constitution §11.1): are runtime single-instruction `TLBI`/`DC`/`IC`/`TTBR` intrinsics in `arch/**` within it (§9), or does the clause need amending? -8. **One programme order** across RFC-0005/0006/0007 — §19 proposes one for the maintainer to fix. +8. **One programme order** across RFC-0005/0006/0007 — §18 proposes one for the maintainer to fix. -## 19. Implementation increments +## 18. Implementation increments One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate — `no_std` in the kernel, `std` under `cargo test`, built for both targets in CI, no `unsafe`, as `capability/`. @@ -465,19 +441,17 @@ empty `MemoryMap`, on which memory initialisation prints `mm: no memory map (UEF so the second Tier-1 build compiles the whole path from increment 2. 1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU off. `--expect "DTB 0xd00dfeed at 0x40000000"`. -2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture and hostile blobs (an - OVMF `GetMemoryMap` dump once the stub exists); workspace `members`, `.github/workflows` and `CLAUDE.md` - § Layout gain `memory/`. `--expect "mm: 512 MiB RAM at 0x40000000"`. +2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture, hostile blobs and later an + OVMF dump; workspace `members`, `.github/workflows`, `CLAUDE.md` § Layout. `--expect "mm: 512 MiB RAM at 0x40000000"`. 3. **[demo] Frame bitmap.** Churn tests against a shadow model: no frame twice, none reserved, no image frame free. 4. **[demo] `EXECUTE` in `capability/`** with RFC-0003's dated amendment: bit 5, `ALL` and its comment, `Debug`, the test widened to `0..64`. Demo-needed because the root maps the client's text. 5. **[demo] `MapPerms` and both encoders.** Exhaustive: nothing writable and executable; every user entry `PXN`, `AF`, not-global; every row of §8's table. -6. **[demo] The table engine** over the frame-access trait, on both formats: overlap, the 512-leaf and - six-table bound, nothing mapped when unpaid, tables zeroed before linking, a kernel-half map never - writing the root. -7. **[demo] Slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale - references count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. Host-only. +6. **[demo] The table engine** on both formats: overlap, the 512-leaf and six-table bound, nothing mapped when + unpaid, tables zeroed before linking, a kernel-half map never writing the root. +7. **[demo] Slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale references + count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. Host-only. 8. **`pool_split`/`move`/`merge` and closure forwarding**, under random churn. Host-only; not demo. 9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode; prints the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. @@ -493,13 +467,12 @@ so the second Tier-1 build compiles the whole path from increment 2. self-test maps a fresh Region and prints `[el0] map ok`. 15. **`unmap` and `protect`** with invalidation and count decrements (§9, §12). Not demo. 16. **User-memory module (§11)**: a message beyond the register budget. Not demo. -17. **Destruction (§10)**, after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches - its handler. Not demo. +17. **Destruction (§10)**, after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches its handler. 18. **Stack overflow**, with RFC-0006's overflow stack: `provoke-stack-overflow` reaches the reporter. 19. **PAN, SMAP, SMEP** where reported, with a `provoke-user-deref` self-test under `-cpu max`. 20. **x86_64** tables, `CR3` and PCID, after the UEFI stub; **Device Regions** in Phase 2. -**Programme order proposed** (§18.8): 1–8 beside RFC-0006 1–2 and RFC-0007 1–5; RFC-0007's MMU-off EL0 +**Programme order proposed** (§17.8): 1–8 beside RFC-0006 1–2 and RFC-0007 1–5; RFC-0007's MMU-off EL0 self-test (its 3) before 9, which retires it; 9–10 before RFC-0006 3 (or 10 repoints the GIC too) and 4, which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7; 14 with RFC-0007 6. @@ -510,19 +483,16 @@ which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7; 14 w designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. - **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. - **Until increment 10 the kernel runs RWX throughout**; cache maintenance stays unverified. -- **1 GiB boot blocks** cover 512 MiB with no RAM behind them, which speculation may touch on real - hardware: there, 2 MiB blocks sized from the DTB. -- **Zeroing in one step** until RFC-0006's preemption points; the demo's Regions are a few pages. +- **1 GiB boot blocks** cover 512 MiB with no RAM behind them, which speculation may touch on real hardware: + there, 2 MiB blocks sized from the DTB. **Zeroing in one step** until RFC-0006's preemption points. - **A guarded-stack overflow is a recursive abort** until increment 18. - **Not an interim:** the root's image reaches it through boot-minted and kernel-filled Regions and increment 13's `map`, so §8's rights check runs even at boot. -## 20. What this unblocks +## 19. What this unblocks Increments 1–14 give RFC-0006 (proposed) address spaces, a guarded per-core stack and a Pool to charge threads to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a -constraint it inherits rather than discovers: **the kernel never dereferences a user virtual address; where -a syscall carries one, it is a number, validated as a number and used only to program tables or registers.** -The root receives all memory as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient -grantor. Increments 3–8 are host-only and 1–2 change only where the kernel loads and what it prints: all can -land now. +constraint it inherits rather than discovers: §11's sentence on user addresses. The root receives all memory +as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor. Increments 3–8 are +host-only and 1–2 change only where the kernel loads and what it prints: all can land now. diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 796201a..344d024 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -7,21 +7,36 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 ("IPC is the product", priority inheritance, bolted-on multicore); the HAL (`kernel/src/arch/**`) and `aarch64.ld`'s stack plan; RFC-0003 §14.2; RFC-0004 §4, §8, §9.2 and its 2026-08-01 amendment; every userspace driver | -| Depends on | RFC-0003 (capability table); RFC-0004 (IPC) and its 2026-08-01 amendment; RFC-0005 (proposed) for object memory, kernel stacks and address-space switching; RFC-0007 (proposed) for the trap path, the syscall encoding, the Process object and the root task | -| Discharges | O-7 for CPU time and the scheduler's own structures; O-4 for time; both commitments of RFC-0004's 2026-08-01 amendment; answers RFC-0003 §14.2 and RFC-0004 §9.2 in shape; contributes to O-5, O-9, O-17 | +| Affects | Constitution §3 ("IPC is the product", priority inheritance, bolted-on multicore); the HAL (`kernel/src/arch/**`), `boot.s`'s park loop and `aarch64.ld`'s stack plan; RFC-0003 §14.2 (a dated note); RFC-0004 §4, §8, §9.2, §9.3 and its 2026-08-01 amendment (three proposed amendments, §13); research/0002 Part 6 (one correction); RFC-0005 §13 step 5 and §18.2; `CLAUDE.md` § Architectures (FP enabled per thread) and § Layout (gains `scheduler/`); every userspace driver | +| Depends on | RFC-0003 (capability table); RFC-0004 (IPC) and its 2026-08-01 amendment; RFC-0005 (proposed) for object memory, kernel stacks, discovery and address-space switching; RFC-0007 (proposed) for the trap path, the syscall encoding, the Process object and the root task | +| Discharges | O-7 for CPU time and the scheduler's own structures; O-4 for time; both commitments of RFC-0004's 2026-08-01 amendment **as amended by verdict 12**; answers RFC-0004 §9.2 in shape; contributes to O-5, O-9, O-17 | > **Proposed verdicts.** > -> - **V1 — Event kernel (the load-bearing choice).** One kernel stack per core; the kernel is non-preemptible; every wait is a thread state. -> - **V2 — Thread.** An RFC-0003 object with a fixed-size TCB holding its user frame; nine states; `yield` and `exit` are authority-free. -> - **V3 — Time is a capability.** A `SchedContext` (budget, period, ≤ 8 refills) is an object, configured only through a per-core `Core`. -> - **V4 — Priorities and dispatch.** 256 fixed levels; an MCP guard, no scheduling right; O(1) bitmap run queues; FIFO endpoint queues. -> - **V5 — Donation and the amendment.** `call` lends time to a passive server at `max(own, caller's)` priority; an RFC-14 threshold; one-hop `BudgetExpired`. -> - **V6 — Timer.** Tickless one-shot deadlines: the AArch64 EL1 virtual timer; the x86_64 LAPIC, TSC-deadline only with an invariant TSC. -> - **V7 — FP/SIMD.** Off by default, enabled per thread, loaded on first use, saved lazily by owner tracking; the kernel never touches it. -> - **V8 — Interrupts.** GICv3 first; LAPIC and I/O APIC; a device line is an `IrqHandler` that signals a notification, masked until `ack`. -> - **V9 — Multi-core shape.** Per-core state; a big lock when SMP lands; cores parked until funded; SMT placement classes; exclusivity as a degenerate case. +> 1. **Event kernel:** one kernel stack per core; the kernel is not preemptible but its long paths take +> preemption points; every wait is a thread state (§3). +> 2. **Thread:** an RFC-0003 object holding its user frame; ten states, one of them `Unfunded`; born at +> priority 0 with ceiling 0; its fault and timeout endpoints held in the thread, not its process (§4). +> 3. **Time is a capability:** a `SchedContext` (budget and period in nanoseconds, at most eight refills), +> configured only through a per-core `SchedControl` whose one method sells time on that core (§5). +> 4. **Budgets are caps, not minima:** the kernel enforces the cap; a guaranteed minimum is admission +> policy — a stated departure from research/0002 (§5). +> 5. **Priorities:** 256 fixed levels; any capability to a thread lends its priority ceiling (seL4's MCP); +> no scheduling right; O(1) bitmap run queues (§6). +> 6. **Endpoint queues are priority-ordered,** first-come within a level, as seL4 MCS (§6). +> 7. **Donation:** `call` lends time to a *passive* server, which runs at the higher of its own and its +> caller's priority; a caller's death never takes time from a server mid-request (§7). +> 8. **Budget guard:** an endpoint's `min_budget` is checked at `call`; a caller short *now* waits; expiry +> inside a server hands time back one hop with `BudgetExpired` and a timeout fault (§7). +> 9. **Timer:** tickless per-core deadlines — the AArch64 virtual timer; the x86_64 LAPIC, TSC-deadline +> only with an invariant TSC; userspace timers will be deadline objects on the same queue (§8). +> 10. **FP/SIMD:** off by default, enabled per thread, saved lazily; the kernel's only FP instructions +> are its save and restore stubs (§10). +> 11. **Interrupts and cores:** GICv3 first, LAPIC and I/O APIC; a device line signals a notification and +> stays masked until `ack`; a big lock whose waiters serve IPIs; cores started by PSCI or INIT-SIPI +> when first funded (§11, §12). +> 12. **Amendments proposed:** three to RFC-0004 and its 2026-08-01 amendment, one note on RFC-0003 +> §14.2, one correction to research/0002 (§13). ## 1. The question @@ -38,11 +53,11 @@ admits no workloads and balances no load; that is for whoever holds the capabili **The kernel doctrine (§3) directly, and pillar 2 by consequence.** "IPC is the product" holds only if a `call` hands over the CPU and the budget without a scheduler round trip (RFC-0004). Time is the one resource every thread consumes; if it is not a capability, pillar 2 has a hole exactly where -availability (A5) lives. seL4 MCS made time a capability, and this RFC adopts that shape. +availability (A5) lives. seL4 MCS made time a capability, and this RFC adopts that shape. Borrow Ledger: +scheduler, timer and interrupt-controller code are the **microkernel core — write ourselves**; Redox's +drivers are read for reference only, since the kernel masks and signals but never services a device. -## 3. The execution model — one kernel stack per core (V1) - -The load-bearing choice, because it decides what a context switch *is*. +## 3. The execution model — one kernel stack per core (verdict 1) - **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). **Rejected:** a guarded stack per thread in a kernel with no heap, and blocking in the kernel is how "hold a lock across @@ -50,401 +65,521 @@ The load-bearing choice, because it decides what a context switch *is*. - **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). **Verdict sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with interrupts masked, and leaves by restoring *some* thread's user frame. A syscall that must wait records - a state and the object waited on; on wake, the result is written into the waiter's frame, or the call + a state and the object waited on; on wake the result is written into the waiter's frame, or the call restarts by rewinding the saved PC one instruction (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & - Elphinstone, "L4 Microkernels: The Lessons from 20 Years of Research and Deployment", TOCS 2016; Ford - et al., "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* - -**The cost:** every kernel path must be bounded. The long ones — object destruction, RFC-0005 -(proposed)'s `reissue` and address-space teardown — take **preemption points**: on a pending interrupt -(`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. The worst path is measured and -published: it is the interrupt latency. This **reverses `aarch64.ld`'s plan** that the boot stack "will be -replaced by per-thread stacks": it becomes core 0's kernel stack, in RFC-0005 (proposed)'s guarded window. - -## 4. The Thread object (V2) + Elphinstone, TOCS 2016; Ford et al., "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* + +**The cost:** every kernel path must be bounded. Long ones — object destruction, RFC-0005 (proposed)'s +`reissue` and teardown, RFC-0007 (proposed)'s console write — take **preemption points**: on a pending +interrupt (`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. **A restarted +operation re-resolves every handle and re-checks every right from scratch**, and an object mid-teardown +carries a *dying* mark so no other entry invokes it between the exit and the restart. The worst path +between preemption points is measured and published: it is the interrupt latency. This **reverses +`aarch64.ld`'s plan** that the boot stack "will be replaced by per-thread stacks": it becomes core 0's +kernel stack, moved into RFC-0005 (proposed)'s guarded window (increment 5). + +**Idle is a kernel loop, not a thread,** billing nobody. It runs `wfi` with `PSTATE.I` set — a pending +interrupt still wakes it — then unmasks for one instruction; Linux warns that masking at `ICC_PMR_EL1` +would *not* wake the core (`arch/arm64/kernel/idle.c`), so this kernel masks with DAIF only. x86_64 uses +`sti; hlt`, whose STI shadow issues the `hlt` before delivery. **An interrupt taken in idle abandons it:** +the current-EL IRQ stub (this RFC's; RFC-0007 owns the lower-EL and `syscall` entries) saves nothing, +resets `SP` to the per-core stack top from the `TPIDR_EL1` block (`TSS.RSP0`'s twin on x86_64) and enters +the dispatcher; idle is never resumed. Idle is the only place the kernel unmasks, and it drops the big +lock (§12) before waiting. + +## 4. The Thread object (verdict 2) A **Thread** is an RFC-0003 object belonging to one Process, which RFC-0007 (proposed) provides. Its TCB -is a slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool presented at creation; -queue links are slot **indices**, not pointers. +is a slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool presented at creation. | Field | Meaning | |-------|---------| | `frame` | the full user register file, written at every kernel entry (§9) | | `state`, `blocked_on`, links | a state below; the endpoint, reply object or notification waited on; intrusive queue links | -| `priority`, `mcp` | base priority and maximum controlled priority (§6) | +| `priority`, `mcp` | base priority and maximum controlled priority (§6); both **0 at creation** | | `home_sc`, `sc`, `effective` | the bound context, if any; the context held now, own or donated; the priority run at now (§7) | -| `bound_notification`, `ipc_buffer` | the notification a `recv` may also wake on (RFC-0004 §10); the IPC-buffer page bound by capability (RFC-0005, RFC-0007, proposed) | -| `fault_ep`, `timeout_ep` | **handles in the thread's own process table**, resolved when raised, so a resolved capability lives only in a table (RFC-0003) and a revoked handler fails closed *(seL4 master; MCS caches them in the TCB)* | -| `fp_enabled`, `fp_state`, `class` | the FP/SIMD flag and save area (§10); the SMT placement class (§12) | +| `bound_notification`, `ipc_buffer` | the notification a `recv` may also wake on (RFC-0004 §10); the IPC-buffer binding (RFC-0007, proposed) | +| `fault_ep`, `timeout_ep` | **two kernel-held capability slots in the TCB** *(seL4 MCS)*: the capability is moved in by its binder, generation-checked when a fault is raised, and fails closed if its endpoint died; the thread cannot close it | +| `fp_enabled`, `fp_state` | the FP/SIMD flag and save area (§10) | + +**Every internal link is safe against slot reuse.** A queue link, `sc`, `home_sc`, a reply object's +record of a lent context, an `IrqHandler`'s notification and a core's FP owner are each either a counted +reference that holds the slot (RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare +index. This closes the ABA holes a slot array otherwise opens (§10's FP owner is the sharp one). | State | Meaning | Left by | |-------|---------|---------| -| `Inactive` | created or suspended; in no queue | `resume` | -| `Ready` | runnable with released budget; in its core's run queue | dispatch | -| `Running` | the current thread on its core; in no queue | preemption, block, `yield`, fault, `exit` | -| `Throttled` | runnable, but no released refill; in the release queue | refill release | -| `BlockedOnSend` | queued on an endpoint to `send` or `call` | rendezvous, endpoint destruction | -| `BlockedOnRecv` | waiting on an endpoint (and any bound notification) | rendezvous, signal | +| `Inactive` | created, suspended, or faulted with no handler; in no queue | `resume` | +| `Ready` | runnable with released budget; in its core's run queue | dispatch; `suspend`; context removed (→ `Unfunded`) | +| `Running` | the current thread on its core; in no queue | preemption, timeslice end, budget exhaustion (→ `Throttled`), block, `yield`, fault, `suspend`, `thread_exit`, context removed | +| `Throttled` | runnable, holding a context whose released budget is exhausted or below what its pending `call` needs; in the release queue | refill release; `suspend`; context removed | +| `Unfunded` | runnable but holding no context — a passive thread woken by a non-donating `send` or `notify`, one `resume`d with none, or a server left stranded (§7); in no queue | `bind_sc`, a donation, `suspend` | +| `BlockedOnSend` | queued on an endpoint to `send` or `call` | rendezvous, endpoint destruction, `suspend` | +| `BlockedOnRecv` | waiting on an endpoint (and any bound notification) | rendezvous, signal, `suspend` | | `BlockedOnReply` | waiting on a reply object after `call` or a fault message | reply, error completion (§7) | -| `BlockedOnNotification` | waiting on a notification alone | signal | +| `BlockedOnNotification` | waiting on a notification alone | signal, `suspend` | | `Exited` | finished; in no queue; the object lingers until destroyed | never | -Faults reuse IPC: the kernel `call`s the fault endpoint on the thread's behalf and it waits in -`BlockedOnReply`; RFC-0007 (proposed) defines the message. Destroying a thread bumps its generation -(RFC-0003 §7), unlinks it in O(1) from whichever doubly linked queue holds it, destroys any reply object -it waits on (RFC-0004 §8) and returns any donated context as budget expiry does (§7). - -| Operation | Authority | Effect | -|-----------|-----------|--------| -| `set_priority(t, auth, p)` / `set_mcp(t, auth, m)` | `t`: `WRITE`; `auth`: `READ`; value `≤ mcp(auth)` | §6 | -| `bind_sc(t, sc)` / `unbind_sc(t)` | `t`, `sc`: `WRITE` | active or passive; class checked (§12) | -| `resume(t)` / `suspend(t)` | `t`: `WRITE` | `Inactive` ↔ runnable; a first `resume` closes the Process's grant window (RFC-0007, proposed) | -| `read_regs(t)` / `write_regs(t, …)` | `READ` / `WRITE` | frame and TLS bases, sanitised (§9) | -| `set_fp(t, on)` / `set_class(t, c)` | `t`: `WRITE` | §10 / §12 | -| `sc_configure(core, sc, B, T, refills, class)` / `sc_stats(sc)` | `core`, `sc`: `WRITE` / `sc`: `READ` | §5 | -| `yield()` / `exit()` | none — two of RFC-0003 §9's three authority-free exceptions | tail of own priority / `Exited` | - -No right joins RFC-0003's set; RFC-0007 (proposed) owns the encoding. - -## 5. Scheduling contexts and `Core` — time as a capability (V3) +Faults reuse IPC: the kernel `call`s `fault_ep` on the thread's behalf, the thread waits `BlockedOnReply`, +and RFC-0007 (proposed) defines the message. Fault and timeout messages bypass `min_budget`. Destroying a +thread bumps its generation (RFC-0003 §7), unlinks it in O(1) from whichever doubly linked queue holds it, +and destroys any reply object it waits on (RFC-0004 §8) — with §7's rule for a context it had lent. + +Every operation is a method `call` on the **invoked** capability (RFC-0007 (proposed) verdict 1). Some +also name a second, **presented** capability, which is resolved in the caller's table and checked but +never moved and needs no `TRANSFER` — RFC-0007 (proposed) provides that encoding (seL4's `extraCaps`). + +| Operation | Invoked: right | Presented: right | Effect | +|-----------|----------------|------------------|--------| +| `set_priority(t, auth, p)` / `set_mcp(t, auth, m)` | `t`: `WRITE` | `auth`: any capability to a Thread, no right checked | value `≤ mcp(auth)` (§6) | +| `bind_sc(t, sc)` / `unbind_sc(t)` | `t`: `WRITE` | `sc`: `WRITE` / — | active ↔ passive | +| `bind_notification(t, n)` / unbind | `t`: `WRITE` | `n`: `READ` | a `recv` also wakes on `n` | +| `set_fault_ep(t, ep)` / `set_timeout_ep(t, ep)` | `t`: `WRITE` | `ep`: `WRITE`, **moved** into the TCB slot | the caller derives first to keep a copy | +| `resume(t)` / `suspend(t)` | `t`: `WRITE` | — | `Inactive` ↔ runnable; a first `resume` closes the Process's grant window (RFC-0007) | +| `read_regs(t)` / `write_regs(t, …)` | `t`: `READ` / `WRITE` | — | frame and `TPIDR_EL0`/`FS_BASE` only, sanitised (§9) | +| `set_fp(t, on)` | `t`: `WRITE` | — | §10 | +| `sc_configure(ctl, sc, B, T, refills, 0)` | `ctl`: `WRITE` | `sc`: `WRITE` | §5; the last word is the placement class, reserved zero (§12) | +| `sc_stats(sc)` / `ctl_info(ctl)` | `sc` / `ctl`: `READ` | — | §5's counters / §5's exclusivity check | +| `yield()` / `thread_exit()` | none — two of RFC-0003 §9's three authority-free exceptions | — | tail of own level / `Exited` | + +No right joins RFC-0003's set. **The supervision cost:** a process holding `WRITE` on its own thread can +rebind that thread's `fault_ep`. A supervisor keeps the binding by handing the child no `WRITE` on threads +it created for it; threads the child creates itself are supervised through the Process (RFC-0007 open +question 1). `thread_exit` takes one `WRITE` on self for a TLS runtime to set `FS_BASE` via `write_regs`. + +## 5. Scheduling contexts and `SchedControl` — time as a capability (verdicts 3, 4) **Options.** A timeslice on the thread (classic L4, Zircon) makes creating a thread *acquiring CPU*, with nothing to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are -**subsumed**: a partition is what a `Core` holder builds from several contexts. **Verdict sought:** +**subsumed**: a partition is what a `SchedControl` holder builds from several contexts. **Verdict sought:** contexts as objects *(seL4 MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", EuroSys 2018)*. -A **SchedContext** holds budget `B`, period `T` (`B ≤ T`), at most eight refills, its core, its placement -class, its home thread and its holder. A thread with a bound context is **active**; one without is -**passive** and runs only on donated time (§7). Its *memory* is charged to a Pool; its *time* comes from -a `Core`. One accounting domain for both (RFC-0005 (proposed)'s open question) is left to the resource RFC. - -- **`Core`, one per core, minted to the root task** (RFC-0007 (proposed)). `sc_configure` through *c*'s - `Core` binds a context to *c*: the authority to sell time on *c* and nothing else. `READ` yields - topology — package and SMT siblings, via RFC-0005 (proposed)'s discovery. No syscall names a core by - number. *(seL4 `SchedControl`; research/0002's "core-set capability".)* `B` below **`MIN_BUDGET`** — - twice the measured worst-case kernel path, seL4 MCS's rule (`include/kernel/sporadic.h`) — is refused. +A **SchedContext** holds budget `B` and period `T` (`B ≤ T`), at most eight refills, its core and its +bound thread. A thread with a bound context is **active**; one without is **passive** and runs only on +donated time (§7). **The ABI unit is the nanosecond** (`u64`) for `B`, `T` and `min_budget`, converted to +counter ticks once, in `sc_configure` and at endpoint creation — budgets round down, periods up — so no +per-machine frequency (62.5 MHz on QEMU's `cortex-a72`, a TSC rate on x86_64) enters the ABI. + +- **`SchedControl`, one per core, minted to the root task** (RFC-0007 (proposed) lists them in BootInfo), + has exactly two methods. `sc_configure` (`WRITE`) binds a context to that core with its parameters — + moving a context between cores is `sc_configure` through another core's `SchedControl`, and the first + context configured onto a parked core starts it (§12). `ctl_info` (`READ`) returns the number of contexts + configured on the core and the object's reference count, so a recipient promised exclusivity can check + it (§12). Topology is BootInfo *data* from RFC-0005 (proposed)'s discovery, not a method; interrupt + routing is not a `SchedControl` method (§11). No syscall names a core by number. *(seL4 `SchedControl`; + research/0002's "core-set capability".)* +- **`MIN_BUDGET`:** `B` below twice the published worst-case kernel path is refused — seL4 MCS's rule, + `2u * getKernelWcetTicks() * CONFIG_KERNEL_WCET_SCALE` (`include/kernel/sporadic.h`), without the scale. + The path is measured *between preemption points*, which is why RFC-0007 (proposed)'s console write must + take them (§11); a 44 ms console hold would otherwise make `MIN_BUDGET` 88 ms. - **Full contexts** (`B = T`) are never throttled and round-robin with timeslice `T`. **Partial contexts** - (`B < T`) are sporadic servers: within any window of length `T` the holder consumes at most `B`. - Refills fall due a period after consumption; when all eight are in use they merge, forfeiting - bandwidth rather than exceeding it. *(Sprunt, Sha & Lehoczky 1989.)* -- **What a budget means** (research/0002: minima, not caps; burst and redistribution stated). The kernel - enforces the **cap**; dispatch is **work-conserving**; the **minimum** follows from admission, the `Core` - holder's policy. Unused budget never carries past its window. -- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21, research/0002): - the window is each context's own sliding period; billing is exact, from the counter (`CNTVCT_EL0`; the - TSC) read at every entry and exit — 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 62.5 MHz - default (flag set in QEMU `target/arm/tcg/cpu64.c`, applied in `target/arm/cpu.c`). Kernel time, - interrupts included, is billed to the context running on entry. Budgets round down, periods up. + (`B < T`) are sporadic servers: within any window of length `T` the context consumes at most `B`. Refills + fall due a period after consumption; when all eight are in use they merge, forfeiting bandwidth rather + than exceeding it. *(Sprunt, Sha & Lehoczky 1989.)* +- **What a budget means — caps, not minima (verdict 4).** research/0002 Part 6 asks for guaranteed + reservations with capping as broker policy. This kernel enforces the **cap**, because a sporadic cap is + what makes fixed-priority admission sound: a holder can prove a minimum only if every higher-priority + context is capped. The **minimum** follows from admission, the `SchedControl` holder's policy. Dispatch + is work-conserving *only among threads with released budget*: a partial context idles its core by + design when its budget is spent. Unused budget never carries past its window; there is no burst. +- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21): the window + is each context's own sliding period; billing is exact, from the counter (`CNTVCT_EL0`; the TSC) read at + every entry and exit — 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 62.5 MHz default (QEMU + `target/arm/tcg/cpu64.c`, `target/arm/cpu.c`). Kernel time, **interrupts included, is billed to the + context running on entry** — a cross-domain leak costed in §16 *(as seL4)*. - **Stall counters from the first version** (research/0002 Part 6, after PSI and oomd): per context, time consumed, time and count throttled, time ready but waiting, and budget expiries, read-only via `sc_stats` — the O-17 supervisor's wedged-server signal. The kernel measures; userspace acts. +- **Lifecycle.** Destroying or reconfiguring a context (a) preempts the thread running on it — an IPI on + SMP; (b) clears every link to it — the thread's `sc` or `home_sc`, a reply object's record, its + release-queue entry — before its slot's count can reach zero; and (c) leaves affected threads + `Unfunded`. A reply to a reply object whose recorded context has gone moves nothing. +- **Accounting domain — decided: separate for Phase 1.** A context's *memory* is a slot charged to a Pool + (RFC-0005); its *time* comes only from a `SchedControl`. One domain holding both, as research/0002 + wants, is deferred to the broker RFC; the cost is a broker reasoning over two ledgers. This answers + RFC-0005 (proposed) open question 2. -## 6. Priorities and dispatch (V4) +## 6. Priorities and dispatch (verdicts 5, 6) **256 levels**, 255 most urgent, meaningful within a core. `set_priority(t, auth, p)` needs `WRITE` on -`t`, `READ` on `auth` and `p ≤ mcp(auth)`; MCP is set the same way, so authority over urgency only -narrows as it is delegated — O-2's shape. The root task's thread starts with MCP 255 and a full context -on core 0, the one bootstrap act; the kernel assigns no other priority. *(seL4 `TCB_SetPriority`.)* - -**Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all right in embryo, one bit whose meaning -grows with every scheduling operation (the CAP_SYS_ADMIN grave). The MCP is a monotone ceiling, a relation -between two capabilities, and needs no bit. **Rejected: dynamic priorities** (CFS, BSD decay, boosts): -kernel-resident policies churned in every kernel that carried them while accounting and enforcement -endured (research/0002 Part 1, "Mechanism/policy split"). +`t`, any capability to the Thread `auth`, and `p ≤ mcp(auth)`; MCP is set the same way, so authority over +urgency only narrows as it is delegated — O-2's shape *(seL4 `TCB_SetPriority`, whose authority is any +TCB capability)*. **Every thread is born at priority 0 and MCP 0**, so a fresh thread is no authority; +the root task's thread starts at priority 255 and MCP 255 with a full context on core 0, the one +bootstrap act (RFC-0007 (proposed) §9). To delegate priority authority, derive a **rightless** capability +to a thread with the MCP wanted — it confers the ceiling and nothing else. + +**Rejected: `READ` or `WRITE` on `auth`** — each would make priority authority and register access +(`read_regs`, `write_regs`) the same token. **Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all +right in embryo, one bit whose meaning grows with every scheduling operation (the CAP_SYS_ADMIN grave). +**Rejected: dynamic priorities** (CFS, BSD decay, boosts): kernel-resident policies churned while +accounting and enforcement endured (research/0002 Part 1). **The cost:** any capability to a high-MCP +thread is priority authority, so such threads' capabilities must be handed out with care. - **Run queues.** Per core, 256 intrusive FIFO lists and a 256-bit bitmap: the next thread is a count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; one whose timeslice ended, at the tail. The running thread is never queued, and a thread switched to directly is never enqueued at all. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* -- **Release queue.** Per core, sorted by next refill: O(n) in contexts only the `Core` holder can add. -- **Idle is a kernel loop, not a thread**, billing nobody, and the lost wake-up is designed out. AArch64 - runs `wfi` with `PSTATE.I` set — a pending interrupt still wakes it — then unmasks briefly; Linux warns - masking at `ICC_PMR_EL1` would *not* wake the core (`arch/arm64/kernel/idle.c`), so this kernel masks - with DAIF only. x86_64 uses `sti; hlt`, whose STI shadow issues the `hlt` before delivery. +- **Release queue.** Per core, sorted by next refill: O(n) in contexts only the `SchedControl` holder adds. - **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has released budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority is at least the highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also strictly more urgent than *C*. Otherwise *W* is queued. A *W* without released budget goes to the release queue; one on another core is enqueued there and sent an IPI (§12). -- **Endpoint queues are FIFO** — a refinement of RFC-0004, which fixed no order: O(1) insertion keeps the - hottest path bounded with interrupts masked. *(seL4's current MCS `tcbEPAppend` is FIFO.)* **Rejected: - priority order** *(QNX)* — an O(n) walk over a thread count any process can grow from its own Pool. - -## 7. Donation, inheritance and budget expiry (V5) - -**Donation.** When *C* `call`s and the receiver *S* is passive, the kernel moves *C*'s current context to -*S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves it back. -*S* runs at `max(prio(S), eff(C))`: the constitution's priority inheritance along the call chain, with -*S*'s own priority a **ceiling floor** that policy can raise to the highest client priority for the -priority-ceiling bound seL4 MCS relies on. Lending priority is safe because time is lent with it. A `call` -to an **active** receiver donates nothing. Chains compose one hop at a time. *(Ford & Lepreau, migrating -threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* - -**Rejected:** a passive server at its own priority only (seL4 MCS), leaving inversion to a global -convention; and boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess -which thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. - -**The 2026-08-01 amendment, discharged**, following seL4 RFC-14 (Mitchell Johnston, "Budget limit -thresholds on endpoints for SC Donation", proposed 2023-08-15; seL4/rfcs pull request 24, opened -2024-06-17, still open; read in full for this RFC): - -- **Prevention — the threshold.** An endpoint carries a `threshold`, fixed at creation (0 means none; - RFC-0007 (proposed) encodes it, as `min_budget`). A `call` needs released budget of at least `threshold` - plus twice the kernel's worst-case path, paying for the `call` and `reply` themselves — RFC-14's rule. - A context whose `B` can never meet that fails with `BudgetRefused`; a non-donating `send` to a - thresholded endpoint is invalid. With `threshold` at the server's WCET, expiry inside it is a true error. -- **A flagged deviation.** The amendment says donation below the threshold is "refused up front". RFC-14, - which it cites, refuses only a context that can *never* pass; one that cannot pass *now* has its refills - merged and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in an - event kernel, O(refills) ≤ 8. Proposed: RFC-14's behaviour. No donation happens below the threshold, so - the amendment's purpose holds; refusing a transient shortfall would make every client spin to retry. If - accepted, it is logged against RFC-0004 as a dated amendment. +- **Endpoint queues are priority-ordered, first-come within a level** — seL4 MCS's `tcbAppend` + (`include/object/tcb.h`, "priority ordered endpoint or notification queue"), which walks back from the + tail; the FIFO `tcbEPAppend` is compiled only without MCS. The walk is O(n) with interrupts masked, *n* + bounded by the thread array's fixed capacity (RFC-0005 §5) and counted in the published worst path; a + `set_priority` on a queued thread repositions it. **Rejected: FIFO** *(classic L4, non-MCS seL4)*: O(1), + but a process could create threads from its own Pool and queue *N* calls ahead of a more urgent client + of a shared server — unbounded inversion the server cannot reorder. + +## 7. Donation, inheritance and budget expiry (verdicts 7, 8) + +**Donation.** When *C* `call`s and the receiver *S* is **passive**, the kernel moves *C*'s current +context to *S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves +it back. *S* runs at `max(prio(S), eff(C))`, **fixed when the donation is made** and not propagated if +*C*'s priority later changes — O(1), one hop at a time. Lending priority is safe because time is lent with +it. *(Ford & Lepreau, migrating threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* + +**An active receiver receives no donation (amendment A1, §13).** It runs on its own context at its own +priority, as in seL4 MCS (`reply_push`, `src/object/reply.c`, donates only to a receiver with no context): +a thread cannot hold two contexts, and an active server has chosen to pay for itself. RFC-0004 §4 and §8 +say `call` lends unconditionally, so this is an amendment. **The cost:** an active server gets no +inheritance, and a low-priority active server blocks high-priority clients. Constitution §3's +inheritance therefore holds for passive servers; **a server shared across trust domains must run passive, +or active at a priority no lower than its most urgent client** — the ceiling, now required, not optional. + +**Priority inversion, bounded.** Priority-ordered queues serve the most urgent waiter next; a waiter +still waits for the request in service, which runs on its client's context and can be preempted by a +middle-priority thread. `prio(S)` is therefore a **ceiling floor**: set to the highest client priority, +it bounds inversion to one request *(the priority-ceiling bound seL4 MCS relies on)*. **Rejected:** +boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess which thread +serves an endpoint — policy — and walk unbounded chains with interrupts masked. + +**Caller death never unfunds a server.** If *C* exits or is killed while *S* serves it, *R* is destroyed +but the context stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails +`PeerGone`); the context then detaches, `Unfunded` for no thread, and only its capability holder may +rebind or destroy it. Only a context's holder can pull time from a server, never a client's +`thread_exit` *(seL4 `reply_remove_tcb` likewise leaves the context with the server)*. + +The 2026-08-01 amendment, discharged as amended, after seL4 RFC-14 (Mitchell Johnston, "Budget limit +thresholds on endpoints for SC Donation", seL4/rfcs pull request 24, open): + +- **Prevention — `min_budget`.** An endpoint carries `min_budget`, fixed at creation (0 means none; + RFC-0007 (proposed) encodes it). A `call` needs released budget of at least `min_budget + MIN_BUDGET`, + the margin paying for the `call` and `reply` themselves — **this RFC's rule**; RFC-14's exact comparison + is to be quoted before acceptance. A context whose `B` can never meet it fails `BudgetRefused`; a + non-donating `send` to such an endpoint is invalid. With `min_budget` at the server's WCET, expiry + inside it is a true error. +- **A caller short *now* waits (amendment A2).** The amendment says donation below the threshold is + "refused up front". Proposed instead: a partial context that cannot pass now has its refills merged + and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in an event + kernel, O(refills) ≤ 8, and the deferral the RFC-14 discussion describes. A **full** context short of + its slice ends its timeslice: tail of its level, a fresh slice, the call restarts. No donation happens + below the bar, so the amendment's purpose holds; refusing a transient shortfall would make every client + spin to retry. - **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* runs on it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with `BudgetExpired` (observed when the context next refills); *R* is consumed, so a late `reply` fails - closed. *S* is left runnable but unfunded, registers intact, and the context's expiry counter rises. If - *S*'s `timeout_ep` resolves, the kernel raises a **timeout fault** carrying *S*'s identity and consumed - time; the handler, active with time of its own, abandons the request (`write_regs`) or lends *S* a - context to finish. With no handler, *S* stays stranded, contained to its own clients. *(seL4 MCS: - `tcbTimeoutHandler`, `seL4_Fault_Timeout`.)* + closed. *S* is left `Unfunded`, registers intact, and the expiry counter rises. If `timeout_ep` holds a + live endpoint, the kernel sends a **timeout fault** carrying consumed time and donating nothing (seL4 + MCS sends timeout faults without donation); the handler, active with time of its own, abandons the + request (`write_regs`) or lends *S* a context to finish. The message carries **no identity**: badges + are reserved zero until RFC-0003a, so a handler binds one timeout endpoint per server. - **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply - to A with an error; unwinding to A would walk the reply stack — the O(n) walk RFC-14 also rejects. This - kernel-initiated completion is the one the amendment left to this RFC: a status, no new object. -- **A reconciliation.** The amendment says seL4's timeout exceptions "have remained future work"; they - ship in MCS, and what is unmerged is RFC-14's threshold. The phrase wants a dated correction only. + to A with an error; unwinding to A would walk the reply stack — O(n) with interrupts masked. +- **With `min_budget` 0 there is no prevention:** one client can strand a shared server for every client + until a handler funds it (O-26, A5). Zero is for servers with one client or a supervisor. **Rejected: the unwind without timeout faults**, where the stranded server is visible only through its -state and counters. Timeout faults are no second protocol — they travel RFC-0007 (proposed)'s fault path -— and a supervisor woken by a message beats one polling every passive server; the no-handler case *is* -that unwind. **Deferred: RFC-14's budget limits** (capping what a server spends of a donated context): -RFC-14 measured 22% extra fastpath cost on `call` and 21% on `reply_recv`, against 13% for thresholds. +state and counters; a supervisor woken by a message beats one polling. **Deferred: RFC-14's budget +limits** (capping what a server spends of a donated context): the RFC-14 discussion reports 22% extra +fastpath cost; the per-operation figures are to be re-checked before this is revisited (Open question 3). -## 8. The timer and preemption (V6) +## 8. The timer and preemption (verdict 9) One HAL surface: `now() -> Ticks`, `frequency() -> Hz`, `set_deadline(Ticks)`, `cancel()`, and a `timer_fired()` up-call. The kernel is **tickless**: each exit programs one per-core deadline — the earliest of a partial context's exhaustion, the timeslice boundary if an equal-priority thread is ready, -and the next refill due. With no competitor (§12's exclusive core) the timer is cancelled. +and the next refill due. A lone *full* context arms nothing; a lone partial one still arms its exhaustion. - **AArch64: the EL1 virtual timer** — `CNTV_CVAL_EL0`, `CNTV_CTL_EL0` (`ENABLE`, `IMASK`, `ISTATUS`), `CNTVCT_EL0`, and `CNTFRQ_EL0` always read, never assumed. PPI INTID 27, the Arm BSA assignment QEMU's `virt` uses (`include/hw/arm/bsa.h`). Preferred to the physical timer, whose EL1 access an EL2 beneath - us can trap (`CNTHCTL_EL2`); the EL2-to-EL1 descent `boot.s` owes must grant EL1 timer access and zero - `CNTVOFF_EL2`. *(Arm ARM DDI 0487, "The Generic Timer".)* -- **x86_64: the local APIC timer**, x2APIC where offered (`IA32_APIC_BASE` bit 10). TSC-deadline mode (LVT - mode `10b` in bits 18:17; `IA32_TSC_DEADLINE`, MSR `0x6E0`) only when `CPUID.01H:ECX[24]` **and** - invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; otherwise one-shot through the initial-count - register, calibrated at boot, behind the same `set_deadline`. QEMU's TCG lacks TSC-deadline - (`target/i386/cpu.c` lists it as missing), so one-shot is what the emulator runs, not dead code. The - console prints the mode chosen. *(Intel SDM Vol. 3A, the APIC chapter.)* -- **Kernel-reserved.** The timer line is never minted (§11). EL0 `wfi` traps (`SCTLR_EL1.nTWI = 0`) and - completes as `yield`; `hlt` at CPL 3 faults. EL0 `wfe` does **not** trap (`nTWE = 1`): spin-lock backoff - uses it, trapping would make every spin a kernel entry, and the armed timer's interrupt wakes it anyway. - RFC-0005 (proposed) writes `SCTLR_EL1` and must carry these two bits. + us can trap (`CNTHCTL_EL2`); any EL2-to-EL1 descent must grant EL1 timer access and zero `CNTVOFF_EL2`. + *(Arm ARM DDI 0487, "The Generic Timer".)* +- **x86_64: the local APIC timer**, x2APIC where `CPUID.01H:ECX[21]` reports it, enabled by + `IA32_APIC_BASE.EXTD` (bit 10). TSC-deadline mode (LVT mode `10b` in bits 18:17; `IA32_TSC_DEADLINE`, + MSR `0x6E0`) only when `CPUID.01H:ECX[24]` **and** invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; + otherwise one-shot through the initial-count register, calibrated at boot, a past or zero deadline + clamped to a count of 1 since 0 stops the timer. QEMU's TCG lacks TSC-deadline (`target/i386/cpu.c`), so + one-shot is what the emulator runs, not dead code. The console prints the mode. *(Intel SDM Vol. 3A.)* +- **Kernel-reserved; this RFC owns these bits.** `SCTLR_EL1.nTWI` (bit 16) `= 0`: EL0 `wfi` traps (EC + `0x01`) and completes as `yield`, advancing the saved PC by 4, since `ELR_EL1` holds the `wfi` itself. + `nTWE` (bit 18) `= 1`: EL0 `wfe` does not trap, so spin-lock backoff costs no kernel entry. `UMA` (bit 9) + `= 0`; `SA` (bit 3) `= 1`. `CNTKCTL_EL1` and `PMUSERENR_EL0` are written 0 at boot — no EL0 counter, + timer or PMU — because their reset values are not architecturally fixed. `hlt` at CPL 3 faults; `CR4.TSD` + is set. RFC-0005 (proposed) writes `SCTLR_EL1` whole and must carry these values; its §13 step 5 today + says `nTWE = 0` and needs correcting. +- **Userspace timers (RFC-0004 §9.3), named now, built later.** RFC-0004 §8 calls the watchdog + load-bearing, and the timer is kernel-reserved, so the kernel must provide the source: a **deadline + object**, bound to a notification, armed with an absolute time in nanoseconds, kept in the per-core + release queue beside refills, charged to a Pool, and signalled on expiry *(Zircon's timer object)*. One + armed deadline per object; the sorted insert is bounded by the deadline array's capacity. **Rejected:** + minting a second hardware timer — the EL1 physical timer (PPI 30) exists per core on AArch64, but x86_64 + has no second per-core timer without the HPET, so the source would differ per Tier-1 target. ## 9. The context switch — what is saved, where In an event kernel a context switch is *which frame the exit path restores*; assembly is confined to the entry and exit stubs in `kernel/src/arch/**`. **The user frame lives in the TCB.** On AArch64, while a thread runs at EL0, `SP_EL1` points at the end of its TCB frame, so the entry stub's stores land there -before it loads the per-core kernel stack from the block `TPIDR_EL1` addresses. On x86_64 an interrupt or -exception from ring 3 loads `RSP` from `TSS.RSP0`, pointing at the TCB frame — but **`syscall` does not -switch `RSP`**: its stub must `swapgs`, park the user `RSP` and load the frame pointer itself. RFC-0007 -(proposed) owns both entry sequences; this RFC owns the layout. The address-space switch (`TTBR0_EL1` with -ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit when the process differs — the single point -where a time-protection flush would go (Ge et al., EuroSys 2019; research/0002 Part 6). +before it loads the per-core kernel stack from the block `TPIDR_EL1` addresses; `SCTLR_EL1.SA` is on and +the frame 16-byte aligned, and the current-EL `SP_ELx` synchronous vector reports a frame overwrite if +`SP` lies inside a TCB. On x86_64 an interrupt from ring 3 loads `RSP` from `TSS.RSP0`, pointing at the +TCB frame; **`syscall` does not switch `RSP`**, so its stub runs `swapgs`, parks the user `rsp` in per-CPU +scratch, loads the TCB frame pointer, saves into the frame, and only then loads the kernel stack. RFC-0007 +(proposed) owns both entry sequences; this RFC owns the layout and the current-EL IRQ stub (§3). The +address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit when +the process differs — where a time-protection flush would go (Ge et al., EuroSys 2019). | What | AArch64 | x86_64 | |------|---------|--------| | Every entry, into the TCB | `x0`–`x30`, `SP_EL0`, `ELR_EL1`, `SPSR_EL1`: 34 words | `SS`, `RSP`, `RFLAGS`, `CS`, `RIP`, an error-or-vector slot, 15 GPRs: 21 words | -| Thread-local bases, per switch | `TPIDR_EL0` (EL0-writable) saved and restored; `TPIDRRO_EL0` restored | `FS_BASE` and user GS base restored; `CR4.FSGSBASE` off, so only the kernel sets them | +| Thread-local bases, per switch: 2 more words | `TPIDR_EL0` (EL0-writable) saved and restored; `TPIDRRO_EL0` restored | `FS_BASE` and user GS base restored; `CR4.FSGSBASE` off, so only the kernel sets them | | Also per switch | `CPACR_EL1.FPEN` (§10) | `CR0.TS` (§10); `TSS.RSP0` | | Lazily (§10) | `V0`–`V31`, `FPCR`, `FPSR`: 520 bytes, 16-byte aligned | XSAVE area, `XCR0` = x87, SSE, AVX: 832 bytes, 64-byte aligned | -| Never | kernel registers, kernel FP (soft-float), debug and PMU state | the same | -| **TCB size** | 36-word frame (288 B) + FP area padded to 528 B + ~150 B of fields: **1 KiB** | 23-word frame (184 B, padded to 192) + 832 B XSAVE + ~150 B: **1.25 KiB** | +| Never | kernel registers, debug and PMU state | the same | +| **TCB size** | 36-word frame (288 B) + FP area padded to 528 B + ~180 B of fields: **1 KiB** | 23-word frame (184 B, padded to 192) + 832 B XSAVE + ~180 B: **1.25 KiB** | - **O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` (EL0t, DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only - status flags (IOPL 0, IF set), and a non-canonical `RIP` or base is refused. `SCTLR_EL1.UMA = 0`. + status flags (IOPL 0, IF set), and a non-canonical `RIP` or base is refused. It writes `TPIDR_EL0` and + `FS_BASE` only: `TPIDRRO_EL0` and the user GS base are kernel-owned IPC-buffer discovery (RFC-0007 §5). - NMI, `#DB`, `#DF` and `#MC` run on IST stacks, as RFC-0007 (proposed) asks: they can arrive in the `syscall` window before `RSP` is switched. AArch64 has no such window. -## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (V7) - -- **Enabled per thread** by `set_fp`, off by default; the kernel is soft-float, so this state is purely - user state. A thread with FP off that touches FP takes an ordinary fault, so servers never force a save. -- **Loaded on first use, saved lazily.** Each core records an FP *owner*; every other thread runs with - access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or `CR0.TS = 1` - (`#NM`, vector 7). On that trap, for an FP-enabled thread, the kernel saves the owner's registers, - loads the new owner's (zeroed on first use), grants access (`FPEN = 0b11`; `clts`) and retries. A `call` - to an FP-off server saves nothing. State is saved eagerly before a context moves core. -- **AArch64's asymmetry, stated.** No `FPEN` value traps EL1 while allowing EL0 (`0b01` is the reverse), - so while an owner runs a stray kernel FP instruction would not trap; the soft-float target is the guarantee. +## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (verdict 10) + +- **Enabled per thread** by `set_fp`, off by default. `CLAUDE.md` says "per process"; per thread is finer + so an FP-off server thread in an FP-using process never forces a save — the § Architectures wording + changes with this RFC. A thread with FP off that touches FP takes an ordinary fault. +- **Loaded on first use, saved lazily.** Each core records an FP *owner* as an `(index, generation)` pair; + every other thread runs with access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter + already names) or `CR0.TS = 1` (`#NM`, vector 7). `FPEN = 0b00` traps EL1 too and `XSAVE` raises `#NM` + while `TS` is set, so the trap handler runs: `FPEN = 0b01` (EL1 permitted, EL0 still trapped) and `isb`, + or `clts`; save the owner's registers if the owner is live; load the new owner's, or zeros on first use; + `FPEN = 0b11`; `isb`; return. A `call` to an FP-off server saves nothing. +- **The owner is cleared** — its registers unsaved — when the owner thread is destroyed or `set_fp(off)`, + so a thread reusing its slot finds a stale generation and loads zeros, never a dead thread's registers. +- **The kernel's only FP instructions** are those save and restore stubs in `kernel/src/arch/**`, + executed only with access granted; the soft-float target is the guarantee for generated code. No `FPEN` + value traps EL1 while allowing EL0, so while an owner runs a stray kernel FP instruction would not trap. - **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format area (512 legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. - **The known cost — LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a non-owner can read the owner's registers speculatively before `#NM` resolves. Hardware side channels are out of scope (threat model §7); the cure is Open question 2, costed in §16. -## 11. Interrupts — the controller, and device IRQs as notifications (V8) +## 11. Interrupts — the controller, and device IRQs as notifications (verdict 11) -The kernel owns the controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`; Redox; QNX.)* +The kernel owns the controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`.)* - **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. Kernel-reserved lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. - `bind(irq, notification, bits)` needs `WRITE` on both; `ack(irq)` needs `WRITE`. Routing a line to core - *c* needs the `IrqHandler` and *c*'s `Core`. + `bind(irq, notification, bits)` invokes the `IrqHandler` (`WRITE`) and presents the notification + (`WRITE`); `ack(irq)` needs `WRITE`. **Phase 1 routes every line to the boot core**; the SMP increment + decides routing as an `IrqControl` concern, never a `SchedControl` method. Shared PCI INTx lines on + `q35` are **unsupported** until MSI — a Phase-2 cost on the second Tier-1 target. - **Delivery.** The kernel acknowledges, **masks the line**, signals end of interrupt, ORs `bits` into the notification and wakes a waiter through §6's dispatcher; the driver's `ack` unmasks. A level-triggered - line cannot storm. A destroyed notification leaves the line masked (fail closed); the `IrqHandler` - outlives a crashed driver, and a restarted one re-binds (O-17). -- **AArch64: GICv3 first.** System-register CPU interface (`ICC_SRE_EL1.SRE`, `ICC_IGRPEN1_EL1`; acknowledge - `ICC_IAR1_EL1`; end `ICC_EOIR1_EL1`; `ICC_PMR_EL1` open); SPIs through `GICD_ISENABLER`/`ICENABLER` - and `GICD_IROUTER`; SGIs and PPIs through the redistributor's `GICR_ISENABLER0`, woken by - `GICR_WAKER`. Each line the kernel enables is put in Group 1 explicitly, since Group 0 would arrive as - FIQ. On `virt`: distributor `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`). - Chosen for affinity routing past eight cores and an interface needing no MMIO. *(Arm IHI 0069.)* **Cost:** - `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), so xtask gains `gic-version=3`; - the Raspberry Pi 4 in Constitution §6's bring-up order has a GIC-400 (GICv2) — a **known** second driver. + line cannot storm, but can fire once per `ack` (§16). A destroyed notification leaves the line masked + (fail closed); the `IrqHandler` outlives a crashed driver, and a restarted one re-binds (O-17). +- **AArch64: GICv3 first.** Distributor: `GICD_CTLR` with `ARE` and `EnableGrp1` (`virt` without + `secure=on` has one security state, `DS = 1`); SPIs through `GICD_ISENABLER`/`ICENABLER`, + `GICD_IPRIORITYR` and `GICD_IROUTER`. Redistributor: woken by `GICR_WAKER`; SGIs and PPIs through + `GICR_ISENABLER0` and `GICR_IPRIORITYR`. Every enabled line is put in Group 1 explicitly (Group 0 would + arrive as FIQ) at a priority numerically below the `ICC_PMR_EL1` mask. CPU interface: `ICC_SRE_EL1.SRE`, + `ICC_IGRPEN1_EL1`, acknowledge `ICC_IAR1_EL1`, end `ICC_EOIR1_EL1`. On `virt`: distributor + `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`), reached through RFC-0005 + (proposed)'s MMIO window. *(Arm IHI 0069.)* **Cost:** `virt` picks GICv2 at eight CPUs or fewer + (`finalize_gic_version_do`), so xtask gains `gic-version=3`; the Raspberry Pi 4 in Constitution §6's + bring-up order has a GIC-400 (GICv2) — a **known** second driver. - **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask bit; end of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** without IOMMU interrupt remapping a device can aim any vector at any core — O-18's precondition again. -- **Latency.** Interrupts are taken at EL0, in idle or at a preemption point, so latency is §3's worst - path — which includes RFC-0007 (proposed)'s kernel console write until a userspace UART driver exists. +- **Latency** is §3's worst path between preemption points. RFC-0007 (proposed)'s console write must take + one per FIFO-full poll, restartable by byte count; until it does, it holds the kernel ~44 ms per 504 + bytes on real hardware (none on QEMU, which has no baud delay). -## 12. Multi-core shape, placement and exclusive cores (V9) +## 12. Multi-core shape, placement and exclusive cores (verdict 11) Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. -- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds; moving - a context is `sc_configure` through another `Core`. **The kernel never load-balances.** +- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds, so + donation can move a server thread between cores. **The kernel never load-balances.** - **Cross-core wake-ups** (RFC-0004 §9.2): enqueue *W* on its core and send an IPI — an SGI through `ICC_SGI1R_EL1`; a fixed vector through the LAPIC ICR. Direct switch is same-core only. -- **One big kernel lock when SMP lands**, taken on entry *(Peters, Danis, Elphinstone & Heiser, "For a - Microkernel, a Big Lock Is Fine", APSys 2015)*. **This answers RFC-0003 §14.2:** a table is touched only - under the lock, so the resolve→check→act window its 2026-07-30 amendment makes a correctness dependency - holds across the whole entry, with no per-table lock on the fast path. -- **Cores are parked until funded (default-deny).** Secondary cores stay in `boot.s`'s `wfe` loop until a - context is configured onto them through their `Core`; a core nobody funds never runs anything. -- **SMT placement invariant** (research/0002 Part 6, after L1TF and MDS). Contexts and threads carry a - placement class set by policy. `sc_configure` refuses a class that differs from any context on the - funded SMT sibling; `bind_sc` and **donation** refuse a thread of another class — the class travels with - the context, so a passive server serving two classes needs a thread per class. O(1) at bind and call; - no gang scheduling. SMT-off is "never fund the sibling"; one class is the vacuous default. +- **One big kernel lock when SMP lands**, taken on every entry, the IPC fast path included *(Peters, + Danis, Elphinstone & Heiser, "For a Microkernel, a Big Lock Is Fine", APSys 2015)*. This **rejects the + premise** of RFC-0003 §14.2, which asks for table sharing without a global lock on the fast path; within + one entry the resolve→check→act window holds under the lock (§3 covers restarts). A dated note against + RFC-0003 records it (§13). +- **A core waiting for the lock services IPIs while it spins** *(as seL4's CLH lock does)*, because two + operations need a synchronous answer from another core: RFC-0005 (proposed)'s x86_64 TLB shootdown (it + has no broadcast invalidate) and **FP state live on another core** — when a thread whose registers are + owned lazily on core X next needs them on core Y, Y asks X by IPI to save them. Cost: a cross-core FP + handoff is an IPI round trip. +- **Cores are parked until funded (default-deny).** On QEMU `virt` with `-kernel`, the DTB advertises PSCI + and secondaries start powered off (checked on QEMU 8.2.2: CPU 1 never leaves the entry address), so + `boot.s`'s `wfe` park loop only matters on spin-table firmware. The first `sc_configure` onto a core + starts it — PSCI `CPU_ON`, conduit from the DTB's `/psci` node; INIT-SIPI-SIPI on x86_64. Deferred to + increment 14; `boot.s`'s comment changes with it. +- **SMT placement invariant — one sentence now** (research/0002 Part 6, after L1TF and MDS): a context + will carry a placement class set only by the `SchedControl` holder; SMT siblings never run different + classes and donation never crosses one. The field is reserved zero in `sc_configure` until increment 14, + so no thread can label itself; SMT-off is "never fund the sibling". - **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one thread, a full context, nothing else on the core. The bitmap has one bit, the timer is never armed, no - IPI arrives, and device lines are routed elsewhere. No flag: only a `Core` holder can bind another - context, and one promising exclusivity to a third party transfers the capability, keeping no copy. + IPI arrives unless the `SchedControl` holder acts, and lines are routed elsewhere; counters advance only + at the next kernel entry. A recipient checks the promise with `ctl_info`: one context, reference count 1. + +## 13. Amendments this RFC proposes to accepted documents (verdict 12) + +Each, if accepted, is logged in `docs/CHANGELOG.md` under Changed and appended to the target as a dated +amendment — never a silent edit. -## 13. Lineage +| # | Target | Change | Why | +|---|--------|--------|-----| +| A1 | RFC-0004 §4, §8 | `call` donates only to a **passive** receiver; an active one runs on its own context | a thread holds one context; seL4 MCS `reply_push`; cost in §7 | +| A2 | RFC-0004 amendment 2026-08-01 | a caller short *now* waits `Throttled`; only a context that can *never* pass is refused | refusal makes clients spin; no donation happens below the bar either way | +| A3 | RFC-0004 amendment 2026-08-01; research/0002 Part 6 | seL4 timeout faults ship in MCS (`seL4_Fault_Timeout`); what is unmerged is RFC-14's threshold | a factual correction | +| A4 | RFC-0003 §14.2 | the big lock is taken on the fast path by design; the question is answered by rejecting its premise | §12; Peters et al. 2015 | + +## 14. Lineage | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl` as `Core`; MCP; sporadic refills; `MIN_BUDGET`; timeout faults; `IRQHandler` | own-priority-only passive servers; handlers cached in the TCB | -| **seL4 RFC-14** (Johnston) | the endpoint threshold, its 2 × WCET rule, deferral, one-layer return | budget limits, for now | +| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl`; MCP with any-capability authority; sporadic refills; `MIN_BUDGET`; timeout faults; handlers held in the TCB; priority-ordered endpoint queues; no donation to active receivers | the WCET scale factor; ceiling left wholly to convention | +| **seL4 RFC-14** (Johnston) | the endpoint threshold, deferral, one-layer return | budget limits, for now | | **Heiser & Elphinstone 2016; Fluke 1999** | the event kernel | per-thread kernel stacks | | **Elphinstone & Heiser 2013** | the running thread unqueued; priority-aware direct switch | lazy scheduling; L4's unconditional switch | -| **QNX Neutrino** | priority inheritance across the message; partitions built on budgets | full inheritance to busy servers; priority-ordered receive | +| **QNX Neutrino** | priority inheritance across the message; partitions built on budgets | inheritance to busy servers | | **Ford & Lepreau 1994** | the migrating-thread picture of `call` | Mach | | **Sprunt, Sha & Lehoczky 1989** | the sporadic server | an unbounded replenishment list | +| **Zircon** | the timer object, for deadline objects | per-thread timeslices | | **Jailhouse** | refusing to share: parked cores, exclusive assignment | static-only partitioning | | **Peters et al. 2015; Linux** | one big lock; the idle wake-up rule; PSI-style stall counters | fine-grained locking before measurement; dynamic priorities | -## 14. Obligations +## 15. Obligations | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| -| O-7 no unprivileged exhaustion | time only through contexts; `MIN_BUDGET`; eight refills; O(1) expiry return; FIFO endpoint queues; no wait holding a lock; lines masked until `ack` | **discharged for CPU time and scheduler structures**; kernel memory is RFC-0005's half | -| O-4 no ambient authority | no thread runs without a context; priority through MCP; time through `Core`; interrupts through `IrqControl`; only `yield` and `exit` authority-free | **discharged for time** | -| O-5 argument validation | `write_regs` sanitises `SPSR_EL1`/`RFLAGS`, refuses non-canonical `RIP` and bases; every operation takes handles | **discharged for this surface** | -| O-9 no ambient side channel | no EL0 timer or PMU; SMT placement classes; parked cores | **contributes** — fine-grained timing channels stay out of scope | -| O-17 contained drivers | lines as notifications, masked until `ack`; `IrqHandler` outlives a crashed driver; stall counters | **enables** | +| O-7 no unprivileged exhaustion | time only through contexts; `MIN_BUDGET`; eight refills; O(1) expiry return; bounded queue walks; no wait holding a lock; lines masked until `ack` | **discharged for CPU time and scheduler structures, except interrupt-entry time**, billed to the running context and bounded per `ack`; memory is RFC-0005's half | +| O-4 no ambient authority | no thread runs without a context; priority through MCP, born 0; time through `SchedControl`; interrupts through `IrqControl`; only `yield` and `thread_exit` authority-free | **discharged for time** | +| O-5 argument validation | `write_regs` sanitises `SPSR_EL1`/`RFLAGS`, refuses non-canonical `RIP` and bases, cannot touch discovery registers; restarts re-resolve | **discharged for this surface** | +| O-9 no ambient side channel | no EL0 counter, timer or PMU; FP owner cleared on reuse; parked cores. **Residual:** untrapped `wfe` lets any thread observe another's `sev` — one bit of wake-up, no data | **contributes** — fine-grained timing channels stay out of scope | +| O-17 contained drivers | lines as notifications, masked until `ack`; `IrqHandler` outlives a crashed driver; fault endpoints the thread cannot close; stall counters | **enables**, weakened where a child holds `WRITE` on its own threads (§4) | | O-18 confined DMA | MSI deferred until interrupt remapping is designed | **named, not discharged** | -| RFC-0004 amendment, 2026-08-01 | (1) threshold at `call`, deferral flagged; (2) `BudgetExpired` on the reply object, plus timeout faults (§7) | **both commitments discharged** | - -## 15. Graves checked (§3) - -- **Policy in the kernel.** No admission, balancing or budget choice; the one number chosen is 256. -- **The catch-all right.** Priority authority is a relation (MCP); time is per core; interrupts per line. -- **Bolted-on multicore.** Per-core state, IPIs, the lock and placement are stated before any SMP code. +| RFC-0004 amendment, 2026-08-01 | (1) `min_budget` at `call`, with deferral (A2); (2) `BudgetExpired` on the reply object, plus timeout faults (§7) | **both discharged as amended by A1–A2**; caller death cannot bypass (1) | + +## 16. Graves checked (§3) + +- **Policy in the kernel.** No admission, balancing or budget choice. The constants the kernel does + choose: 256 levels; eight refills; `MIN_BUDGET`'s factor 2; the XCR0 extent; the thread and deadline + array capacities (RFC-0005); and §18's interim constants until measured. +- **The catch-all right.** Priority authority is a relation (MCP); `SchedControl` has one mutating method, + selling time on one core, and a read of its own state; topology is data and routing belongs to + `IrqControl`; interrupts are per line. The set is closed by listing it (§4). +- **Bolted-on multicore.** Per-core state, IPIs, the lock, its IPI-servicing spin, FP handoff, shootdowns + and core bring-up are stated before any SMP code. - **Drivers in the kernel; compiled-in unused device paths.** The kernel masks and signals, never services a device; one controller and one timer per target; unclaimed lines never enabled. - **Baroque hierarchies; multi-copy IPC.** Contexts are flat; donation moves a reference to time. -## 16. Costs — what this makes harder +## 17. Costs — what this makes harder - **Every server needs someone to think about time.** A passive server cannot run until called; an active - one needs a configured context. "Spawn a thread and it runs" is gone — O-4's price, in time. -- **The one-hop unwind leaves a stranded server to userspace.** The kernel makes the failure visible and - restarts nothing: a supervisor is required before any passive server on partial contexts is trusted. + one needs a configured context and gets no inheritance (A1). "Spawn a thread and it runs" is gone. +- **The one-hop unwind leaves a stranded server to userspace**, and `min_budget` 0 lets one client strand + it for all. A supervisor is required before any passive server on partial contexts is trusted. +- **Caps, not minima:** a partial context idles its core when spent, even with the core otherwise idle. +- **Interrupt time is billed to whoever runs:** a driver `ack`ing a level line in a loop charges other + processes' budgets, one interrupt per `ack` — seL4's choice; charging an `IrqHandler`'s bound context + becomes possible with passive drivers (Open question 4). +- **Priority-ordered queues** cost an O(n) walk with interrupts masked, *n* up to the thread capacity. +- **No userspace timer yet:** RFC-0004 §8's load-bearing watchdog has no time source until deadline + objects (increment 15) — an unmet RFC-0004 dependency, named. +- **Exclusivity is checkable, not revocable:** `ctl_info` shows a grantor kept no copy, but a grantor who + did can still configure onto the core. - **MCS is seL4's least-verified part**, and sporadic refills are subtle — hence property tests. - **About 1 KiB per TCB on AArch64 and 1.25 KiB on x86_64**, FP space included for threads that never use it. -- **FIFO endpoint queues** let a high-priority client wait behind earlier low-priority ones. - **Lazy FP's cure, costed.** Eager restore on a switch into an FP-enabled thread of another process moves 520 bytes (16 `ldp q` pairs) or an 832-byte `XRSTOR` — 9 or 13 cache lines — and nothing on IPC to - FP-off servers. Its cycles are a measurement owed, beside a `call` round trip of roughly 380–630 cycles + FP-off servers; its cycles are a measurement owed, beside a `call` round trip of roughly 380–630 cycles (twice research/0001's one-way 188–316). -## 17. Open questions +## 18. Open questions -1. **EL0 counter access.** Proposed default: a per-thread flag set with `WRITE` on the Thread and switched - with it (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`, written only when it differs). Denial costs a web server a - kernel entry per timestamp; threat model §7 puts fine-grained timing out of scope. EL0 timers stay off. -2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §16)? -3. **RFC-14 budget limits.** Worth 21–22% on the fastpath to stop a passive server overspending? -4. **Passive drivers** need a context bound to a notification (MCS); **userspace timers** (RFC-0004 §9.3) - need kernel deadline objects off the release queue, or a time server. Neither is Phase 1. +1. **EL0 counter access.** Proposed: a property of the *context*, set by the `SchedControl` holder in + `sc_configure` and switched with it (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`), default deny — not a thread + flag, which a process could set on itself. Denial costs a kernel entry per timestamp. +2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §17)? +3. **RFC-14 budget limits.** Worth the fastpath cost to stop a passive server overspending? +4. **Passive drivers** need a context bound to a notification (MCS); with them, interrupt entry can be + charged to the driver. 5. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst kernel path; x86_64's clock when the TSC is not invariant. +6. **Interrupt routing on SMP** (§11): `IrqControl` minting per-(line, core) handlers, or a separate + per-core interrupt-target capability — decided with increment 14. -## 18. Implementation increments +## 19. Implementation increments Pure logic lands in a new host-tested `no_std` workspace crate, `scheduler/`, generic over a thread index -as `capability/` is over `ObjectRef`; hardware behaviour is proved by QEMU boot-test strings. New `unsafe` -stays in `kernel/src/arch/**`. The x86_64 HAL gains each signature as a documented no-op that returns — -its entry still calls straight through into the kernel proper (CLAUDE.md § Architectures), so the second -build compiles the scheduler. **[demo]** marks what the Phase-1 demo needs. - -1. **`scheduler` I — run queues, states, MCP. [demo]** O(1) pick, head/tail re-entry, legal transitions, - the MCP relation; host tests and an exhaustive transition table. Built `no_std` for both targets in CI. -2. **`scheduler` II — full contexts and billing. [demo]** Round-robin budgets, rounding, stall counters, - `MIN_BUDGET`; host tests against a shadow model. -3. **GICv3 and the virtual timer, kernel-only. [demo]** xtask's machine line gains `gic-version=3`; the - kernel's own handler prints `timer: tick` twice — the GIC, the PPI and the idle loop, proved. -4. **Event-kernel dispatch. [demo]** Trap frame in the TCB; two kernel self-test threads (EL1t, on - `SP_EL0`, behind a `sched-selftest` feature as `provoke-exception` is) alternate under the timer: - `sched: A B A B`. The boot stack becomes core 0's kernel stack; `aarch64.ld`'s comment changes with it. -5. **EL0 entry and exit** (on RFC-0005's user mappings, RFC-0007's trap stub, both proposed). **[demo]** A - spinning EL0 thread is preempted anywhere and resumes with its registers intact: `user: still here`. -6. **Thread, SchedContext and `Core` objects; `sc_configure`, `bind_sc`, `resume`, `set_priority`, `exit`. - [demo]** The root task configures the demo processes; two EL0 threads at equal priority interleave. -7. **IPC blocking states, FIFO endpoint queues, direct switch** (with RFC-0004's endpoint PR). **[demo]** - Host tests for the switch; the client `call`s an **active** server and RFC-0007's `Kaya!` strings appear. -8. **Passive servers, donation, inheritance.** A passive server runs only when called, at its client's - priority; a demo stretch goal. -9. **`scheduler` III — partial contexts.** Sporadic refills, merge, release queue. Property test: no - window of length `T` sees more than `B` consumed. -10. **Threshold, deferral, expiry return, timeout faults.** Host tests; boot-test `BudgetRefused`, - then `BudgetExpired` and a timeout fault for an overrunning server. Required before partial-context - passive servers are trusted. -11. **Lazy FP/SIMD.** Two FP-enabled EL0 threads load different markers into `v0`, are preempted in turn - and each finds its own: `fp: isolated`. -12. **`IrqControl`/`IrqHandler`.** PL011 receive (SPI 1, INTID 33) reaches an EL0 driver as a notification - when xtask writes a byte to the serial port: `irq: notified`. -13. **x86_64 scheduling HAL** (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`), after its boot RFC: 3–5, 11 again. -14. **SMP** — secondary cores from the park loop, the big lock, IPIs, placement classes. - -**What the demo takes as interim, honestly.** The demo needs increments 1–7 and runs full contexts only. -Its `call` goes to an **active** server, so RFC-0004 §4's accepted `call` semantics — donation — are not -exercised by it; that is increment 8, one more PR, safe to pull in before 9 and 10 because a lent full -context (`B = T`) refills at once and merely re-queues its holder: expiry inside a server needs a partial -context, which the demo never configures. Its userspace stays soft-float, so 11 is not needed. The x86_64 -no-op HAL functions are the other interim, removed by 13. Nothing else is smuggled. - -## 19. What this unblocks +as `capability/` is over `ObjectRef` (the `CLAUDE.md` § Layout edit lands with increment 1); hardware +behaviour is proved by counted QEMU boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The +x86_64 HAL gains stubs that compile, `-> !` ones halting, so its entry still calls straight through into +the kernel proper and the second build compiles the scheduler. **D** marks what the Phase-1 demo needs. + +| # | PR | Proves; how tested | Lands after | D | +|---|----|--------------------|-------------|---| +| 1 | `scheduler` I: run queues, the ten states, MCP, priority-ordered endpoint queue | exhaustive host transition table against §4's "Left by" column; "a new thread cannot serve as `auth` above 0"; CI builds it for both targets | — | D | +| 2 | `scheduler` II: full contexts, nanosecond conversion and rounding, billing, stall counters, the interim constants | host tests against a shadow model | 1 | D | +| 3 | GICv3 and the virtual timer, kernel-only; `CNTKCTL_EL1`, `PMUSERENR_EL0` zeroed; idle and its abandon-idle IRQ stub | xtask gains `gic-version=3`; boot-test `timer: tick 3` | 0005#9 (GIC in the MMIO window) | D | +| 4 | Event-kernel dispatch at EL0, MMU off: TCB frames, dispatch and RFC-0006's frame layout under RFC-0007#3's trap stub; `SCTLR_EL1` bits of §8 set | two EL0 self-test threads alternate under the timer: `sched: A B A B`; an EL0 `wfi` completes: `sched: wfi yields` | 3, 0007#3 | D | +| 5 | Threads in address spaces; the boot stack moved into the guarded window, `aarch64.ld`'s comment with it | a spinning EL0 thread is preempted and resumes intact: `user: still here` | 4, 0005#10 | D | +| 6a | Thread, SchedContext and `SchedControl` as slots in RFC-0005's object store, state machine wired, links `(index, generation)` | host tests in `scheduler/`: destroy a lent context, reuse its slot, `reply` — nothing moves | 2, 0005#7 | D | +| 6b | The methods behind RFC-0007's dispatcher: `sc_configure`, `bind_sc`, `resume`, `set_priority`, `write_regs`, `thread_exit` | the root configures two EL0 threads at equal priority: `[a] 1 [b] 1 [a] 2` | 5, 6a, 0007#6 | D | +| 7a | Direct-switch decision as a pure function | host tests over every §6 case | 1 | D | +| 7b | Endpoint queues and IPC blocking states, wired | host tests: queue order, `suspend` from each blocked state | 6b, 7a | D | +| 7c | Integration with RFC-0007#10 | RFC-0007#10's three `Kaya!` lines, then `[client] preempted` from a spin after the reply | 7b, 0007#10 | D | +| 8 | Passive servers, donation, inheritance, the caller-death rule | host test: a client exits mid-request, the server finishes and serves the next; boot-test `passive: served` | 7c | stretch | +| 9 | `scheduler` III: partial contexts, refills, merge, release queue | property test: no window of length `T` sees more than `B` consumed | 2 | | +| 10 | `min_budget`, deferral, expiry return, fault and timeout endpoints, `Unfunded` | boot-tests `BudgetRefused`, then `BudgetExpired` and `timeout fault` for an overrunning server | 8, 9, 0007#11 | | +| 11 | Lazy FP/SIMD | two FP threads load markers into `v0`, each finds its own: `fp: isolated`; host test: destroy the owner, reuse its slot, first use loads zeros | 6b | | +| 12 | `IrqControl` / `IrqHandler`; xtask gains serial input injection | PL011 receive (SPI 1, INTID 33) reaches an EL0 driver: `irq: notified` | 7c | | +| 13 | x86_64 scheduling HAL (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`) | 3–5 and 11 again | its boot RFC, 0007#12 | | +| 14 | SMP: PSCI `CPU_ON` and INIT-SIPI, the big lock with IPI-servicing spin, FP handoff, shootdowns, placement classes, routing | `-smp 2` boot-tests | 13 | | +| 15 | Deadline objects (userspace timers, RFC-0004 §9.3) | host tests on the shared release queue; boot-test `deadline: fired` | 9 | | + +**What the demo takes as interim, honestly.** It needs 1–7c and runs full contexts only. + +- **Its server is active**, the root doubling as server (RFC-0007 §18), so the demo exercises **A1's + amended semantics, not RFC-0004 §4's**: no donation happens. Increment 8 is one more PR, safe before 9 + and 10 because a lent full context refills at once; expiry inside a server needs a partial context. +- **Provisional constants**, in `scheduler/src/config.rs` until measured: `MIN_BUDGET_INTERIM` = 100 µs; + `ROOT_TIMESLICE` = 5 ms (a full context); root priority and MCP 255, the root lowering itself and the + client to 100 to share a level; `KERNEL_STACK_SIZE` = 16 KiB. Cost: the budget floor is a guess. +- **No fault handler:** a faulting thread goes `Inactive` and the reporter prints it (RFC-0007 §7). +- **The console write is non-preemptible** until RFC-0007 adds its preemption points — free on QEMU, + about 44 ms on hardware. **No userspace timer**, so RFC-0004's watchdog cannot yet be built. +- **The x86_64 stubs** compile and halt, removed by 13. Userspace is soft-float, so 11 is not needed. + +## 20. What this unblocks RFC-0004's implementation gets threads to rendezvous between. The driver framework receives interrupts as -notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `Core`, MCP, -thresholds and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment (14), -and dated notes against RFC-0004 for §7's deviation and reconciliation. +notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `SchedControl`, +MCP, `min_budget` and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment +(14), and the dated entries of §13. From 9ba399a4272fc79f34f03f6ec9806713ffc257ec Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:40:47 +0000 Subject: [PATCH 13/20] wip: autosave 14:40Z Automatic five-minute backup of work in progress. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0005-address-spaces-and-physical-memory.md | 299 +++---- docs/rfcs/0006-threads-and-scheduling.md | 790 ++++++++---------- 2 files changed, 494 insertions(+), 595 deletions(-) diff --git a/docs/rfcs/0005-address-spaces-and-physical-memory.md b/docs/rfcs/0005-address-spaces-and-physical-memory.md index 54b11d6..2a9c1e0 100644 --- a/docs/rfcs/0005-address-spaces-and-physical-memory.md +++ b/docs/rfcs/0005-address-spaces-and-physical-memory.md @@ -56,48 +56,43 @@ and 47-bit user and kernel halves whose top level uses 256 entries each** — AA | Accessed | `AF` (bit 10), set by software on this CPU | bit 5 (A), set by hardware | | Address-space tag | 16-bit ASID in `TTBR0_EL1[63:48]` | 12-bit PCID in `CR3[11:0]` | -*(Bits checked against Linux's `arch/arm64/include/asm/pgtable-hwdef.h` — Arm's site was unreachable — and -Intel SDM Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, no hardware -access-flag update and no PAN: `target/arm/tcg/cpu64.c` sets `ID_AA64MMFR0_EL1 = 0x1124`, `MMFR1` 0.)* -Hence **one table engine**, generic over a per-architecture `Descriptor` encoder and a frame-access trait, -host-tested on both in `memory/`; only turning a frame number into `&mut [u64; 512]` through a kernel window -(§6, §13) is `unsafe`, in `kernel/src/mm/`. +*(Checked against Linux's `arch/arm64/include/asm/pgtable-hwdef.h` — Arm's site was unreachable — and SDM +Vol. 3A §4.5. QEMU's `cortex-a72` has 44-bit physical addresses, 16-bit ASIDs, no hardware access flag and +no PAN: `target/arm/tcg/cpu64.c`, `ID_AA64MMFR0_EL1 = 0x1124`, `MMFR1` 0.)* Hence **one table engine**, +generic over a `Descriptor` encoder and a frame-access trait, host-tested in `memory/`; only turning a frame +number into `&mut [u64; 512]` through a kernel window (§6, §13) is `unsafe`, in `kernel/src/mm/`. ## 4. Physical memory: discovery and the frame allocator -**Discovery.** QEMU fixes two facts about `virt` — flash at 0, RAM at 0x4000_0000 — and says every other -address "may vary" and must be read from the DTB, found "at the start of RAM" with no register pointing at -it (`docs/system/arm/virt.rst`); `hw/arm/boot.c` places it only *below* the image (`dtb_limit`). Checked on -QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 boots with **no DTB in RAM**; at 0x4020_0000 one is found at -0x4000_0000. **So the image loads at 0x4020_0000**, block-aligned, capping the DTB at 2 MiB; increment 1 -re-checks the pinned QEMU 11.0.3. A minimal FDT reader (Borrow Ledger boot-path row; Devicetree -Specification v0.4) reads the cell sizes, `/memory`, `/memreserve/` and `/reserved-memory`, bounds-checks -every offset and fails closed. Threat model §10 gains a proposed first firmware assumption — *firmware -tables describe memory truthfully* — via `docs/CHANGELOG.md`. On x86_64 the UEFI stub (no RFC yet) hands -over `GetMemoryMap` after `ExitBootServices`. **Both produce one `MemoryMap`**: same-kind neighbours -coalesced, then at most **256** sorted, disjoint `(PhysRange, Kind)` entries (6 KiB; raw UEFI maps run to -100–200) — `Usable`, `KernelImage`, `DeviceTree`, `Reserved`, `Mmio` — failing closed beyond that; image, -boot stack, DTB and boot tables tagged first. Phase 1's `Mmio` entries are the interim hard-coded -addresses (§18). Nothing above `arch` names FDT or UEFI. - -**The frame allocator is a bitmap** — one bit per 4 KiB frame, stored in the first free frames (16 KiB per -512 MiB) — allocating from a *colour set*, or runs of `n` at an alignment; a double free fails closed. -Colour is `pfn mod C`, `C = 1` until time protection *(Ge et al., EuroSys '19, via research/0002 Part 6)*. A -run search is O(frames / 64) words, bounded by RAM rather than the caller; a per-alignment cursor is the -remedy if measurement asks. **Fresh frames are zeroed before a process can observe them, and a table frame -before its parent descriptor is written** (a `DSB` between): a recycled table's stale entries would -translate into another process's frames. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing -fights colouring; a **free list**, which cannot find `n` contiguous frames without a scan. +**Discovery.** QEMU fixes RAM at 0x4000_0000 on `virt`; every other address "may vary" and comes from the +DTB, "at the start of RAM" with no register naming it (`docs/system/arm/virt.rst`) — and `hw/arm/boot.c` +puts it only *below* the image (`dtb_limit`). Checked on QEMU 8.2.2 on 2026-09-27: an ELF at 0x4000_0000 +finds **no DTB in RAM**; at 0x4020_0000 it finds one at 0x4000_0000. **So the image loads at 0x4020_0000**, +capping the DTB at 2 MiB; increment 1 re-checks QEMU 11.0.3. A minimal FDT reader (Borrow Ledger boot-path +row; Devicetree Specification v0.4) reads the cell sizes, `/memory`, `/memreserve/` and `/reserved-memory`, +bounds-checks every offset and fails closed; threat model §10 gains a proposed assumption, *firmware tables +describe memory truthfully*. x86_64's UEFI stub (no RFC yet) hands over `GetMemoryMap` after +`ExitBootServices`. **Both produce one `MemoryMap`**: same-kind neighbours coalesced, then at most **256** +sorted, disjoint `(PhysRange, Kind)` entries (raw UEFI maps run to 100–200) — `Usable`, `KernelImage`, +`DeviceTree`, `Reserved`, and `Mmio` from Phase 1's hard-coded addresses — failing closed beyond that; image, +boot stack, DTB and boot tables tagged first. Nothing above `arch` names FDT or UEFI. + +**The frame allocator is a bitmap** (16 KiB per 512 MiB, in the first free frames) allocating from a *colour +set* — `pfn mod C`, `C = 1` until time protection *(Ge et al., EuroSys '19, via research/0002 Part 6)* — or +runs of `n` at an alignment; a double free fails closed. A run search is bounded by RAM, not the caller; a +per-alignment cursor if measurement asks. **Fresh frames are zeroed before a process can observe them, and a +table frame before its parent descriptor is written** (a `DSB` between), lest stale entries translate into +another's frames. *Rejected:* a **buddy allocator** *(Linux)*, whose coalescing fights colouring; a **free +list**, which cannot find `n` contiguous frames without a scan. ## 5. Who pays: the object store and Pools (O-7) -The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped and retype**: -reusing an Untyped revokes what was retyped from it through a derivation tree, deciding RFC-0003a's B1 by -the back door (RFC-0003 §7 and its 2026-07-30 amendment) and making object layouts ABI. **Rejected.** **B — a -kernel heap with creation policy** *(Zircon)* limits *whether* a process creates objects, not how much memory -they take. **Rejected.** **C — static per-type tables** *(Hubris)* cannot say *who* filled one (research/0002 -Part 6's Kinesis outage). **Rejected alone.** **D — C with counted attribution: verdict sought** *(the -requester pays, as in seL4; counted quotas, as in Fiasco.OC's factory limits and Genode's quota trading)*. +The kernel has no heap and must never acquire one a process can drain. **A — seL4 Untyped/retype** reclaims +through a derivation tree, deciding RFC-0003a's B1 by the back door (RFC-0003 §7 and its 2026-07-30 +amendment), and makes object layouts ABI: **rejected**. **B — a heap with creation policy** *(Zircon)* limits +*whether*, not *how much*: **rejected**. **C — static per-type tables** *(Hubris)* cannot say *who* filled one +(research/0002 Part 6): **rejected alone**. **D — C with counted attribution: verdict sought** *(the requester +pays, as in seL4; counted quotas, as in Fiasco.OC's factory limits and Genode's quota trading)*. **The store is typed slot arrays**, one fixed-capacity array per type in `.bss`; a slot holds the object, a generation and a count. The kernel's implementation of `ObjectRef` (`capability/src/object.rs`, a trait) is @@ -107,10 +102,10 @@ counts nothing on clone or drop. A destroyed object's slot is therefore **reused advanced**, stale capabilities failing closed — what RFC-0003 §8's generation exists for — and no past holder pins another's charge. A slot whose generation cannot advance retires, debited once from its payer (objects keep 64-bit generations, RFC-0007 (proposed) §6). `object.rs`'s "takes a reference count" is qualified to -match in increment 7. **Where it lives:** the generic array and Pool are safe code in `memory/`; the concrete -store — one `Kernel` value with every type's array, RFC-0004/0006/0007 objects included — sits behind **one** -`static` wrapper in `kernel/src/mm/` with one SAFETY argument: one core, kernel entered with interrupts -masked (RFC-0006 (proposed) V1), the big lock once SMP lands. No other kernel global accretes. +match in increment 7. **Where it lives:** generic array and Pool in `memory/`; the concrete store, one +`Kernel` value with every type's array (RFC-0004/0006/0007 objects too), behind **one** `static` in +`kernel/src/mm/` whose SAFETY argument is one core entered with interrupts masked (RFC-0006 (proposed) V1), +then the big lock. No other kernel global accretes. **A Pool is the accounting layer**: an RFC-0003 object holding balances of frames and of each type's slots. At boot every free unit becomes the balance of one root Pool (RFC-0007 (proposed) hands it over). Then: @@ -132,19 +127,17 @@ every unit of r in use == kept by the kernel (at boot, or retired), or ch its charges come home — a chain bounded by the Pool slot count; the root's balance goes to the kernel. - **Rights:** `READ` queries; `WRITE` spends and moves; `EXECUTE` lets minted Regions carry `EXECUTE` (§9). -**One accounting domain — decided here, since the programme gives this RFC O-7's model.** research/0002 -Part 6 asks for one capability-named domain for memory, object quotas and scheduling contexts. **The Pool is -that domain for every counted unit**, Threads' and SchedContexts' slots included. **Time is not a unit**: -RFC-0006 (proposed) sells per-core bandwidth through `Core`, which moving balance cannot conserve, and -folding it in would make every Pool holder a seller of CPU — a deliberate split from the research, with -congruence kept by attribution. O-7's text would add *"kernel memory is charged to a capability-named Pool; -CPU time to a SchedContext funded through a `Core`"*, proposed through `docs/CHANGELOG.md`. +**One accounting domain — decided here, as the programme gives this RFC O-7's model.** research/0002 Part 6 +wants one for memory, object quotas and scheduling contexts. **The Pool is it for every counted unit**, +SchedContext slots included. **Time is not a unit**: RFC-0006 (proposed) sells per-core bandwidth through +`Core`, which moving balance cannot conserve, and folding it in would make every Pool holder a CPU seller — a +deliberate split, congruent by attribution. O-7 would add *"kernel memory is charged to a capability-named +Pool; CPU time to a SchedContext funded through a `Core`"*, via `docs/CHANGELOG.md`. ## 6. The kernel's virtual layout -Both architectures get the **same layout**, the cheapest way to keep the HAL honest. The image sits in the -top 2 GiB, which `x86_64-unknown-none`'s default kernel code model expects (overridable, as RFC-0007 does -for userspace, but that buys nothing); AArch64 follows. +Both architectures get the **same layout**, the cheapest way to keep the HAL honest; the image sits in the +top 2 GiB, as `x86_64-unknown-none`'s default kernel code model expects (overridable, but that buys nothing). | Range (both architectures) | Contents | Attributes | |----------------------------|----------|------------| @@ -162,10 +155,9 @@ for userspace, but that buys nothing); AArch64 follows. unmapped**, closing Intel's `SYSRET` hazard (CVE-2012-0217) for RFC-0007 by layout. The physmap is RAM only. - **The kernel's top-level entries never change after boot.** AArch64 shares `TTBR1_EL1`; x86_64 copies PML4 entries 256–511 into each address space *(seL4 x86)*, so every kernel-half PML4E ever used is populated at - boot — one filled later would be missing from earlier copies (Linux's vmalloc-sync bugs). Lower kernel - tables may change, as global changes invalidated on every core (§12). *Rejected:* an identity-mapped - kernel, which sits where user addresses go and breaks the symmetric halves; no physmap, which costs an - invalidation per IPC-buffer touch. + boot — one filled later would be missing from earlier copies (Linux's vmalloc-sync bugs); lower kernel + tables may change, invalidated on every core (§12). *Rejected:* an identity-mapped kernel, where user + addresses go; no physmap, an invalidation per IPC-buffer touch. ## 7. The objects: Region, AddressSpace and Mapping @@ -178,31 +170,28 @@ under a live mapping *(Zircon: a mapping keeps its VMO alive)*. One object, thre - **Ordinary** (`region_create`): eager, zeroed, charged; unmet in eight extents it fails `Fragmented`; DMA memory is one extent at an alignment. Zeroing runs 512 pages per step between RFC-0006 (proposed)'s preemption points, the half-built Region held on the caller's thread; no capability exists until done. -- **Boot-minted, read-only**, once, over the DTB and each boot module RFC-0007 (proposed) embeds: their - `DeviceTree`/`KernelImage` frames **move** into the Region, never zeroed (public boot data: O-8 is - untouched), never charged, never `WRITE`. The DTB gets `READ`; a module `READ | EXECUTE | DUPLICATE | - TRANSFER`, so text maps `RX` in place and W^X holds because no writable use can exist. RFC-0007's bundle - must page-align every module segment. +- **Boot-minted, read-only**, over the DTB and each boot module RFC-0007 (proposed) embeds (every segment + page-aligned): their frames **move** into the Region, never zeroed (public boot data, so O-8 is untouched), + never charged, never `WRITE` — the DTB `READ`, a module `READ | EXECUTE | DUPLICATE | TRANSFER`, so text + maps `RX` in place and W^X holds because no writable use can exist. - **Device**, minted at boot over `Mmio` the kernel does not keep; never executable, zeroed or charged; Phase 2. No operation maps a physical address given as a number: the catch-all right in costume. -What the kernel fills for the root before handover (writable segments, stack, BootInfo) is ordinary Regions, -charged to the root Pool, bytes copied in through the physmap, and only then a capability minted. - +What the kernel fills for the root before handover (writable segments, stack, BootInfo) goes into ordinary +Regions, charged to the root Pool, bytes copied in through the physmap, and only then a capability minted. **An AddressSpace** is a user root table, a tag (§12), the Pool paying for its tables — the only Pool `map` -may charge — and its mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query. Binding it to a +may charge — and its mappings; `WRITE` permits `map`/`unmap`/`protect`, `READ` a query; binding it to a process is RFC-0007's. **A Mapping** is a charged kernel record, not a capability: `{as, region, first_page, -vaddr, pages, perms}`, linked by slot index into its AddressSpace's list and its Region's — because RFC-0003's -2026-07-30 amendment makes load-bearing that *a resolved capability is never cached outside the kernel -table*, and **a page-table entry is exactly such a cache**, which hardware never re-checks. *Rejected:* -**Zircon VMOs and VMARs** — lazy commit, copy-on-write, pagers and nesting are policy-rich and put faults in -kernel paths; **seL4's capability per frame** — 256 handles per MiB; **userspace-built tables** (Xen PV). +vaddr, pages, perms}`, linked into its AddressSpace's list and its Region's, because RFC-0003's 2026-07-30 +amendment makes load-bearing that *a resolved capability is never cached outside the kernel table* — and **a +page-table entry is exactly such a cache**. *Rejected:* **Zircon VMOs and VMARs** (lazy commit, copy-on-write, +pagers, nesting: policy, and faults in kernel paths); **seL4's capability per frame**; **user-built tables** (Xen PV). ## 8. Map, unmap, protect — and the rights they check -Every operation takes handles (RFC-0003 §9); RFC-0007 (proposed) encodes them as methods. A second handle — -`map`'s Region and Pool — travels as a **handle word in a message register**, resolved in the caller's -table, never transferred: RFC-0003 §6's move would take the Region from its mapper. +Every operation takes handles (RFC-0003 §9), encoded as methods by RFC-0007 (proposed); a further handle +(`map`'s Region and Pool) is a **handle word in a message register**, resolved in the caller's table, never +transferred — RFC-0003 §6's move would take the Region from its mapper. | Operation | Needs | Refused when | |-----------|-------|--------------| @@ -213,23 +202,21 @@ table, never transferred: RFC-0003 §6's move would take the Region from its map | `protect(as, vaddr, perms)` | as `WRITE` | `perms` not a subset of the mapping's current `perms` | User numbers are validated *as numbers*; **the kernel never chooses a virtual address** *(seL4's VSpace -discipline)*. **`map` names a Pool because spending is authority** (O-4): an AddressSpace capability alone -cannot drain one, and requiring the space's *own* Pool keeps one payer per space, so refunds need no -per-table record *(seL4 has the caller supply page-table objects)*. *Rejected:* any Pool (a payer record per -table frame); a Pool bound at `as_create` and spent through `WRITE` (spending never granted). **`map` is -atomic**: it charges missing tables and the Mapping slot before writing anything, then zeroes and links -tables, writes leaves and ends with `DSB ISHST`, so the walker sees them before `eret`; at most 512 leaves -and six new tables per call. `protect` only narrows; widening is `map` again *(seL4's remap)*. 4 KiB pages -only; blocks later, invisible in the ABI. - -**Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as RFC-0003 §5 allows "when a subsystem -needs one". Folding it into `READ` makes every readable Region potential code; a Region *kind* fails the -loader, which writes code before running it. seL4 leaves execution ungoverned; Fuchsia's `ZX_RIGHT_EXECUTE` -is the lineage. It means *may map executable* on a Region, *may mint executable Regions* on a Pool, and is -**refused at mint on every other type** by the store (host-tested, increment 7); `derive` cannot add it. -`rights.rs`'s `ALL` is an explicit union despite its comment; increment 4 edits both. **On acceptance, -RFC-0003 gains a dated amendment adding `EXECUTE` to §5's table, logged in `docs/CHANGELOG.md`.** RFC-0003 -§14.3's handshake: **the installed permission is `perms`, refused unless a subset of the Region's rights.** +discipline)*. **`map` names a Pool because spending is authority** (O-4); requiring the space's *own* Pool +keeps one payer per space, so refunds need no per-table record *(seL4's caller-supplied page tables)*. +*Rejected:* any Pool (a payer record per table); a Pool bound at `as_create` and spent through `WRITE` +(spending never granted). **`map` is atomic**: it charges missing tables and the Mapping slot first, then +zeroes and links tables, writes leaves and ends with `DSB ISHST` before `eret`; at most 512 leaves and six +new tables. `protect` only narrows; widening is `map` again *(seL4)*. 4 KiB pages; blocks later. + +**Verdict sought: `EXECUTE` joins RFC-0003's rights as bit 5**, as §5 there allows "when a subsystem needs +one". Folded into `READ`, every readable Region is potential code; a Region *kind* fails the loader, which +writes code before running it. seL4 leaves execution ungoverned; Fuchsia's `ZX_RIGHT_EXECUTE` is the lineage. +It means *map executable* on a Region, *mint executable Regions* on a Pool, and is **refused at mint on every +other type** (increment 7); `derive` cannot add it. `rights.rs`'s `ALL` is an explicit union despite its +comment; increment 4 fixes both. **On acceptance, RFC-0003 gains a dated amendment adding `EXECUTE` to §5's +table, logged in `docs/CHANGELOG.md`.** RFC-0003 §14.3's handshake: **installed permission is `perms`, +refused unless a subset of the Region's rights.** | perms | needs on the Region | AArch64 stage 1 | x86_64 | |-------|---------------------|-----------------|--------| @@ -248,20 +235,18 @@ execute-only exists on AArch64 but not on x86_64 without protection keys, so nei bindings) and `RX` mappings: `map RX` is refused while any writable use exists, `map RW` and binding while any `RX` mapping does. `unmap`, `protect` and unbinding decrement **only once their invalidation has completed on every core** (§12) — until then a stale writable entry may be live somewhere. -3. **Over time.** `EXECUTE` is minted only from a Pool carrying it: the broker hands apps Pools without it, - the loader holds one, a JIT gets one by explicit grant *(Fuchsia's `VmexResource`)*. O-13's Phase-3 - load-path clause is policy on this. +3. **Over time.** `EXECUTE` is minted only from a Pool carrying it — the loader holds one, a JIT gets one by + explicit grant *(Fuchsia's `VmexResource`)*. O-13's Phase-3 load-path clause is policy on this. 4. **Hardware backstop.** AArch64 sets `SCTLR_EL1.WXN` (bit 19) once the final tables are in (§13); user entries set `PXN`; kernel text has no writable alias, the physmap mapping it read-only *(Linux `mark_linear_text_alias_ro`)*. x86_64 has no `WXN`: `CR0.WP` and `SMEP` where present. -`map RX` makes AArch64's instruction side coherent: `DC CVAU` through the physmap, `DSB ISH`, `IC IVAU` on a -PIPT instruction cache (`IC IALLUIS` otherwise; Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`; QEMU's -cortex-a72 reports PIPT (`CTR_EL0 = 0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so -errors here pass every test.** These and §12's `TLBI`s and `TTBR` writes are single-instruction `asm!` -intrinsics in `kernel/src/arch/**` with no Rust equivalent, as `wfe` is today — a reading of "minimal -assembly" the maintainer is asked to confirm (§17.7). The physmap still aliases *user* code writable at EL1; -writing through it only to zero, fill tables and reach IPC buffers is a convention (§16). +`map RX` makes AArch64's instruction side coherent: `DC CVAU` via the physmap, `DSB ISH`, `IC IVAU` if PIPT +(`IC IALLUIS` otherwise; Linux's `sync_icache_aliases`), `DSB ISH`, `ISB`; cortex-a72 reports PIPT (`CTR_EL0 = +0x8444c004`); x86_64 is coherent in hardware. **QEMU models no cache, so errors here pass every test.** These +and §12's `TLBI`s and `TTBR` writes are single-instruction `asm!` intrinsics in `kernel/src/arch/**`, as `wfe` +is today — a reading of "minimal assembly" for the maintainer to confirm (§17.7). The physmap still aliases +*user* code writable at EL1; writing through it only to zero, fill and reach IPC buffers is a convention (§16). ## 10. Sharing and destruction (RFC-0004 §9.1) @@ -280,9 +265,9 @@ writing through it only to zero, fill tables and reach IPC buffers is a conventi **Concurrency.** Under RFC-0006 (proposed) V1 — non-preemptible, one big lock once SMP lands — no kernel access to a Region's frames (zeroing, table fill, IPC copy) interleaves with its destruction: each runs whole -in one kernel entry and re-resolves the Region, and the dying mark covers the gaps between destruction's -steps. Finer locking, if ever, takes **Region before AddressSpace**. A former sharer takes an ordinary fault: -RFC-0006 (proposed) binds delivery, RFC-0007 (proposed) defines the message; under its interim the thread stops. +in one entry and re-resolves the Region; the dying mark covers destruction's gaps. Finer locking takes +**Region before AddressSpace**. A former sharer faults: RFC-0006 (proposed) binds delivery, RFC-0007 (proposed) +defines the message; under its interim the thread stops. ## 11. How the kernel touches user memory (O-5) @@ -299,9 +284,8 @@ writes for a process is named by capability and reached through the physmap: frame, and the in-kernel fault concurrency that sank L4's long IPC cannot arise. - **Peer-writable memory is never referenced**: one `kernel/src/mm/` module copies with volatile accesses; anything interpreted is copied once and validated as that snapshot *(virtio's double fetch, research/0002 - Part 5)*. *Rejected:* `copy_from_user` with fixups (Linux), making kernel faults a normal path. - -`SMAP` and `PSTATE.PAN` back the rule where present; **cortex-a72 has no PAN**, so there it stands on structure. + Part 5)*. *Rejected:* `copy_from_user` with fixups (Linux). `SMAP` and `PAN` back the rule where present; + **cortex-a72 has no PAN**, so there it stands on structure. ## 12. TLBs, ASIDs and PCIDs @@ -310,15 +294,15 @@ writes for a process is named by capability and reached through the physmap: `TLBI ASIDE1IS` (x86_64: `INVPCID` single-context), so churn does not force rollover. *Rejected:* seL4's ASID pools — a tag is not authority. - **Kernel entries are global, user entries never.** `map` needs no invalidation: neither architecture caches - a faulting entry (the Arm ARM's rule, section uncited as the manual was unreachable; SDM Vol. 3A §4.10), - and a fault whose entry is valid on re-walk is retried, as x86_64's paging-structure caches can cause. - `unmap`, `protect` and teardown store, then `DSB ISHST`, `TLBI VALE1IS`, `DSB ISH`, `ISB`; x86_64 uses - `INVLPG`. Freeing an intermediate table uses `TLBI VAE1IS` or `ASIDE1IS`, since `VALE1IS` spares walk - caches; `INVLPG` and `INVPCID` already flush paging-structure caches (SDM Vol. 3A §4.10.4.1). -- **O-8's ordering: invalidate, then free.** No frame returns to a Pool while any core can translate to it. + a faulting entry (the Arm ARM, section uncited as it was unreachable; SDM Vol. 3A §4.10), and a fault whose + entry is valid on re-walk is retried. `unmap`, `protect` and teardown store, `DSB ISHST`, `TLBI VALE1IS`, + `DSB ISH`, `ISB`; x86_64 `INVLPG`. Freeing an intermediate table needs `TLBI VAE1IS` or `ASIDE1IS`, as + `VALE1IS` spares walk caches; `INVLPG`/`INVPCID` flush them (SDM Vol. 3A §4.10.4.1). **O-8's ordering: + invalidate, then free** — no frame returns to a Pool while any core can translate to it. -**Multi-core: this RFC owns the protocol; RFC-0006 (proposed) provides the IPI.** Each AddressSpace keeps -the set of cores that have run it since its last full flush *(Linux `mm_cpumask`)*. +**Multi-core: this RFC owns the protocol; RFC-0006 (proposed) provides the IPI**, built with its SMP +increment; the `-smp 1` demo already uses only broadcast `TLBI`s. Each AddressSpace keeps the set of cores +that have run it since its last full flush *(Linux `mm_cpumask`)*. - **Shootdown.** AArch64 needs no IPI: the `…IS` forms reach the inner-shareable domain and `DSB ISH` waits for completion everywhere. x86_64 has no broadcast: the initiator IPIs the cores in the set, each @@ -331,14 +315,11 @@ the set of cores that have run it since its last full flush *(Linux `mm_cpumask` - **x86_64 PCIDs are per core**: each core caches a few AddressSpace→PCID pairs *(Linux keeps six)*; an invalidation where the space is not current is a per-core stale mark consumed at the next switch to it. -The demo is `-smp 1`, but every `TLBI` is the broadcast form; the rest lands with RFC-0006's SMP increment. - ## 13. Early boot: turning the MMU on -The image is linked at `0xFFFF_FFFF_8020_0000` and loaded at 0x4020_0000 (`AT()` in `aarch64.ld`). QEMU loads -segments at `p_paddr` and translates the entry to its physical alias **only if** it lies in an executable -segment with `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, `load_elf`, v8.2.2) — which `AT()` guarantees -while `.text.boot` leads the first executable segment. With the MMU off, the stub: +The image is linked at `0xFFFF_FFFF_8020_0000`, loaded at 0x4020_0000 (`AT()`). QEMU translates the entry to its +physical alias **only if** it lies in an executable segment with `p_vaddr != p_paddr` (`include/hw/elf_ops.h`, +`load_elf`, v8.2.2), which `AT()` gives while `.text.boot` leads that segment. With the MMU off, the stub: 1. Parks secondaries, installs the stack and zeroes `.bss` at physical addresses, as today. 2. `MAIR_EL1`: index 0 = `0xFF` (Normal write-back, read/write-allocate), index 1 = `0x04` (Device-nGnRE). @@ -362,16 +343,14 @@ while `.text.boot` leads the first executable segment. With the MMU off, the stu | `WXN` (19) | 0, then 1 | set by Rust step (5), giving `0x30DC581D` | | 11, 20, 22, 23, 28, 29 | 1 | RES1 on Armv8.0: Linux v4.19's `SCTLR_EL1_RES1`, plus bit 23 (`SPAN`, RES1 before Armv8.1), which Linux always sets; not re-checked against the Arm ARM | -Every other bit is 0: RES0 on Armv8.0, or a later extension left off. Rust then, never leaving the reporter -without a console: (1) builds the `MemoryMap`, bitmap and slot arrays through the **boot window** — the -frame-access trait takes an offset, here `pa − 0x4000_0000 + 0xFFFF_FFFF_8000_0000`, so everything touched -this early lies in the first boot GiB; (2) builds §6's final tables, stack window included; (3) swaps -`TTBR1_EL1` by an assembly routine run from the identity map — empty table, `TLBI VMALLE1`, final table — -as break-before-make requires *(Linux `idmap_cpu_replace_ttbr1`)*, then re-references the bitmap and -switches the trait to the physmap offset; (4) **repoints every kernel MMIO user** — the UART, and the GIC if -RFC-0006 has brought it up; (5) sets `TCR_EL1.EPD0 = 1` (cleared by the first AddressSpace install) and -`WXN`, then `TLBI VMALLE1`, `DSB ISH`, `ISB`. **The boot tables are never freed**: 16 KiB of `.bss`, and -freeing them would give image frames a second owner. +Every other bit is 0 (RES0 on Armv8.0, or a later extension left off). Rust then, never leaving the reporter +without a console: (1) builds the `MemoryMap`, bitmap and arrays through the **boot window** — the +frame-access trait's offset is `pa − 0x4000_0000 + 0xFFFF_FFFF_8000_0000`, so all it touches lies in the boot +GiB; (2) builds §6's final tables; (3) swaps `TTBR1_EL1` from the identity map in assembly — empty table, +`TLBI VMALLE1`, final table, as break-before-make requires *(Linux `idmap_cpu_replace_ttbr1`)* — and moves +the trait and bitmap to the physmap offset; (4) **repoints every kernel MMIO user** (UART; GIC if up); +(5) sets `TCR_EL1.EPD0 = 1` (the first AddressSpace install clears it) and `WXN`, then `TLBI VMALLE1`, `DSB +ISH`, `ISB`. **The boot tables are never freed**: freeing `.bss` would give image frames a second owner. The reporter gains a DFSC and fault-level decode (it prints `FAR` already); x86_64's `#PF` prints `CR2` and its error code. **x86_64**, once the UEFI stub exists, is already paging: Rust builds §6's PML4 with a @@ -399,72 +378,58 @@ jumps high and drops the identity entry — the same "Rust runs only high" rule. - **Policy in the kernel.** No address choice, demand paging, copy-on-write, overcommit or OOM response. - **Baroque capability hierarchies.** No Untyped tree; Pools are flat, with one funder index for closure. - **Multi-copy IPC.** The slow path copies once, frame to frame; bulk data is shared, never copied (§10). -- **Bolted-on multicore.** Broadcast invalidation from the first line; shootdown, rollover and per-core tags - designed now (§12); only the IPI is RFC-0006's. +- **Bolted-on multicore.** Broadcast `TLBI` from the first line; shootdown, rollover, per-core tags designed now (§12). - **The catch-all right.** `EXECUTE` is narrow and refused off-type; no "map any physical address" right. - **Compiled-in unused device paths (VENOM).** The kernel maps only the devices it drives itself. ## 16. Costs — what this makes harder -- **Capacities are ceilings**: a slot count is a rebuild; `pool_move` is one call per resource. -- **Regions are eager and bounded**: no lazy or sparse Regions; a large one can fail `Fragmented`. +- **Capacities are ceilings** (a rebuild); **Regions are eager and bounded** (no lazy or sparse ones). - **`map` needs the AddressSpace's own Pool**: a loader mapping into a child holds the child's Pool. - **47-bit halves on AArch64** give up half of what `T0SZ = 16` allows; **the 64 KiB floor is fixed**. -- **User addresses are numbers only** (§11): every RFC-0007 syscall works in registers, the IPC buffer and - Region handles, for a bug class gone. -- **The physmap is a large target**: one kernel write bug reaches all RAM, user code included (§9) — a - software invariant, as are W^X on x86_64 and O-5 without PAN on cortex-a72. +- **User addresses are numbers only** (§11): every RFC-0007 syscall pays in ergonomics, for a bug class gone. +- **The physmap is a large target** (§9) — a software invariant, as are W^X on x86_64 and O-5 without PAN. - **Every boot runs RWX** from MMU-on to Rust step (5), before any process exists; a stray early write goes uncaught. **Cache maintenance is untestable under QEMU**; **the load address is a QEMU `virt` fact**. -- **Reclaim across a transfer needs cooperation**: a transferred object keeps its payer charged until - destroyed, so `pool_merge` waits on other processes — RFC-0003a's problem made concrete. +- **Reclaim across a transfer needs cooperation**: the payer stays charged until destruction (RFC-0003a). ## 17. Open questions 1. **Page-table frames on unmap.** Freed only with the AddressSpace; eager freeing needs §12's walk-cache rule. 2. **Donation and reclaim across a transfer** — `reissue` with a charge move, offered to RFC-0003a (§10). 3. **Which right authorises explicit destruction** — `REVOKE` would widen as `reissue` would; RFC-0003a. -4. **Hardware backing for two software rules:** software PAN on Armv8.0 (a `TTBR0_EL1` change per kernel - entry), and unmapping executable user frames from the physmap. +4. **Hardware backing for software rules:** software PAN on Armv8.0; unmapping executable user frames from the physmap. 5. **Preemptible teardown**: the O(mappings) walk made restartable at RFC-0006 (proposed)'s preemption points. 6. **The firmware assumption** (§4) and O-7's new text (§5), via `docs/CHANGELOG.md`. -7. **"Minimal assembly"** (Constitution §11.1): are runtime single-instruction `TLBI`/`DC`/`IC`/`TTBR` - intrinsics in `arch/**` within it (§9), or does the clause need amending? +7. **"Minimal assembly"** (Constitution §11.1): are runtime one-instruction `TLBI`/`DC`/`IC`/`TTBR` intrinsics within it (§9)? 8. **One programme order** across RFC-0005/0006/0007 — §18 proposes one for the maintainer to fix. ## 18. Implementation increments -One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate — -`no_std` in the kernel, `std` under `cargo test`, built for both targets in CI, no `unsafe`, as `capability/`. -Each new `arch` function (memory map, window, TLB, address-space switch) gets a documented x86_64 stub: an -empty `MemoryMap`, on which memory initialisation prints `mm: no memory map (UEFI stub pending)` and halts, -so the second Tier-1 build compiles the whole path from increment 2. +One reviewable PR each; **[demo]** marks what the demo needs. Pure logic lands in a new `memory/` crate as +`capability/` does (`no_std` in the kernel, `std` under test, both targets in CI, no `unsafe`). Each new `arch` +function gets a documented x86_64 stub — an empty `MemoryMap`, on which memory initialisation prints `mm: no +memory map (UEFI stub pending)` and halts — so the second Tier-1 build compiles the whole path. 1. **[demo] Relink at 0x4020_0000; prove the DTB.** MMU off. `--expect "DTB 0xd00dfeed at 0x40000000"`. 2. **[demo] Addresses, `MemoryMap`, FDT reader.** Host tests on a `dumpdtb` fixture, hostile blobs and later an OVMF dump; workspace `members`, `.github/workflows`, `CLAUDE.md` § Layout. `--expect "mm: 512 MiB RAM at 0x40000000"`. 3. **[demo] Frame bitmap.** Churn tests against a shadow model: no frame twice, none reserved, no image frame free. -4. **[demo] `EXECUTE` in `capability/`** with RFC-0003's dated amendment: bit 5, `ALL` and its comment, `Debug`, - the test widened to `0..64`. Demo-needed because the root maps the client's text. -5. **[demo] `MapPerms` and both encoders.** Exhaustive: nothing writable and executable; every user entry - `PXN`, `AF`, not-global; every row of §8's table. -6. **[demo] The table engine** on both formats: overlap, the 512-leaf and six-table bound, nothing mapped when - unpaid, tables zeroed before linking, a kernel-half map never writing the root. -7. **[demo] Slot store and one Pool.** Charge, refund, conservation; reuse at generation + 1; stale references - count nothing; `EXECUTE` refused at mint off-type; `object.rs`'s comment. Host-only. +4. **[demo] `EXECUTE` in `capability/`** with RFC-0003's amendment: bit 5, `ALL`, `Debug`, tests to `0..64`. +5. **[demo] `MapPerms` and both encoders.** Exhaustive: never W+X; every user entry `PXN`, `AF`, not-global. +6. **[demo] Table engine**, both formats: overlap, bounds, unpaid maps nothing, tables zeroed, root untouched by kernel maps. +7. **[demo] Slot store and one Pool.** Conservation; reuse at generation + 1; stale references count nothing; mint refuses off-type `EXECUTE`. 8. **`pool_split`/`move`/`merge` and closure forwarding**, under random churn. Host-only; not demo. -9. **[demo] MMU on (§13, stub).** Boot tables, the `SCTLR_EL1` value, `VBAR_EL1` high, DFSC decode; prints - the address of `kernel_main`: `--expect "mm: running at 0xffffffff80"`. +9. **[demo] MMU on (§13, stub)**, `SCTLR_EL1`, `VBAR_EL1` high, DFSC decode: `--expect "mm: running at 0xffffffff80"`. 10. **[demo] Final kernel map (§13, Rust)**, the store behind its static: physmap with the image read-only, MMIO window with every kernel MMIO user repointed, core 0's stack behind a guard; `EPD0`, `WXN`. One `--expect` per self-test: `provoke-ro-text` and `provoke-ro-alias` write `.text` through the image and the physmap (`data abort, same EL`); `provoke-wxn` branches into `.data` (`instruction abort, same EL`). 11. **[demo] Region and Mapping records** with the W^X counts. Host-only. 12. **[demo] The tag allocator**: rollover, per-core reservation, return on destruction. Host-only. -13. **[demo] AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID, `EPD0` cleared on first install. - `AT S1E0R` under two ASIDs resolves a shared Region to one frame, private ones apart: `mm: shared region agrees`. -14. **[demo] Pool, Region and AddressSpace methods through RFC-0007's `call`**, with its increment 6: an EL0 - self-test maps a fresh Region and prints `[el0] map ok`. +13. **[demo] AddressSpace and boot-minted Regions**; `TTBR0_EL1` with ASID; `EPD0` cleared. `AT S1E0R` under + two ASIDs finds a shared Region at one frame, private ones apart: `mm: shared region agrees`. +14. **[demo] Pool, Region, AddressSpace methods via RFC-0007's `call`** (its increment 6): `[el0] map ok`. 15. **`unmap` and `protect`** with invalidation and count decrements (§9, §12). Not demo. 16. **User-memory module (§11)**: a message beyond the register budget. Not demo. 17. **Destruction (§10)**, after RFC-0007 increment 11's fault message: a destroyed sharer's fault reaches its handler. @@ -481,8 +446,7 @@ which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7; 14 w - **One root Pool** (increment 8 is not a syscall yet): server and client charges are indistinguishable. - **No destruction or `unmap`**: a killed process's frames and slots stay charged; O-3's memory case is designed, not built. **No IPC-buffer module**: messages beyond the register budget cannot be sent. -- **Hard-coded UART and GIC physical addresses**, against QEMU's warning that they may vary. -- **Until increment 10 the kernel runs RWX throughout**; cache maintenance stays unverified. +- **Hard-coded UART and GIC addresses**, against QEMU's warning. **Until increment 10 the kernel runs RWX.** - **1 GiB boot blocks** cover 512 MiB with no RAM behind them, which speculation may touch on real hardware: there, 2 MiB blocks sized from the DTB. **Zeroing in one step** until RFC-0006's preemption points. - **A guarded-stack overflow is a recursive abort** until increment 18. @@ -491,8 +455,7 @@ which needs the stack window; 11–13 before RFC-0006 5 and RFC-0007 6–7; 14 w ## 19. What this unblocks -Increments 1–14 give RFC-0006 (proposed) address spaces, a guarded per-core stack and a Pool to charge -threads to, and RFC-0007 (proposed) the objects its Process, root task and boot info are made of, plus a -constraint it inherits rather than discovers: §11's sentence on user addresses. The root receives all memory -as one boot-minted Pool — RFC-0003 §14.4's bootstrap without an ambient grantor. Increments 3–8 are -host-only and 1–2 change only where the kernel loads and what it prints: all can land now. +Increments 1–14 give RFC-0006 (proposed) address spaces, a guarded per-core stack and a Pool to charge threads +to; RFC-0007 (proposed) the objects its Process, root task and boot info are made of, and §11's sentence on +user addresses as a constraint inherited, not discovered. The root receives all memory as one boot-minted +Pool — RFC-0003 §14.4's bootstrap without an ambient grantor. Increments 1–8 leave the MMU off: all can land now. diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 344d024..7b539c4 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -7,90 +7,75 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 ("IPC is the product", priority inheritance, bolted-on multicore); the HAL (`kernel/src/arch/**`), `boot.s`'s park loop and `aarch64.ld`'s stack plan; RFC-0003 §14.2 (a dated note); RFC-0004 §4, §8, §9.2, §9.3 and its 2026-08-01 amendment (three proposed amendments, §13); research/0002 Part 6 (one correction); RFC-0005 §13 step 5 and §18.2; `CLAUDE.md` § Architectures (FP enabled per thread) and § Layout (gains `scheduler/`); every userspace driver | +| Affects | Constitution §3 ("IPC is the product", priority inheritance, bolted-on multicore); the HAL (`kernel/src/arch/**`), `boot.s`'s park loop and `aarch64.ld`'s stack plan; RFC-0003 §14.2 (a dated note); RFC-0004 §4, §8, §9.2, §9.3 and its 2026-08-01 amendment (§13's amendments); research/0002 Part 6 (one correction); RFC-0005 §13 step 5 and §18.2; `CLAUDE.md` § Architectures (FP enabled per thread) and § Layout (gains `scheduler/`); every userspace driver | | Depends on | RFC-0003 (capability table); RFC-0004 (IPC) and its 2026-08-01 amendment; RFC-0005 (proposed) for object memory, kernel stacks, discovery and address-space switching; RFC-0007 (proposed) for the trap path, the syscall encoding, the Process object and the root task | | Discharges | O-7 for CPU time and the scheduler's own structures; O-4 for time; both commitments of RFC-0004's 2026-08-01 amendment **as amended by verdict 12**; answers RFC-0004 §9.2 in shape; contributes to O-5, O-9, O-17 | > **Proposed verdicts.** > -> 1. **Event kernel:** one kernel stack per core; the kernel is not preemptible but its long paths take -> preemption points; every wait is a thread state (§3). -> 2. **Thread:** an RFC-0003 object holding its user frame; ten states, one of them `Unfunded`; born at -> priority 0 with ceiling 0; its fault and timeout endpoints held in the thread, not its process (§4). -> 3. **Time is a capability:** a `SchedContext` (budget and period in nanoseconds, at most eight refills), -> configured only through a per-core `SchedControl` whose one method sells time on that core (§5). -> 4. **Budgets are caps, not minima:** the kernel enforces the cap; a guaranteed minimum is admission -> policy — a stated departure from research/0002 (§5). -> 5. **Priorities:** 256 fixed levels; any capability to a thread lends its priority ceiling (seL4's MCP); -> no scheduling right; O(1) bitmap run queues (§6). -> 6. **Endpoint queues are priority-ordered,** first-come within a level, as seL4 MCS (§6). -> 7. **Donation:** `call` lends time to a *passive* server, which runs at the higher of its own and its -> caller's priority; a caller's death never takes time from a server mid-request (§7). -> 8. **Budget guard:** an endpoint's `min_budget` is checked at `call`; a caller short *now* waits; expiry -> inside a server hands time back one hop with `BudgetExpired` and a timeout fault (§7). -> 9. **Timer:** tickless per-core deadlines — the AArch64 virtual timer; the x86_64 LAPIC, TSC-deadline -> only with an invariant TSC; userspace timers will be deadline objects on the same queue (§8). -> 10. **FP/SIMD:** off by default, enabled per thread, saved lazily; the kernel's only FP instructions -> are its save and restore stubs (§10). -> 11. **Interrupts and cores:** GICv3 first, LAPIC and I/O APIC; a device line signals a notification and -> stays masked until `ack`; a big lock whose waiters serve IPIs; cores started by PSCI or INIT-SIPI -> when first funded (§11, §12). -> 12. **Amendments proposed:** three to RFC-0004 and its 2026-08-01 amendment, one note on RFC-0003 -> §14.2, one correction to research/0002 (§13). +> 1. **Event kernel:** one kernel stack per core; long paths take preemption points; every wait is a thread state (§3). +> 2. **Thread:** ten states, one of them `Unfunded`; born at priority 0; fault handlers held in the thread (§4). +> 3. **Time is a capability:** a `SchedContext`, set only through its core's one-method `SchedControl` (§5). +> 4. **Budgets are caps, not minima:** a guaranteed minimum is admission policy — a departure from research/0002 (§5). +> 5. **Priorities:** 256 fixed levels; any capability to a thread lends its priority ceiling; no scheduling right (§6). +> 6. **Endpoint queues are priority-ordered,** first come first served within a level, as in seL4 MCS (§6). +> 7. **Donation:** `call` lends time to a passive server; a caller's death never takes it back mid-request (§7). +> 8. **Budget guard:** an endpoint's `min_budget` is checked at `call`; expiry inside a server fails one hop back (§7). +> 9. **Timer:** tickless per-core deadlines; userspace timers will be deadline objects on the same queue (§8). +> 10. **FP/SIMD:** off by default, enabled per thread, saved lazily; the kernel touches it only to save it (§10). +> 11. **Interrupts and cores:** a device line signals a notification; a big lock; cores start when first funded (§11, §12). +> 12. **Amendments:** three to RFC-0004, a note on RFC-0003 §14.2, a correction to research/0002 (§13). ## 1. The question -**Who runs next, on which core, for how long, and on whose authority — such that no thread runs on time -it was not granted, a server never runs on time nobody paid for, and the kernel never waits on anything -a process controls?** +**Who runs next, on which core, for how long, and on whose authority — such that no thread runs on time it was +not granted, a server never runs on time nobody paid for, and the kernel never waits on anything a process +controls?** -That is O-7's scheduler half plus RFC-0004's three scheduling hazards: priority-aware direct switch, -donation, and budget expiry inside a server. This RFC designs the mechanism. It chooses no priorities, -admits no workloads and balances no load; that is for whoever holds the capabilities defined here. +That is O-7's scheduler half plus RFC-0004's three scheduling hazards: priority-aware direct switch, donation, +and budget expiry inside a server. This RFC designs the mechanism. It chooses no priorities, admits no +workloads and balances no load; that is for whoever holds the capabilities defined here. ## 2. Which pillar -**The kernel doctrine (§3) directly, and pillar 2 by consequence.** "IPC is the product" holds only if a -`call` hands over the CPU and the budget without a scheduler round trip (RFC-0004). Time is the one -resource every thread consumes; if it is not a capability, pillar 2 has a hole exactly where -availability (A5) lives. seL4 MCS made time a capability, and this RFC adopts that shape. Borrow Ledger: -scheduler, timer and interrupt-controller code are the **microkernel core — write ourselves**; Redox's -drivers are read for reference only, since the kernel masks and signals but never services a device. +**The kernel doctrine (§3) directly, and pillar 2 by consequence.** "IPC is the product" holds only if a `call` +hands over the CPU and the budget without a scheduler round trip (RFC-0004). Time is the one resource every +thread consumes; if it is not a capability, pillar 2 has a hole exactly where availability (A5) lives. seL4 MCS +made time a capability, and this RFC adopts that shape. **Borrow Ledger:** scheduler, timer and +interrupt-controller code are the microkernel-core row — *write ourselves*; Redox is reference only, since the +kernel masks and signals but never services a device. ## 3. The execution model — one kernel stack per core (verdict 1) -- **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). **Rejected:** a - guarded stack per thread in a kernel with no heap, and blocking in the kernel is how "hold a lock across - a wait a process controls" (O-7) becomes possible. -- **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). - **Verdict sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with - interrupts masked, and leaves by restoring *some* thread's user frame. A syscall that must wait records - a state and the object waited on; on wake the result is written into the waiter's frame, or the call - restarts by rewinding the saved PC one instruction (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & - Elphinstone, TOCS 2016; Ford et al., "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* - -**The cost:** every kernel path must be bounded. Long ones — object destruction, RFC-0005 (proposed)'s -`reissue` and teardown, RFC-0007 (proposed)'s console write — take **preemption points**: on a pending -interrupt (`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. **A restarted -operation re-resolves every handle and re-checks every right from scratch**, and an object mid-teardown -carries a *dying* mark so no other entry invokes it between the exit and the restart. The worst path -between preemption points is measured and published: it is the interrupt latency. This **reverses -`aarch64.ld`'s plan** that the boot stack "will be replaced by per-thread stacks": it becomes core 0's -kernel stack, moved into RFC-0005 (proposed)'s guarded window (increment 5). - -**Idle is a kernel loop, not a thread,** billing nobody. It runs `wfi` with `PSTATE.I` set — a pending -interrupt still wakes it — then unmasks for one instruction; Linux warns that masking at `ICC_PMR_EL1` -would *not* wake the core (`arch/arm64/kernel/idle.c`), so this kernel masks with DAIF only. x86_64 uses -`sti; hlt`, whose STI shadow issues the `hlt` before delivery. **An interrupt taken in idle abandons it:** -the current-EL IRQ stub (this RFC's; RFC-0007 owns the lower-EL and `syscall` entries) saves nothing, -resets `SP` to the per-core stack top from the `TPIDR_EL1` block (`TSS.RSP0`'s twin on x86_64) and enters -the dispatcher; idle is never resumed. Idle is the only place the kernel unmasks, and it drops the big -lock (§12) before waiting. +- **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). **Rejected:** a guarded + stack per thread in a kernel with no heap, and blocking in the kernel is how "hold a lock across a wait a + process controls" (O-7) becomes possible. +- **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). **Verdict + sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with interrupts masked, + and leaves by restoring *some* thread's user frame. A syscall that must wait records a state and the object + waited on; on wake the result is written into the waiter's frame, or the call restarts by rewinding the saved + PC one instruction (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & Elphinstone, TOCS 2016; Ford et al., + "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* + +**The cost:** every kernel path must be bounded. Long ones — destruction, RFC-0005 (proposed)'s `reissue` and +teardown, RFC-0007 (proposed)'s console write — take **preemption points**: on a pending interrupt +(`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. **A restarted operation +re-resolves every handle and re-checks every right**, and an object mid-teardown carries a *dying* mark so no +entry invokes it between the exit and the restart. The worst path between points is measured and published: it +is the interrupt latency. The boot stack becomes core 0's kernel stack, moved into RFC-0005 (proposed)'s +guarded window — **reversing `aarch64.ld`'s plan** to replace it "by per-thread stacks". + +**Idle is a kernel loop, not a thread,** billing nobody. It runs `wfi` with `PSTATE.I` set — a pending interrupt +still wakes it — then unmasks; Linux warns that masking at `ICC_PMR_EL1` would *not* wake the core +(`arch/arm64/kernel/idle.c`), so this kernel masks with DAIF only. x86_64 runs `sti; hlt`, whose STI shadow +issues the `hlt` before delivery. **An interrupt taken in idle abandons it:** the current-EL IRQ stub — this +RFC's; RFC-0007 owns the lower-EL and `syscall` entries — saves nothing, resets `SP` to the per-core stack top +and enters the dispatcher. Idle is the only place the kernel unmasks, and it drops the big lock (§12) first. ## 4. The Thread object (verdict 2) -A **Thread** is an RFC-0003 object belonging to one Process, which RFC-0007 (proposed) provides. Its TCB -is a slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool presented at creation. +A **Thread** is an RFC-0003 object belonging to one Process, which RFC-0007 (proposed) provides; its TCB is a +slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool presented at creation. | Field | Meaning | |-------|---------| @@ -99,251 +84,218 @@ is a slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool | `priority`, `mcp` | base priority and maximum controlled priority (§6); both **0 at creation** | | `home_sc`, `sc`, `effective` | the bound context, if any; the context held now, own or donated; the priority run at now (§7) | | `bound_notification`, `ipc_buffer` | the notification a `recv` may also wake on (RFC-0004 §10); the IPC-buffer binding (RFC-0007, proposed) | -| `fault_ep`, `timeout_ep` | **two kernel-held capability slots in the TCB** *(seL4 MCS)*: the capability is moved in by its binder, generation-checked when a fault is raised, and fails closed if its endpoint died; the thread cannot close it | +| `fault_ep`, `timeout_ep` | **two kernel-held capability slots** *(seL4 MCS)*: moved in by the binder, generation-checked when a fault is raised, failing closed if the endpoint died; the thread cannot close them | | `fp_enabled`, `fp_state` | the FP/SIMD flag and save area (§10) | -**Every internal link is safe against slot reuse.** A queue link, `sc`, `home_sc`, a reply object's -record of a lent context, an `IrqHandler`'s notification and a core's FP owner are each either a counted -reference that holds the slot (RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare -index. This closes the ABA holes a slot array otherwise opens (§10's FP owner is the sharp one). +**Every internal link survives slot reuse.** A queue link, `sc`, `home_sc`, a reply object's record of a lent +context, an `IrqHandler`'s notification and a core's FP owner are each a counted reference holding the slot +(RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare index. | State | Meaning | Left by | |-------|---------|---------| | `Inactive` | created, suspended, or faulted with no handler; in no queue | `resume` | -| `Ready` | runnable with released budget; in its core's run queue | dispatch; `suspend`; context removed (→ `Unfunded`) | -| `Running` | the current thread on its core; in no queue | preemption, timeslice end, budget exhaustion (→ `Throttled`), block, `yield`, fault, `suspend`, `thread_exit`, context removed | -| `Throttled` | runnable, holding a context whose released budget is exhausted or below what its pending `call` needs; in the release queue | refill release; `suspend`; context removed | -| `Unfunded` | runnable but holding no context — a passive thread woken by a non-donating `send` or `notify`, one `resume`d with none, or a server left stranded (§7); in no queue | `bind_sc`, a donation, `suspend` | +| `Ready` | runnable with released budget; in its core's run queue | dispatch, `suspend`, context removed | +| `Running` | the current thread on its core; in no queue | preemption, timeslice end, exhaustion (→ `Throttled`), block, `yield`, fault, `suspend`, `thread_exit`, context removed (→ `Unfunded`) | +| `Throttled` | holds a context whose released budget is spent, or below what its pending `call` needs; in the release queue | refill release, `suspend`, context removed | +| `Unfunded` | runnable, no context: a passive thread woken by `send` or `notify`, `resume`d with none, or stranded (§7); in no queue | `bind_sc`, a donation, `suspend` | | `BlockedOnSend` | queued on an endpoint to `send` or `call` | rendezvous, endpoint destruction, `suspend` | | `BlockedOnRecv` | waiting on an endpoint (and any bound notification) | rendezvous, signal, `suspend` | | `BlockedOnReply` | waiting on a reply object after `call` or a fault message | reply, error completion (§7) | | `BlockedOnNotification` | waiting on a notification alone | signal, `suspend` | | `Exited` | finished; in no queue; the object lingers until destroyed | never | -Faults reuse IPC: the kernel `call`s `fault_ep` on the thread's behalf, the thread waits `BlockedOnReply`, -and RFC-0007 (proposed) defines the message. Fault and timeout messages bypass `min_budget`. Destroying a -thread bumps its generation (RFC-0003 §7), unlinks it in O(1) from whichever doubly linked queue holds it, -and destroys any reply object it waits on (RFC-0004 §8) — with §7's rule for a context it had lent. +Faults reuse IPC: the kernel `call`s `fault_ep` for the thread, which waits `BlockedOnReply`; RFC-0007 +(proposed) defines the message. Fault and timeout messages bypass `min_budget`. Destroying a thread bumps its +generation (RFC-0003 §7), unlinks it in O(1) from any doubly linked queue and destroys any reply object it waits +on (RFC-0004 §8); §7 governs a context it had lent. -Every operation is a method `call` on the **invoked** capability (RFC-0007 (proposed) verdict 1). Some -also name a second, **presented** capability, which is resolved in the caller's table and checked but -never moved and needs no `TRANSFER` — RFC-0007 (proposed) provides that encoding (seL4's `extraCaps`). +Every operation is a method `call` on the **invoked** capability (RFC-0007 (proposed) verdict 1). Some also +name a **presented** capability: resolved in the caller's table and checked, never moved, needing no +`TRANSFER` — an encoding RFC-0007 (proposed) provides *(seL4's `extraCaps`)*. The list is closed: | Operation | Invoked: right | Presented: right | Effect | |-----------|----------------|------------------|--------| | `set_priority(t, auth, p)` / `set_mcp(t, auth, m)` | `t`: `WRITE` | `auth`: any capability to a Thread, no right checked | value `≤ mcp(auth)` (§6) | | `bind_sc(t, sc)` / `unbind_sc(t)` | `t`: `WRITE` | `sc`: `WRITE` / — | active ↔ passive | -| `bind_notification(t, n)` / unbind | `t`: `WRITE` | `n`: `READ` | a `recv` also wakes on `n` | -| `set_fault_ep(t, ep)` / `set_timeout_ep(t, ep)` | `t`: `WRITE` | `ep`: `WRITE`, **moved** into the TCB slot | the caller derives first to keep a copy | -| `resume(t)` / `suspend(t)` | `t`: `WRITE` | — | `Inactive` ↔ runnable; a first `resume` closes the Process's grant window (RFC-0007) | -| `read_regs(t)` / `write_regs(t, …)` | `t`: `READ` / `WRITE` | — | frame and `TPIDR_EL0`/`FS_BASE` only, sanitised (§9) | +| `bind_notification(t, n)` / `unbind_notification(t)` | `t`: `WRITE` | `n`: `READ` / — | a `recv` also wakes on `n` | +| `set_fault_ep(t, ep)` / `set_timeout_ep(t, ep)` | `t`: `WRITE` | `ep`: `WRITE`, **moved** into the TCB | the binder derives first to keep a copy | +| `resume(t)` / `suspend(t)` | `t`: `WRITE` | — | `Inactive` ↔ runnable; a first `resume` closes the grant window (RFC-0007) | +| `read_regs(t)` / `write_regs(t, …)` | `t`: `READ` / `WRITE` | — | the frame, `TPIDR_EL0` and `FS_BASE` only, sanitised (§9) | | `set_fp(t, on)` | `t`: `WRITE` | — | §10 | -| `sc_configure(ctl, sc, B, T, refills, 0)` | `ctl`: `WRITE` | `sc`: `WRITE` | §5; the last word is the placement class, reserved zero (§12) | -| `sc_stats(sc)` / `ctl_info(ctl)` | `sc` / `ctl`: `READ` | — | §5's counters / §5's exclusivity check | -| `yield()` / `thread_exit()` | none — two of RFC-0003 §9's three authority-free exceptions | — | tail of own level / `Exited` | +| `sc_configure(ctl, sc, B, T, refills, 0)` | `ctl`: `WRITE` | `sc`: `WRITE` | §5; the last word, the placement class, is reserved zero (§12) | +| `sc_stats(sc)` / `ctl_info(ctl)` | `sc` / `ctl`: `READ` | — | §5's counters; the exclusivity check (§12) | +| `yield()` / `thread_exit()` | none: two of RFC-0003 §9's three authority-free exceptions | — | tail of own level / `Exited` | -No right joins RFC-0003's set. **The supervision cost:** a process holding `WRITE` on its own thread can -rebind that thread's `fault_ep`. A supervisor keeps the binding by handing the child no `WRITE` on threads -it created for it; threads the child creates itself are supervised through the Process (RFC-0007 open -question 1). `thread_exit` takes one `WRITE` on self for a TLS runtime to set `FS_BASE` via `write_regs`. +No right joins RFC-0003's set. **The supervision cost:** a process holding `WRITE` on its own thread can rebind +that thread's `fault_ep`. A supervisor keeps the binding by granting no `WRITE` on threads it creates for a +child; threads the child creates are supervised through the Process (RFC-0007 open question 1). A TLS runtime +sets `FS_BASE` through `write_regs`, so it needs `WRITE` on its thread — or its spawner sets it before `resume`. ## 5. Scheduling contexts and `SchedControl` — time as a capability (verdicts 3, 4) -**Options.** A timeslice on the thread (classic L4, Zircon) makes creating a thread *acquiring CPU*, with -nothing to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are -**subsumed**: a partition is what a `SchedControl` holder builds from several contexts. **Verdict sought:** -contexts as objects *(seL4 MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", -EuroSys 2018)*. - -A **SchedContext** holds budget `B` and period `T` (`B ≤ T`), at most eight refills, its core and its -bound thread. A thread with a bound context is **active**; one without is **passive** and runs only on -donated time (§7). **The ABI unit is the nanosecond** (`u64`) for `B`, `T` and `min_budget`, converted to -counter ticks once, in `sc_configure` and at endpoint creation — budgets round down, periods up — so no -per-machine frequency (62.5 MHz on QEMU's `cortex-a72`, a TSC rate on x86_64) enters the ABI. - -- **`SchedControl`, one per core, minted to the root task** (RFC-0007 (proposed) lists them in BootInfo), - has exactly two methods. `sc_configure` (`WRITE`) binds a context to that core with its parameters — - moving a context between cores is `sc_configure` through another core's `SchedControl`, and the first - context configured onto a parked core starts it (§12). `ctl_info` (`READ`) returns the number of contexts - configured on the core and the object's reference count, so a recipient promised exclusivity can check - it (§12). Topology is BootInfo *data* from RFC-0005 (proposed)'s discovery, not a method; interrupt - routing is not a `SchedControl` method (§11). No syscall names a core by number. *(seL4 `SchedControl`; - research/0002's "core-set capability".)* -- **`MIN_BUDGET`:** `B` below twice the published worst-case kernel path is refused — seL4 MCS's rule, - `2u * getKernelWcetTicks() * CONFIG_KERNEL_WCET_SCALE` (`include/kernel/sporadic.h`), without the scale. - The path is measured *between preemption points*, which is why RFC-0007 (proposed)'s console write must - take them (§11); a 44 ms console hold would otherwise make `MIN_BUDGET` 88 ms. -- **Full contexts** (`B = T`) are never throttled and round-robin with timeslice `T`. **Partial contexts** - (`B < T`) are sporadic servers: within any window of length `T` the context consumes at most `B`. Refills - fall due a period after consumption; when all eight are in use they merge, forfeiting bandwidth rather - than exceeding it. *(Sprunt, Sha & Lehoczky 1989.)* -- **What a budget means — caps, not minima (verdict 4).** research/0002 Part 6 asks for guaranteed - reservations with capping as broker policy. This kernel enforces the **cap**, because a sporadic cap is - what makes fixed-priority admission sound: a holder can prove a minimum only if every higher-priority - context is capped. The **minimum** follows from admission, the `SchedControl` holder's policy. Dispatch - is work-conserving *only among threads with released budget*: a partial context idles its core by - design when its budget is spent. Unused budget never carries past its window; there is no burst. -- **Accounting window and billing granularity**, stated as design (the QNX caveat, ECRTS '21): the window - is each context's own sliding period; billing is exact, from the counter (`CNTVCT_EL0`; the TSC) read at - every entry and exit — 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 62.5 MHz default (QEMU - `target/arm/tcg/cpu64.c`, `target/arm/cpu.c`). Kernel time, **interrupts included, is billed to the - context running on entry** — a cross-domain leak costed in §16 *(as seL4)*. -- **Stall counters from the first version** (research/0002 Part 6, after PSI and oomd): per context, time - consumed, time and count throttled, time ready but waiting, and budget expiries, read-only via - `sc_stats` — the O-17 supervisor's wedged-server signal. The kernel measures; userspace acts. -- **Lifecycle.** Destroying or reconfiguring a context (a) preempts the thread running on it — an IPI on - SMP; (b) clears every link to it — the thread's `sc` or `home_sc`, a reply object's record, its - release-queue entry — before its slot's count can reach zero; and (c) leaves affected threads - `Unfunded`. A reply to a reply object whose recorded context has gone moves nothing. -- **Accounting domain — decided: separate for Phase 1.** A context's *memory* is a slot charged to a Pool - (RFC-0005); its *time* comes only from a `SchedControl`. One domain holding both, as research/0002 - wants, is deferred to the broker RFC; the cost is a broker reasoning over two ledgers. This answers - RFC-0005 (proposed) open question 2. +**Options.** A timeslice on the thread (classic L4, Zircon) makes creating a thread *acquiring CPU*, with nothing +to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are **subsumed**: a +partition is what a `SchedControl` holder builds from contexts. **Verdict sought:** contexts as objects *(seL4 +MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", EuroSys 2018)*. + +A **SchedContext** holds budget `B` and period `T` (`B ≤ T`), at most eight refills, its core and its bound +thread. A thread with a bound context is **active**; one without is **passive** and runs only on donated time +(§7). **The ABI unit is the nanosecond** (`u64`) for `B`, `T` and `min_budget`, converted to counter ticks once, +in `sc_configure` and at endpoint creation — budgets round down, periods up — so no per-machine frequency +(62.5 MHz on QEMU's `cortex-a72`; a TSC rate on x86_64) enters the ABI. + +**What a budget means — caps, not minima (verdict 4).** research/0002 Part 6 asks for guaranteed reservations +with capping left to the broker. This kernel enforces the **cap**, because a sporadic cap is what makes +fixed-priority admission sound: a holder can promise a minimum only if every more urgent context is capped. The +**minimum** is admission, the `SchedControl` holder's policy. Dispatch is work-conserving *only among threads +with released budget*: a spent partial context leaves its core idle by design. No budget carries past its window. + +| Aspect | Rule | +|--------|------| +| **`SchedControl`** | one per core, minted to the root task and listed in BootInfo (RFC-0007, proposed). Two methods: `sc_configure` (`WRITE`) binds a context to that core with its parameters — moving a context is `sc_configure` through another core's `SchedControl`, and the first one onto a parked core starts it (§12); `ctl_info` (`READ`) returns the count of contexts on the core and the object's reference count. Topology is BootInfo *data* from RFC-0005 (proposed)'s discovery; routing is `IrqControl`'s (§11). No syscall names a core by number. *(seL4 `SchedControl`; research/0002's "core-set capability".)* | +| **`MIN_BUDGET`** | `B` below twice the published worst kernel path is refused — seL4 MCS's rule, `2u * getKernelWcetTicks() * CONFIG_KERNEL_WCET_SCALE` (`include/kernel/sporadic.h`), without the scale. The path is measured *between preemption points*, which is why RFC-0007's console write must take them (§11): its 44 ms hold would make `MIN_BUDGET` 88 ms. | +| **Full and partial** | full contexts (`B = T`) are never throttled and round-robin with timeslice `T`. Partial ones (`B < T`) are sporadic servers: no window of length `T` sees more than `B` consumed; refills fall due a period after use; when all eight are in use they merge, forfeiting bandwidth rather than exceeding it. *(Sprunt, Sha & Lehoczky 1989.)* | +| **Billing** | the QNX caveat stated as design (ECRTS '21): each context's window is its own sliding period; billing is exact, from `CNTVCT_EL0` or the TSC read at every entry and exit — 16 ns on QEMU's `cortex-a72`, which keeps the pre-Armv8.6 62.5 MHz default (`target/arm/tcg/cpu64.c`, `target/arm/cpu.c`). Kernel time, **interrupts included, is billed to the context running on entry** *(as seL4)* — a leak costed in §17. | +| **Stall counters** | from the first version (research/0002 Part 6, after PSI and oomd): per context, time consumed, time and count throttled, time ready but waiting, expiries; read through `sc_stats` — the O-17 supervisor's wedged-server signal. | +| **Lifecycle** | destroying or reconfiguring a context preempts the thread running on it (an IPI on SMP), clears every link to it — `sc`, `home_sc`, a reply object's record, its release-queue entry — before its slot count can reach zero, and leaves those threads `Unfunded`. A reply whose recorded context has gone moves nothing. | +| **Accounting domain** | **decided: separate in Phase 1.** A context's memory is a slot charged to a Pool (RFC-0005); its time comes only from a `SchedControl`. One domain holding both is the broker RFC's; the cost is a broker reasoning over two ledgers. This answers RFC-0005 (proposed) open question 2. | ## 6. Priorities and dispatch (verdicts 5, 6) -**256 levels**, 255 most urgent, meaningful within a core. `set_priority(t, auth, p)` needs `WRITE` on -`t`, any capability to the Thread `auth`, and `p ≤ mcp(auth)`; MCP is set the same way, so authority over -urgency only narrows as it is delegated — O-2's shape *(seL4 `TCB_SetPriority`, whose authority is any -TCB capability)*. **Every thread is born at priority 0 and MCP 0**, so a fresh thread is no authority; -the root task's thread starts at priority 255 and MCP 255 with a full context on core 0, the one -bootstrap act (RFC-0007 (proposed) §9). To delegate priority authority, derive a **rightless** capability -to a thread with the MCP wanted — it confers the ceiling and nothing else. - -**Rejected: `READ` or `WRITE` on `auth`** — each would make priority authority and register access -(`read_regs`, `write_regs`) the same token. **Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all -right in embryo, one bit whose meaning grows with every scheduling operation (the CAP_SYS_ADMIN grave). -**Rejected: dynamic priorities** (CFS, BSD decay, boosts): kernel-resident policies churned while -accounting and enforcement endured (research/0002 Part 1). **The cost:** any capability to a high-MCP -thread is priority authority, so such threads' capabilities must be handed out with care. +**256 levels**, 255 most urgent, meaningful within a core. `set_priority(t, auth, p)` needs `WRITE` on `t`, any +capability to the Thread `auth`, and `p ≤ mcp(auth)`; MCP is set alike, so authority over urgency only narrows +as it is delegated — O-2's shape *(seL4 `TCB_SetPriority`, whose authority is any TCB capability)*. **Every +thread is born at priority 0 and MCP 0**, so a fresh thread is no authority; the root's thread starts at 255 and +255 with a full context on core 0, the one bootstrap act (RFC-0007 (proposed) §9). To delegate priority +authority alone, derive a **rightless** capability to a thread holding the MCP wanted. + +**Rejected: `READ` or `WRITE` on `auth`** — either makes priority authority and register access the same token. +**Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all right in embryo, one bit whose meaning grows with +every scheduling operation (the CAP_SYS_ADMIN grave). **Rejected: dynamic priorities** (CFS, BSD decay): +kernel-resident policies churned while accounting and enforcement endured (research/0002 Part 1). **The +cost:** any capability to a high-MCP thread is priority authority, so such capabilities are handed out sparingly. - **Run queues.** Per core, 256 intrusive FIFO lists and a 256-bit bitmap: the next thread is a - count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; - one whose timeslice ended, at the tail. The running thread is never queued, and a thread switched to - directly is never enqueued at all. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* -- **Release queue.** Per core, sorted by next refill: O(n) in contexts only the `SchedControl` holder adds. -- **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has - released budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority - is at least the highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also - strictly more urgent than *C*. Otherwise *W* is queued. A *W* without released budget goes to the - release queue; one on another core is enqueued there and sent an IPI (§12). + count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; one + whose timeslice ended, at the tail. The running thread is never queued, nor is a thread switched to + directly. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* **Release queue:** per core, sorted by next + refill, O(n) in contexts only the `SchedControl` holder adds. +- **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has released + budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority is at least the + highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also strictly more urgent than + *C*. Otherwise *W* is queued; without released budget, it goes to the release queue; on another core, it is + enqueued there and sent an IPI (§12). - **Endpoint queues are priority-ordered, first-come within a level** — seL4 MCS's `tcbAppend` - (`include/object/tcb.h`, "priority ordered endpoint or notification queue"), which walks back from the - tail; the FIFO `tcbEPAppend` is compiled only without MCS. The walk is O(n) with interrupts masked, *n* - bounded by the thread array's fixed capacity (RFC-0005 §5) and counted in the published worst path; a - `set_priority` on a queued thread repositions it. **Rejected: FIFO** *(classic L4, non-MCS seL4)*: O(1), - but a process could create threads from its own Pool and queue *N* calls ahead of a more urgent client - of a shared server — unbounded inversion the server cannot reorder. + (`include/object/tcb.h`, "priority ordered endpoint or notification queue"), which walks back from the tail; + the FIFO `tcbEPAppend` is compiled only without MCS. The walk is O(n) with interrupts masked, *n* bounded by + the thread array's fixed capacity (RFC-0005 §5) and counted in the worst path; `set_priority` repositions a + queued thread. **Rejected: FIFO** *(classic L4, non-MCS seL4)*: O(1), but a process could create threads from + its own Pool and queue *N* calls ahead of a more urgent client of a shared server, which cannot reorder them. ## 7. Donation, inheritance and budget expiry (verdicts 7, 8) -**Donation.** When *C* `call`s and the receiver *S* is **passive**, the kernel moves *C*'s current -context to *S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves -it back. *S* runs at `max(prio(S), eff(C))`, **fixed when the donation is made** and not propagated if -*C*'s priority later changes — O(1), one hop at a time. Lending priority is safe because time is lent with -it. *(Ford & Lepreau, migrating threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* - -**An active receiver receives no donation (amendment A1, §13).** It runs on its own context at its own -priority, as in seL4 MCS (`reply_push`, `src/object/reply.c`, donates only to a receiver with no context): -a thread cannot hold two contexts, and an active server has chosen to pay for itself. RFC-0004 §4 and §8 -say `call` lends unconditionally, so this is an amendment. **The cost:** an active server gets no -inheritance, and a low-priority active server blocks high-priority clients. Constitution §3's -inheritance therefore holds for passive servers; **a server shared across trust domains must run passive, -or active at a priority no lower than its most urgent client** — the ceiling, now required, not optional. - -**Priority inversion, bounded.** Priority-ordered queues serve the most urgent waiter next; a waiter -still waits for the request in service, which runs on its client's context and can be preempted by a -middle-priority thread. `prio(S)` is therefore a **ceiling floor**: set to the highest client priority, -it bounds inversion to one request *(the priority-ceiling bound seL4 MCS relies on)*. **Rejected:** -boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess which thread -serves an endpoint — policy — and walk unbounded chains with interrupts masked. - -**Caller death never unfunds a server.** If *C* exits or is killed while *S* serves it, *R* is destroyed -but the context stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails -`PeerGone`); the context then detaches, `Unfunded` for no thread, and only its capability holder may -rebind or destroy it. Only a context's holder can pull time from a server, never a client's -`thread_exit` *(seL4 `reply_remove_tcb` likewise leaves the context with the server)*. - -The 2026-08-01 amendment, discharged as amended, after seL4 RFC-14 (Mitchell Johnston, "Budget limit +**Donation.** When *C* `call`s and the receiver *S* is **passive**, the kernel moves *C*'s current context to +*S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves it back. *S* runs +at `max(prio(S), eff(C))`, **fixed when the donation is made**, not propagated if *C*'s priority later changes — +O(1), one hop at a time. Lending priority is safe because time is lent with it. *(Ford & Lepreau, migrating +threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* + +**An active receiver gets no donation (amendment A1).** It runs on its own context at its own priority, as in +seL4 MCS (`reply_push`, `src/object/reply.c`, donates only to a receiver with none): a thread holds one context, +and an active server has chosen to pay for itself. RFC-0004 §4 and §8 say `call` lends unconditionally, so this +amends them. **The cost:** an active server inherits nothing, so a low-priority one blocks urgent clients. +Constitution §3's inheritance holds for passive servers; **a server shared across trust domains runs passive, or +active at no lower priority than its most urgent client.** + +**Inversion, bounded.** Priority-ordered queues serve the most urgent waiter next, but it still waits for the +request in service, which runs on its client's context and can be preempted by a middle-priority thread. +`prio(S)` is therefore a **ceiling floor**; set to the highest client priority it bounds inversion to one +request — required, not optional, for shared servers *(the priority-ceiling bound seL4 MCS relies on)*. +**Rejected:** boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess which +thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. + +**Caller death never unfunds a server.** If *C* exits or is killed mid-request, *R* is destroyed but the +context stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails `PeerGone`); then it +detaches, and only its capability holder may rebind or destroy it. A client's `thread_exit` never pulls time +from a server *(seL4 `reply_remove_tcb` likewise leaves the context with the server)*. + +**The 2026-08-01 amendment, discharged as amended,** after seL4 RFC-14 (Mitchell Johnston, "Budget limit thresholds on endpoints for SC Donation", seL4/rfcs pull request 24, open): -- **Prevention — `min_budget`.** An endpoint carries `min_budget`, fixed at creation (0 means none; - RFC-0007 (proposed) encodes it). A `call` needs released budget of at least `min_budget + MIN_BUDGET`, - the margin paying for the `call` and `reply` themselves — **this RFC's rule**; RFC-14's exact comparison - is to be quoted before acceptance. A context whose `B` can never meet it fails `BudgetRefused`; a - non-donating `send` to such an endpoint is invalid. With `min_budget` at the server's WCET, expiry - inside it is a true error. -- **A caller short *now* waits (amendment A2).** The amendment says donation below the threshold is - "refused up front". Proposed instead: a partial context that cannot pass now has its refills merged - and deferred, waits `Throttled`, and the call restarts when the head refill suffices — free in an event - kernel, O(refills) ≤ 8, and the deferral the RFC-14 discussion describes. A **full** context short of - its slice ends its timeslice: tail of its level, a fresh slice, the call restarts. No donation happens - below the bar, so the amendment's purpose holds; refusing a transient shortfall would make every client - spin to retry. -- **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* - runs on it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with - `BudgetExpired` (observed when the context next refills); *R* is consumed, so a late `reply` fails - closed. *S* is left `Unfunded`, registers intact, and the expiry counter rises. If `timeout_ep` holds a - live endpoint, the kernel sends a **timeout fault** carrying consumed time and donating nothing (seL4 - MCS sends timeout faults without donation); the handler, active with time of its own, abandons the - request (`write_regs`) or lends *S* a context to finish. The message carries **no identity**: badges - are reserved zero until RFC-0003a, so a handler binds one timeout endpoint per server. -- **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply - to A with an error; unwinding to A would walk the reply stack — O(n) with interrupts masked. -- **With `min_budget` 0 there is no prevention:** one client can strand a shared server for every client - until a handler funds it (O-26, A5). Zero is for servers with one client or a supervisor. - -**Rejected: the unwind without timeout faults**, where the stranded server is visible only through its -state and counters; a supervisor woken by a message beats one polling. **Deferred: RFC-14's budget -limits** (capping what a server spends of a donated context): the RFC-14 discussion reports 22% extra -fastpath cost; the per-operation figures are to be re-checked before this is revisited (Open question 3). +- **Prevention — `min_budget`,** fixed at endpoint creation (0 for none; RFC-0007 (proposed) encodes it). A + `call` needs released budget of at least `min_budget + MIN_BUDGET`, the margin paying for the `call` and + `reply` themselves — **this RFC's rule**; RFC-14's exact comparison is to be quoted before acceptance. A + context whose `B` can never meet it fails `BudgetRefused`; a non-donating `send` to such an endpoint is + invalid. With `min_budget` at the server's WCET, expiry inside it is a true error. +- **A caller short *now* waits (amendment A2).** The amendment says donation below the threshold is "refused up + front". Proposed: a partial context short now has its refills merged and deferred, waits `Throttled`, and the + call restarts when the head refill suffices — free in an event kernel, O(refills) ≤ 8, the deferral the + RFC-14 discussion describes. A **full** context short of its slice ends its timeslice and the call restarts + on the next. Nothing is donated below the bar either way; refusal would make clients spin. +- **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* runs on + it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with `BudgetExpired` + (seen when the context next refills); *R* is consumed, so a late `reply` fails closed. *S* is left + `Unfunded`, registers intact; the expiry counter rises. If `timeout_ep` is live, the kernel sends a **timeout + fault** carrying consumed time and donating nothing (seL4 MCS sends timeout faults without donation); the + handler, active on its own time, abandons the request (`write_regs`) or lends *S* a context to finish. It + carries **no identity** — badges are reserved zero until RFC-0003a — so a handler binds one endpoint per server. +- **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply to A + with an error; unwinding to A would walk the reply stack — O(n) with interrupts masked. +- **`min_budget` 0 prevents nothing:** one client can strand a shared server for all its clients until a + handler funds it (O-26, A5). Zero suits a server with one client, or one under a supervisor. + +**Rejected: the unwind without timeout faults** — a supervisor woken by a message beats one polling counters. +**Deferred: RFC-14's budget limits** (capping what a server spends of a donated context): the RFC-14 +discussion reports 22% extra fastpath cost; the per-operation figures are to be re-checked (Open question 3). ## 8. The timer and preemption (verdict 9) -One HAL surface: `now() -> Ticks`, `frequency() -> Hz`, `set_deadline(Ticks)`, `cancel()`, and a -`timer_fired()` up-call. The kernel is **tickless**: each exit programs one per-core deadline — the -earliest of a partial context's exhaustion, the timeslice boundary if an equal-priority thread is ready, -and the next refill due. A lone *full* context arms nothing; a lone partial one still arms its exhaustion. +One HAL surface: `now() -> Ticks`, `frequency() -> Hz`, `set_deadline(Ticks)`, `cancel()`, and a `timer_fired()` +up-call. The kernel is **tickless**: each exit programs one per-core deadline — the earliest of a partial +context's exhaustion, the timeslice end if an equal-priority thread is ready, and the next refill due. A lone +full context arms nothing; a lone partial one still arms its exhaustion. - **AArch64: the EL1 virtual timer** — `CNTV_CVAL_EL0`, `CNTV_CTL_EL0` (`ENABLE`, `IMASK`, `ISTATUS`), - `CNTVCT_EL0`, and `CNTFRQ_EL0` always read, never assumed. PPI INTID 27, the Arm BSA assignment QEMU's - `virt` uses (`include/hw/arm/bsa.h`). Preferred to the physical timer, whose EL1 access an EL2 beneath - us can trap (`CNTHCTL_EL2`); any EL2-to-EL1 descent must grant EL1 timer access and zero `CNTVOFF_EL2`. - *(Arm ARM DDI 0487, "The Generic Timer".)* + `CNTVCT_EL0`, and `CNTFRQ_EL0` always read, never assumed; PPI INTID 27, the Arm BSA assignment `virt` uses + (QEMU `include/hw/arm/bsa.h`). Preferred to the physical timer, whose EL1 access an EL2 beneath can trap + (`CNTHCTL_EL2`); any EL2-to-EL1 descent grants EL1 timer access and zeroes `CNTVOFF_EL2`. *(Arm ARM DDI + 0487, "The Generic Timer".)* - **x86_64: the local APIC timer**, x2APIC where `CPUID.01H:ECX[21]` reports it, enabled by - `IA32_APIC_BASE.EXTD` (bit 10). TSC-deadline mode (LVT mode `10b` in bits 18:17; `IA32_TSC_DEADLINE`, - MSR `0x6E0`) only when `CPUID.01H:ECX[24]` **and** invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; - otherwise one-shot through the initial-count register, calibrated at boot, a past or zero deadline - clamped to a count of 1 since 0 stops the timer. QEMU's TCG lacks TSC-deadline (`target/i386/cpu.c`), so - one-shot is what the emulator runs, not dead code. The console prints the mode. *(Intel SDM Vol. 3A.)* -- **Kernel-reserved; this RFC owns these bits.** `SCTLR_EL1.nTWI` (bit 16) `= 0`: EL0 `wfi` traps (EC - `0x01`) and completes as `yield`, advancing the saved PC by 4, since `ELR_EL1` holds the `wfi` itself. - `nTWE` (bit 18) `= 1`: EL0 `wfe` does not trap, so spin-lock backoff costs no kernel entry. `UMA` (bit 9) - `= 0`; `SA` (bit 3) `= 1`. `CNTKCTL_EL1` and `PMUSERENR_EL0` are written 0 at boot — no EL0 counter, - timer or PMU — because their reset values are not architecturally fixed. `hlt` at CPL 3 faults; `CR4.TSD` - is set. RFC-0005 (proposed) writes `SCTLR_EL1` whole and must carry these values; its §13 step 5 today - says `nTWE = 0` and needs correcting. -- **Userspace timers (RFC-0004 §9.3), named now, built later.** RFC-0004 §8 calls the watchdog - load-bearing, and the timer is kernel-reserved, so the kernel must provide the source: a **deadline - object**, bound to a notification, armed with an absolute time in nanoseconds, kept in the per-core - release queue beside refills, charged to a Pool, and signalled on expiry *(Zircon's timer object)*. One - armed deadline per object; the sorted insert is bounded by the deadline array's capacity. **Rejected:** - minting a second hardware timer — the EL1 physical timer (PPI 30) exists per core on AArch64, but x86_64 - has no second per-core timer without the HPET, so the source would differ per Tier-1 target. + `IA32_APIC_BASE.EXTD` (bit 10). TSC-deadline mode (LVT mode `10b`, bits 18:17; `IA32_TSC_DEADLINE`, MSR + `0x6E0`) only when `CPUID.01H:ECX[24]` **and** invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; otherwise + one-shot through the initial-count register, calibrated at boot, a past deadline clamped to a count of 1 + since 0 stops the timer. QEMU's TCG lacks TSC-deadline (`target/i386/cpu.c`), so one-shot is what the + emulator runs. The console prints the mode. *(Intel SDM Vol. 3A.)* +- **Userspace timers (RFC-0004 §9.3), named now, built later.** RFC-0004 §8 calls the watchdog load-bearing, and + the timer is kernel-reserved, so the kernel provides the source: a **deadline object**, bound to a + notification, armed with an absolute time in nanoseconds, kept in the per-core release queue beside refills, + charged to a Pool, signalled on expiry *(Zircon's timer object)*; its sorted insert is bounded by the deadline + array's capacity. **Rejected:** minting a second hardware timer — AArch64 has the EL1 physical timer (PPI 30) + per core, but x86_64 has no second per-core timer short of the HPET, so the source would differ per target. + +**Kernel-reserved; this RFC owns these bits.** RFC-0005 (proposed) writes `SCTLR_EL1` whole and must carry +them; its §13 step 5 says `nTWE = 0` today and needs correcting. + +| Register | Value | Why | +|----------|-------|-----| +| `SCTLR_EL1.nTWI` (bit 16) | 0 | EL0 `wfi` traps (EC `0x01`) and completes as `yield`; `ELR_EL1` holds the `wfi` itself, so the saved PC advances by 4 | +| `SCTLR_EL1.nTWE` (bit 18) | 1 | EL0 `wfe` does not trap, so spin-lock backoff costs no kernel entry; the `sev` residual is in §15's O-9 row | +| `SCTLR_EL1.UMA` (bit 9), `SA` (bit 3) | 0, 1 | no EL0 DAIF access; `SP` alignment checked at EL1 (§9) | +| `CNTKCTL_EL1`, `PMUSERENR_EL0` | 0 | no EL0 counter, timer or PMU; written at boot because their reset values are not architecturally fixed | +| x86_64 `CR4.TSD`; `hlt` | set; faults at CPL 3 | the same denials on the second target | ## 9. The context switch — what is saved, where -In an event kernel a context switch is *which frame the exit path restores*; assembly is confined to the -entry and exit stubs in `kernel/src/arch/**`. **The user frame lives in the TCB.** On AArch64, while a -thread runs at EL0, `SP_EL1` points at the end of its TCB frame, so the entry stub's stores land there -before it loads the per-core kernel stack from the block `TPIDR_EL1` addresses; `SCTLR_EL1.SA` is on and -the frame 16-byte aligned, and the current-EL `SP_ELx` synchronous vector reports a frame overwrite if -`SP` lies inside a TCB. On x86_64 an interrupt from ring 3 loads `RSP` from `TSS.RSP0`, pointing at the -TCB frame; **`syscall` does not switch `RSP`**, so its stub runs `swapgs`, parks the user `rsp` in per-CPU -scratch, loads the TCB frame pointer, saves into the frame, and only then loads the kernel stack. RFC-0007 -(proposed) owns both entry sequences; this RFC owns the layout and the current-EL IRQ stub (§3). The -address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit when -the process differs — where a time-protection flush would go (Ge et al., EuroSys 2019). +In an event kernel a context switch is *which frame the exit path restores*; assembly is confined to the entry and +exit stubs in `kernel/src/arch/**`. **The user frame lives in the TCB.** On AArch64, while a thread runs at EL0, +`SP_EL1` points at the end of its TCB frame, so the entry stub's stores land there before it loads the per-core +kernel stack from the block `TPIDR_EL1` addresses; with `SA` on and the frame 16-byte aligned, the current-EL +`SP_ELx` synchronous vector reports a frame overwrite if `SP` lies inside a TCB. On x86_64 an interrupt from +ring 3 loads `RSP` from `TSS.RSP0`, the TCB frame; **`syscall` does not switch `RSP`**, so its stub runs +`swapgs`, parks the user `rsp` in per-CPU scratch, loads the TCB frame pointer, saves into it, then loads the +kernel stack. RFC-0007 (proposed) owns both entry sequences; this RFC owns the layout and §3's current-EL IRQ +stub. The address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit +when the process differs — where a time-protection flush would go (Ge et al., EuroSys 2019). | What | AArch64 | x86_64 | |------|---------|--------| @@ -354,115 +306,107 @@ the process differs — where a time-protection flush would go (Ge et al., EuroS | Never | kernel registers, debug and PMU state | the same | | **TCB size** | 36-word frame (288 B) + FP area padded to 528 B + ~180 B of fields: **1 KiB** | 23-word frame (184 B, padded to 192) + 832 B XSAVE + ~180 B: **1.25 KiB** | -- **O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` - (EL0t, DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only - status flags (IOPL 0, IF set), and a non-canonical `RIP` or base is refused. It writes `TPIDR_EL0` and - `FS_BASE` only: `TPIDRRO_EL0` and the user GS base are kernel-owned IPC-buffer discovery (RFC-0007 §5). -- NMI, `#DB`, `#DF` and `#MC` run on IST stacks, as RFC-0007 (proposed) asks: they can arrive in the - `syscall` window before `RSP` is switched. AArch64 has no such window. +**O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` (EL0t, +DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only status flags +(IOPL 0, IF set), and a non-canonical `RIP` or base is refused. It writes `TPIDR_EL0` and `FS_BASE` only: +`TPIDRRO_EL0` and the user GS base are the kernel's IPC-buffer discovery registers (RFC-0007 §5). NMI, `#DB`, +`#DF` and `#MC` run on IST stacks (RFC-0007 asks): they can arrive before `syscall`'s stack switch. ## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (verdict 10) -- **Enabled per thread** by `set_fp`, off by default. `CLAUDE.md` says "per process"; per thread is finer - so an FP-off server thread in an FP-using process never forces a save — the § Architectures wording - changes with this RFC. A thread with FP off that touches FP takes an ordinary fault. -- **Loaded on first use, saved lazily.** Each core records an FP *owner* as an `(index, generation)` pair; - every other thread runs with access trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter - already names) or `CR0.TS = 1` (`#NM`, vector 7). `FPEN = 0b00` traps EL1 too and `XSAVE` raises `#NM` - while `TS` is set, so the trap handler runs: `FPEN = 0b01` (EL1 permitted, EL0 still trapped) and `isb`, - or `clts`; save the owner's registers if the owner is live; load the new owner's, or zeros on first use; - `FPEN = 0b11`; `isb`; return. A `call` to an FP-off server saves nothing. -- **The owner is cleared** — its registers unsaved — when the owner thread is destroyed or `set_fp(off)`, - so a thread reusing its slot finds a stale generation and loads zeros, never a dead thread's registers. -- **The kernel's only FP instructions** are those save and restore stubs in `kernel/src/arch/**`, - executed only with access granted; the soft-float target is the guarantee for generated code. No `FPEN` - value traps EL1 while allowing EL0, so while an owner runs a stray kernel FP instruction would not trap. -- **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format - area (512 legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. -- **The known cost — LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a - non-owner can read the owner's registers speculatively before `#NM` resolves. Hardware side channels are - out of scope (threat model §7); the cure is Open question 2, costed in §16. +- **Enabled per thread** by `set_fp`, off by default. `CLAUDE.md` says "per process"; per thread is finer, so an + FP-off server thread in an FP-using process never forces a save — its § Architectures wording changes with + this RFC. A thread with FP off that touches FP takes an ordinary fault. +- **Loaded on first use, saved lazily.** Each core records an FP *owner* as an `(index, generation)` pair; every + other thread runs trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or + `CR0.TS = 1` (`#NM`, vector 7). `0b00` traps EL1 too and `XSAVE` raises `#NM` while `TS` is set, so the + handler first grants itself access — `FPEN = 0b01` (EL1 permitted, EL0 still trapped) and `isb`, or `clts` — + then saves a live owner's registers, loads the new owner's or zeros, sets `FPEN = 0b11`, `isb`, and returns. + A `call` to an FP-off server saves nothing. +- **The owner is cleared,** its registers unsaved, when the owner thread is destroyed or `set_fp(off)`; a thread + reusing its slot then fails the generation check and loads zeros, never a dead thread's registers (O-8, O-9). +- **The kernel's only FP instructions** are those save and restore stubs in `kernel/src/arch/**`, run only with + access granted; the soft-float target is the guarantee for generated code. No `FPEN` value traps EL1 while + allowing EL0, so while an owner runs, a stray kernel FP instruction would not trap. +- **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format area (512 + legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. +- **LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a non-owner can read the + owner's registers speculatively before `#NM` resolves. Out of scope (threat model §7); Open question 2. ## 11. Interrupts — the controller, and device IRQs as notifications (verdict 11) -The kernel owns the controller and nothing behind it. *(seL4 `IRQControl`/`IRQHandler`.)* +The kernel owns the controller and nothing behind it *(seL4 `IRQControl`/`IRQHandler`)*. -- **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. - Kernel-reserved lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. - `bind(irq, notification, bits)` invokes the `IrqHandler` (`WRITE`) and presents the notification - (`WRITE`); `ack(irq)` needs `WRITE`. **Phase 1 routes every line to the boot core**; the SMP increment - decides routing as an `IrqControl` concern, never a `SchedControl` method. Shared PCI INTx lines on - `q35` are **unsupported** until MSI — a Phase-2 cost on the second Tier-1 target. +- **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. Kernel-reserved + lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. `bind(irq, + notification, bits)` invokes the `IrqHandler` (`WRITE`) and presents the notification (`WRITE`); `ack(irq)` + needs `WRITE`. **Phase 1 routes every line to the boot core**; SMP routing is `IrqControl`'s, never a + `SchedControl` method (Open question 6). Shared PCI INTx lines on `q35` are **unsupported** until MSI. - **Delivery.** The kernel acknowledges, **masks the line**, signals end of interrupt, ORs `bits` into the - notification and wakes a waiter through §6's dispatcher; the driver's `ack` unmasks. A level-triggered - line cannot storm, but can fire once per `ack` (§16). A destroyed notification leaves the line masked - (fail closed); the `IrqHandler` outlives a crashed driver, and a restarted one re-binds (O-17). -- **AArch64: GICv3 first.** Distributor: `GICD_CTLR` with `ARE` and `EnableGrp1` (`virt` without - `secure=on` has one security state, `DS = 1`); SPIs through `GICD_ISENABLER`/`ICENABLER`, - `GICD_IPRIORITYR` and `GICD_IROUTER`. Redistributor: woken by `GICR_WAKER`; SGIs and PPIs through - `GICR_ISENABLER0` and `GICR_IPRIORITYR`. Every enabled line is put in Group 1 explicitly (Group 0 would - arrive as FIQ) at a priority numerically below the `ICC_PMR_EL1` mask. CPU interface: `ICC_SRE_EL1.SRE`, - `ICC_IGRPEN1_EL1`, acknowledge `ICC_IAR1_EL1`, end `ICC_EOIR1_EL1`. On `virt`: distributor - `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`), reached through RFC-0005 - (proposed)'s MMIO window. *(Arm IHI 0069.)* **Cost:** `virt` picks GICv2 at eight CPUs or fewer - (`finalize_gic_version_do`), so xtask gains `gic-version=3`; the Raspberry Pi 4 in Constitution §6's - bring-up order has a GIC-400 (GICv2) — a **known** second driver. -- **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask - bit; end of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** - without IOMMU interrupt remapping a device can aim any vector at any core — O-18's precondition again. -- **Latency** is §3's worst path between preemption points. RFC-0007 (proposed)'s console write must take - one per FIFO-full poll, restartable by byte count; until it does, it holds the kernel ~44 ms per 504 - bytes on real hardware (none on QEMU, which has no baud delay). + notification and wakes a waiter through §6's dispatcher; the driver's `ack` unmasks. A level-triggered line + cannot storm, though it fires once per `ack` (§17). A destroyed notification leaves the line masked (fail + closed); the `IrqHandler` outlives a crashed driver, and a restarted one re-binds (O-17). +- **AArch64: GICv3 first.** `GICD_CTLR` gets `ARE` and `EnableGrp1` (`virt` without `secure=on` has one security + state, `DS = 1`); SPIs through `GICD_ISENABLER`, `GICD_IPRIORITYR`, `GICD_IROUTER`; the redistributor + woken by `GICR_WAKER`, SGIs and PPIs through `GICR_ISENABLER0` and `GICR_IPRIORITYR`. Each enabled line + goes in Group 1 explicitly (Group 0 arrives as FIQ), at a priority numerically below `ICC_PMR_EL1`'s mask. CPU + interface: `ICC_SRE_EL1.SRE`, `ICC_IGRPEN1_EL1`, `ICC_IAR1_EL1`, `ICC_EOIR1_EL1`. On `virt`: distributor + `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`), through RFC-0005 (proposed)'s MMIO + window. *(Arm IHI 0069.)* **Cost:** `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), + so xtask gains `gic-version=3`; the Raspberry Pi 4 has a GIC-400 (GICv2) — a **known** second driver. +- **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask bit; end + of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** without IOMMU + interrupt remapping a device can aim any vector at any core — O-18's precondition. +- **Latency** is §3's worst path between preemption points. RFC-0007 (proposed)'s console write must take one + per FIFO-full poll, restartable by byte count; until then it holds the kernel about 44 ms per 504 bytes on + real hardware, and nothing on QEMU, which has no baud delay. ## 12. Multi-core shape, placement and exclusive cores (verdict 11) Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. -- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds, so - donation can move a server thread between cores. **The kernel never load-balances.** -- **Cross-core wake-ups** (RFC-0004 §9.2): enqueue *W* on its core and send an IPI — an SGI through - `ICC_SGI1R_EL1`; a fixed vector through the LAPIC ICR. Direct switch is same-core only. -- **One big kernel lock when SMP lands**, taken on every entry, the IPC fast path included *(Peters, - Danis, Elphinstone & Heiser, "For a Microkernel, a Big Lock Is Fine", APSys 2015)*. This **rejects the - premise** of RFC-0003 §14.2, which asks for table sharing without a global lock on the fast path; within - one entry the resolve→check→act window holds under the lock (§3 covers restarts). A dated note against - RFC-0003 records it (§13). -- **A core waiting for the lock services IPIs while it spins** *(as seL4's CLH lock does)*, because two - operations need a synchronous answer from another core: RFC-0005 (proposed)'s x86_64 TLB shootdown (it - has no broadcast invalidate) and **FP state live on another core** — when a thread whose registers are - owned lazily on core X next needs them on core Y, Y asks X by IPI to save them. Cost: a cross-core FP - handoff is an IPI round trip. -- **Cores are parked until funded (default-deny).** On QEMU `virt` with `-kernel`, the DTB advertises PSCI - and secondaries start powered off (checked on QEMU 8.2.2: CPU 1 never leaves the entry address), so - `boot.s`'s `wfe` park loop only matters on spin-table firmware. The first `sc_configure` onto a core - starts it — PSCI `CPU_ON`, conduit from the DTB's `/psci` node; INIT-SIPI-SIPI on x86_64. Deferred to - increment 14; `boot.s`'s comment changes with it. -- **SMT placement invariant — one sentence now** (research/0002 Part 6, after L1TF and MDS): a context - will carry a placement class set only by the `SchedControl` holder; SMT siblings never run different - classes and donation never crosses one. The field is reserved zero in `sc_configure` until increment 14, - so no thread can label itself; SMT-off is "never fund the sibling". -- **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one - thread, a full context, nothing else on the core. The bitmap has one bit, the timer is never armed, no - IPI arrives unless the `SchedControl` holder acts, and lines are routed elsewhere; counters advance only - at the next kernel entry. A recipient checks the promise with `ctl_info`: one context, reference count 1. - -## 13. Amendments this RFC proposes to accepted documents (verdict 12) - -Each, if accepted, is logged in `docs/CHANGELOG.md` under Changed and appended to the target as a dated -amendment — never a silent edit. +- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds, so donation + can move a server thread between cores. **The kernel never load-balances.** Cross-core wake-ups (RFC-0004 + §9.2) enqueue *W* on its core and send an IPI — an SGI through `ICC_SGI1R_EL1`; a fixed vector through the + LAPIC ICR. Direct switch is same-core only. +- **One big kernel lock when SMP lands**, taken on every entry, the IPC fast path included *(Peters, Danis, + Elphinstone & Heiser, "For a Microkernel, a Big Lock Is Fine", APSys 2015)*. This **rejects the premise** of + RFC-0003 §14.2, which asks for sharing without a global lock on the fast path; within one entry the + resolve→check→act window holds under the lock, and §3 covers restarts (amendment A4). +- **A core waiting for the lock services IPIs while it spins** *(as seL4's CLH lock does)*, because two operations + need a synchronous answer from another core: RFC-0005 (proposed)'s x86_64 TLB shootdown, which has no + broadcast, and **FP state live on another core** — a thread whose registers core X owns lazily, run next on + core Y, has Y ask X by IPI to save them. Cost: a cross-core FP handoff is an IPI round trip. +- **Cores are parked until funded (default-deny).** On QEMU `virt` with `-kernel` the DTB advertises PSCI and + secondaries start powered off — checked on QEMU 8.2.2: CPU 1 never leaves the entry address — so `boot.s`'s + `wfe` loop matters only on spin-table firmware. The first `sc_configure` onto a core starts it: PSCI `CPU_ON`, + conduit from the DTB's `/psci` node; INIT-SIPI-SIPI on x86_64. Deferred to increment 14, `boot.s`'s comment + with it. +- **SMT placement, one sentence now** (research/0002 Part 6, after L1TF and MDS): a context will carry a class + set only by the `SchedControl` holder; SMT siblings never run different classes, and donation never crosses + one. The field is reserved zero until increment 14, so no thread labels itself; SMT-off is "never fund the + sibling". +- **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one thread, + a full context, nothing else on the core. The bitmap has one bit, no timer is armed, no IPI arrives unless the + `SchedControl` holder acts, lines are routed elsewhere, and counters advance only at the next kernel entry. A + recipient checks the promise with `ctl_info`: one context, reference count 1. + +## 13. Amendments proposed to accepted documents (verdict 12) + +Each, if accepted, is appended to its target as a dated amendment and logged in `docs/CHANGELOG.md` under Changed. | # | Target | Change | Why | |---|--------|--------|-----| | A1 | RFC-0004 §4, §8 | `call` donates only to a **passive** receiver; an active one runs on its own context | a thread holds one context; seL4 MCS `reply_push`; cost in §7 | -| A2 | RFC-0004 amendment 2026-08-01 | a caller short *now* waits `Throttled`; only a context that can *never* pass is refused | refusal makes clients spin; no donation happens below the bar either way | -| A3 | RFC-0004 amendment 2026-08-01; research/0002 Part 6 | seL4 timeout faults ship in MCS (`seL4_Fault_Timeout`); what is unmerged is RFC-14's threshold | a factual correction | -| A4 | RFC-0003 §14.2 | the big lock is taken on the fast path by design; the question is answered by rejecting its premise | §12; Peters et al. 2015 | +| A2 | RFC-0004 amendment 2026-08-01 | a caller short *now* waits `Throttled`; only a context that can *never* pass is refused | refusal makes clients spin; nothing is donated below the bar either way | +| A3 | the same, and research/0002 Part 6 | seL4's timeout faults ship in MCS (`seL4_Fault_Timeout`); what is unmerged is RFC-14's threshold | a factual correction to "have remained future work" | +| A4 | RFC-0003 §14.2 | the big lock covers the fast path by design; the question is closed by rejecting its premise | §12; Peters et al. 2015 | ## 14. Lineage | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl`; MCP with any-capability authority; sporadic refills; `MIN_BUDGET`; timeout faults; handlers held in the TCB; priority-ordered endpoint queues; no donation to active receivers | the WCET scale factor; ceiling left wholly to convention | +| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl`; MCP with any-capability authority; sporadic refills; `MIN_BUDGET`; timeout faults; handlers held in the TCB; priority-ordered endpoint queues; no donation to an active receiver | the WCET scale factor | | **seL4 RFC-14** (Johnston) | the endpoint threshold, deferral, one-layer return | budget limits, for now | | **Heiser & Elphinstone 2016; Fluke 1999** | the event kernel | per-thread kernel stacks | | **Elphinstone & Heiser 2013** | the running thread unqueued; priority-aware direct switch | lazy scheduling; L4's unconditional switch | @@ -478,108 +422,100 @@ amendment — never a silent edit. | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| | O-7 no unprivileged exhaustion | time only through contexts; `MIN_BUDGET`; eight refills; O(1) expiry return; bounded queue walks; no wait holding a lock; lines masked until `ack` | **discharged for CPU time and scheduler structures, except interrupt-entry time**, billed to the running context and bounded per `ack`; memory is RFC-0005's half | -| O-4 no ambient authority | no thread runs without a context; priority through MCP, born 0; time through `SchedControl`; interrupts through `IrqControl`; only `yield` and `thread_exit` authority-free | **discharged for time** | -| O-5 argument validation | `write_regs` sanitises `SPSR_EL1`/`RFLAGS`, refuses non-canonical `RIP` and bases, cannot touch discovery registers; restarts re-resolve | **discharged for this surface** | -| O-9 no ambient side channel | no EL0 counter, timer or PMU; FP owner cleared on reuse; parked cores. **Residual:** untrapped `wfe` lets any thread observe another's `sev` — one bit of wake-up, no data | **contributes** — fine-grained timing channels stay out of scope | +| O-4 no ambient authority | no thread runs without a context; priority through MCP, born 0; time through `SchedControl`; lines through `IrqControl`; only `yield` and `thread_exit` authority-free | **discharged for time** | +| O-5 argument validation | `write_regs` sanitises `SPSR_EL1`/`RFLAGS`, refuses non-canonical `RIP` and bases, cannot reach the discovery registers; restarts re-resolve | **discharged for this surface** | +| O-9 no ambient side channel | no EL0 counter, timer or PMU; FP owner cleared on reuse; parked cores. **Residual:** untrapped `wfe` lets a thread observe another's `sev` (Arm ARM: an SEV on any PE is a wake-up event) — one bit, no data | **contributes**; fine-grained timing channels stay out of scope | | O-17 contained drivers | lines as notifications, masked until `ack`; `IrqHandler` outlives a crashed driver; fault endpoints the thread cannot close; stall counters | **enables**, weakened where a child holds `WRITE` on its own threads (§4) | | O-18 confined DMA | MSI deferred until interrupt remapping is designed | **named, not discharged** | -| RFC-0004 amendment, 2026-08-01 | (1) `min_budget` at `call`, with deferral (A2); (2) `BudgetExpired` on the reply object, plus timeout faults (§7) | **both discharged as amended by A1–A2**; caller death cannot bypass (1) | +| RFC-0004 amendment, 2026-08-01 | (1) `min_budget` at `call`, deferral per A2; (2) `BudgetExpired` on the reply object, plus timeout faults; caller death cannot bypass (1) | **both discharged as amended by A1–A2** | ## 16. Graves checked (§3) -- **Policy in the kernel.** No admission, balancing or budget choice. The constants the kernel does - choose: 256 levels; eight refills; `MIN_BUDGET`'s factor 2; the XCR0 extent; the thread and deadline - array capacities (RFC-0005); and §18's interim constants until measured. -- **The catch-all right.** Priority authority is a relation (MCP); `SchedControl` has one mutating method, - selling time on one core, and a read of its own state; topology is data and routing belongs to - `IrqControl`; interrupts are per line. The set is closed by listing it (§4). -- **Bolted-on multicore.** Per-core state, IPIs, the lock, its IPI-servicing spin, FP handoff, shootdowns - and core bring-up are stated before any SMP code. -- **Drivers in the kernel; compiled-in unused device paths.** The kernel masks and signals, never services - a device; one controller and one timer per target; unclaimed lines never enabled. +- **Policy in the kernel.** No admission, balancing or budget choice. The constants the kernel does choose: 256 + levels, eight refills, `MIN_BUDGET`'s factor 2, the XCR0 extent, the thread and deadline array capacities + (RFC-0005), and §19's interim constants until measured. +- **The catch-all right.** Priority authority is a relation (MCP); `SchedControl` has one mutating method, selling + time on one core, and a read of its own state; topology is data, routing is `IrqControl`'s; §4's list is closed. +- **Bolted-on multicore.** Per-core state, IPIs, the lock and its IPI-servicing spin, FP handoff, shootdowns and + core start-up are stated before any SMP code. +- **Drivers in the kernel; unused device paths.** The kernel masks and signals, never services a device; one + controller and one timer per target; unclaimed lines never enabled. - **Baroque hierarchies; multi-copy IPC.** Contexts are flat; donation moves a reference to time. ## 17. Costs — what this makes harder -- **Every server needs someone to think about time.** A passive server cannot run until called; an active - one needs a configured context and gets no inheritance (A1). "Spawn a thread and it runs" is gone. -- **The one-hop unwind leaves a stranded server to userspace**, and `min_budget` 0 lets one client strand - it for all. A supervisor is required before any passive server on partial contexts is trusted. -- **Caps, not minima:** a partial context idles its core when spent, even with the core otherwise idle. +- **Every server needs someone to think about time.** A passive server cannot run until called; an active one + needs a configured context and inherits nothing (A1). "Spawn a thread and it runs" is gone. +- **A stranded server is userspace's problem,** and `min_budget` 0 lets one client strand it for all: a + supervisor is required before any passive server on partial contexts is trusted. +- **Caps, not minima:** a spent partial context idles its core. **Priority-ordered queues** cost an O(n) walk + with interrupts masked, *n* up to the thread capacity. - **Interrupt time is billed to whoever runs:** a driver `ack`ing a level line in a loop charges other - processes' budgets, one interrupt per `ack` — seL4's choice; charging an `IrqHandler`'s bound context - becomes possible with passive drivers (Open question 4). -- **Priority-ordered queues** cost an O(n) walk with interrupts masked, *n* up to the thread capacity. -- **No userspace timer yet:** RFC-0004 §8's load-bearing watchdog has no time source until deadline - objects (increment 15) — an unmet RFC-0004 dependency, named. -- **Exclusivity is checkable, not revocable:** `ctl_info` shows a grantor kept no copy, but a grantor who - did can still configure onto the core. + processes, one interrupt per `ack` — seL4's choice; passive drivers (Open question 4) could bill the driver. +- **No userspace timer yet:** RFC-0004 §8's load-bearing watchdog lacks a source until increment 15 — an unmet + RFC-0004 dependency, named. +- **Exclusivity is checkable, not revocable:** `ctl_info` shows whether a grantor kept a copy; if one did, it can + still configure onto the core. - **MCS is seL4's least-verified part**, and sporadic refills are subtle — hence property tests. - **About 1 KiB per TCB on AArch64 and 1.25 KiB on x86_64**, FP space included for threads that never use it. -- **Lazy FP's cure, costed.** Eager restore on a switch into an FP-enabled thread of another process moves - 520 bytes (16 `ldp q` pairs) or an 832-byte `XRSTOR` — 9 or 13 cache lines — and nothing on IPC to - FP-off servers; its cycles are a measurement owed, beside a `call` round trip of roughly 380–630 cycles - (twice research/0001's one-way 188–316). +- **Lazy FP's cure:** eager restore into an FP thread of another process moves 520 bytes or an 832-byte + `XRSTOR` (9 or 13 cache lines), beside a `call` round trip of roughly 380–630 cycles (research/0001). ## 18. Open questions -1. **EL0 counter access.** Proposed: a property of the *context*, set by the `SchedControl` holder in - `sc_configure` and switched with it (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`), default deny — not a thread - flag, which a process could set on itself. Denial costs a kernel entry per timestamp. +1. **EL0 counter access.** Proposed: a property of the *context*, set by the `SchedControl` holder and switched + with it (`CNTKCTL_EL1.EL0VCTEN`; `CR4.TSD`), default deny — not a thread flag a process sets on itself. 2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §17)? -3. **RFC-14 budget limits.** Worth the fastpath cost to stop a passive server overspending? -4. **Passive drivers** need a context bound to a notification (MCS); with them, interrupt entry can be - charged to the driver. -5. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst kernel path; x86_64's clock - when the TSC is not invariant. -6. **Interrupt routing on SMP** (§11): `IrqControl` minting per-(line, core) handlers, or a separate - per-core interrupt-target capability — decided with increment 14. +3. **RFC-14 budget limits.** Worth their fastpath cost, once re-measured, to stop a server overspending? +4. **Passive drivers** need a context bound to a notification (MCS); interrupt entry could then bill the driver. +5. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst path; x86_64's clock without an + invariant TSC. +6. **Interrupt routing on SMP:** per-(line, core) handlers minted by `IrqControl`, or a per-core interrupt-target + capability — decided with increment 14. ## 19. Implementation increments -Pure logic lands in a new host-tested `no_std` workspace crate, `scheduler/`, generic over a thread index -as `capability/` is over `ObjectRef` (the `CLAUDE.md` § Layout edit lands with increment 1); hardware -behaviour is proved by counted QEMU boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The -x86_64 HAL gains stubs that compile, `-> !` ones halting, so its entry still calls straight through into -the kernel proper and the second build compiles the scheduler. **D** marks what the Phase-1 demo needs. +Pure logic lands in a new host-tested `no_std` workspace crate, `scheduler/`, generic over a thread index as +`capability/` is over `ObjectRef` (`CLAUDE.md` § Layout gains it in increment 1); hardware behaviour is proved by +counted boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The x86_64 HAL gains stubs that compile — +`-> !` ones halt — so its entry still calls into the kernel proper. **D** marks what the demo needs. | # | PR | Proves; how tested | Lands after | D | |---|----|--------------------|-------------|---| -| 1 | `scheduler` I: run queues, the ten states, MCP, priority-ordered endpoint queue | exhaustive host transition table against §4's "Left by" column; "a new thread cannot serve as `auth` above 0"; CI builds it for both targets | — | D | -| 2 | `scheduler` II: full contexts, nanosecond conversion and rounding, billing, stall counters, the interim constants | host tests against a shadow model | 1 | D | -| 3 | GICv3 and the virtual timer, kernel-only; `CNTKCTL_EL1`, `PMUSERENR_EL0` zeroed; idle and its abandon-idle IRQ stub | xtask gains `gic-version=3`; boot-test `timer: tick 3` | 0005#9 (GIC in the MMIO window) | D | -| 4 | Event-kernel dispatch at EL0, MMU off: TCB frames, dispatch and RFC-0006's frame layout under RFC-0007#3's trap stub; `SCTLR_EL1` bits of §8 set | two EL0 self-test threads alternate under the timer: `sched: A B A B`; an EL0 `wfi` completes: `sched: wfi yields` | 3, 0007#3 | D | +| 1 | `scheduler` I: run queues, the ten states, MCP, the priority-ordered endpoint queue | exhaustive host transition table against §4's "Left by" column; "a new thread cannot serve as `auth` above 0"; built for both targets | — | D | +| 2 | `scheduler` II: full contexts, nanosecond conversion and rounding, billing, stall counters, interim constants | host tests against a shadow model | 1 | D | +| 3 | GICv3 and the virtual timer, kernel-only; `CNTKCTL_EL1`, `PMUSERENR_EL0` zeroed; idle and its abandon-idle stub | xtask gains `gic-version=3`; `timer: tick 3` | 0005#9 (GIC in the MMIO window) | D | +| 4 | Event-kernel dispatch at EL0 with the MMU off: TCB frames and dispatch under RFC-0007#3's trap stub; §8's `SCTLR_EL1` bits | two EL0 self-test threads alternate under the timer: `sched: A B A B`; an EL0 `wfi` returns: `sched: wfi yields` | 3, 0007#3 | D | | 5 | Threads in address spaces; the boot stack moved into the guarded window, `aarch64.ld`'s comment with it | a spinning EL0 thread is preempted and resumes intact: `user: still here` | 4, 0005#10 | D | -| 6a | Thread, SchedContext and `SchedControl` as slots in RFC-0005's object store, state machine wired, links `(index, generation)` | host tests in `scheduler/`: destroy a lent context, reuse its slot, `reply` — nothing moves | 2, 0005#7 | D | -| 6b | The methods behind RFC-0007's dispatcher: `sc_configure`, `bind_sc`, `resume`, `set_priority`, `write_regs`, `thread_exit` | the root configures two EL0 threads at equal priority: `[a] 1 [b] 1 [a] 2` | 5, 6a, 0007#6 | D | -| 7a | Direct-switch decision as a pure function | host tests over every §6 case | 1 | D | -| 7b | Endpoint queues and IPC blocking states, wired | host tests: queue order, `suspend` from each blocked state | 6b, 7a | D | -| 7c | Integration with RFC-0007#10 | RFC-0007#10's three `Kaya!` lines, then `[client] preempted` from a spin after the reply | 7b, 0007#10 | D | -| 8 | Passive servers, donation, inheritance, the caller-death rule | host test: a client exits mid-request, the server finishes and serves the next; boot-test `passive: served` | 7c | stretch | +| 6a | Thread, SchedContext, `SchedControl` as RFC-0005 object-store slots; state machine wired; `(index, generation)` links | host: destroy a lent context, reuse the slot, `reply` — nothing moves | 2, 0005#7 | D | +| 6b | Methods behind RFC-0007's dispatcher: `sc_configure`, `bind_sc`, `resume`, `set_priority`, `write_regs`, `thread_exit` | the root runs two EL0 threads at one level: `[a] 1 [b] 1 [a] 2` | 5, 6a, 0007#6 | D | +| 7a | The direct-switch decision as a pure function | host tests over every §6 case | 1 | D | +| 7b | Endpoint queues and the IPC blocking states, wired | host: queue order; `suspend` from each blocked state | 6b, 7a | D | +| 7c | Integration with RFC-0007#10 | its three `Kaya!` lines, then `[client] preempted` from a spin after the reply | 7b, 0007#10 | D | +| 8 | Passive servers, donation, inheritance, the caller-death rule | host: a client exits mid-request, the server finishes and serves the next; `passive: served` | 7c | stretch | | 9 | `scheduler` III: partial contexts, refills, merge, release queue | property test: no window of length `T` sees more than `B` consumed | 2 | | -| 10 | `min_budget`, deferral, expiry return, fault and timeout endpoints, `Unfunded` | boot-tests `BudgetRefused`, then `BudgetExpired` and `timeout fault` for an overrunning server | 8, 9, 0007#11 | | -| 11 | Lazy FP/SIMD | two FP threads load markers into `v0`, each finds its own: `fp: isolated`; host test: destroy the owner, reuse its slot, first use loads zeros | 6b | | -| 12 | `IrqControl` / `IrqHandler`; xtask gains serial input injection | PL011 receive (SPI 1, INTID 33) reaches an EL0 driver: `irq: notified` | 7c | | +| 10 | `min_budget`, deferral, expiry return, fault and timeout endpoints, `Unfunded` | `BudgetRefused`, then `BudgetExpired` and `timeout fault` for an overrunning server | 8, 9, 0007#11 | | +| 11 | Lazy FP/SIMD | two FP threads load markers into `v0`, each finds its own: `fp: isolated`; host: owner destroyed, slot reused, zeros loaded | 6b | | +| 12 | `IrqControl` and `IrqHandler`; xtask gains serial input injection | PL011 receive (SPI 1, INTID 33) reaches an EL0 driver: `irq: notified` | 7c | | | 13 | x86_64 scheduling HAL (IDT, TSS, IST, LAPIC, `#NM`, `XSAVE`) | 3–5 and 11 again | its boot RFC, 0007#12 | | -| 14 | SMP: PSCI `CPU_ON` and INIT-SIPI, the big lock with IPI-servicing spin, FP handoff, shootdowns, placement classes, routing | `-smp 2` boot-tests | 13 | | -| 15 | Deadline objects (userspace timers, RFC-0004 §9.3) | host tests on the shared release queue; boot-test `deadline: fired` | 9 | | +| 14 | SMP: PSCI `CPU_ON`, INIT-SIPI, the lock and its IPI-servicing spin, FP handoff, shootdowns, classes, routing | `-smp 2` boot-tests | 13 | | +| 15 | Deadline objects (RFC-0004 §9.3) | host tests on the shared release queue; `deadline: fired` | 9 | | **What the demo takes as interim, honestly.** It needs 1–7c and runs full contexts only. -- **Its server is active**, the root doubling as server (RFC-0007 §18), so the demo exercises **A1's - amended semantics, not RFC-0004 §4's**: no donation happens. Increment 8 is one more PR, safe before 9 - and 10 because a lent full context refills at once; expiry inside a server needs a partial context. -- **Provisional constants**, in `scheduler/src/config.rs` until measured: `MIN_BUDGET_INTERIM` = 100 µs; - `ROOT_TIMESLICE` = 5 ms (a full context); root priority and MCP 255, the root lowering itself and the - client to 100 to share a level; `KERNEL_STACK_SIZE` = 16 KiB. Cost: the budget floor is a guess. +- **Its server is active** — the root doubles as server (RFC-0007 §18) — so the demo exercises **A1's semantics, + not RFC-0004 §4's**: nothing is donated. Increment 8 is one more PR, safe before 9 and 10 because a lent full + context refills at once; expiry inside a server needs a partial context. +- **Provisional constants** in `scheduler/src/config.rs` until measured: `MIN_BUDGET_INTERIM` = 100 µs; + `ROOT_TIMESLICE` = 5 ms, a full context; root priority and MCP 255, the root putting itself and the client at + 100 to share a level; `KERNEL_STACK_SIZE` = 16 KiB. Cost: the budget floor is a guess. - **No fault handler:** a faulting thread goes `Inactive` and the reporter prints it (RFC-0007 §7). -- **The console write is non-preemptible** until RFC-0007 adds its preemption points — free on QEMU, - about 44 ms on hardware. **No userspace timer**, so RFC-0004's watchdog cannot yet be built. -- **The x86_64 stubs** compile and halt, removed by 13. Userspace is soft-float, so 11 is not needed. +- **A non-preemptible console write** until RFC-0007 adds its preemption points — free on QEMU, 44 ms on + hardware; **no userspace timer**, so RFC-0004's watchdog cannot be built; **x86_64 stubs** that halt, until 13. ## 20. What this unblocks RFC-0004's implementation gets threads to rendezvous between. The driver framework receives interrupts as -notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `SchedControl`, -MCP, `min_budget` and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment -(14), and the dated entries of §13. +notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `SchedControl`, MCP, +`min_budget` and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment (14), and +§13's dated entries. From 7ad90e8cb9d6eb0f6aab962ac93b640aeea7d1ab Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:42:17 +0000 Subject: [PATCH 14/20] wip(docs): snapshot RFC revisions in progress Backup only; RFC-0005's workflow has finished, RFC-0006 and RFC-0007 are still being revised. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 252 +++++++++++------------ 1 file changed, 115 insertions(+), 137 deletions(-) diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 7b539c4..00bcf05 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -48,29 +48,27 @@ kernel masks and signals but never services a device. ## 3. The execution model — one kernel stack per core (verdict 1) - **Option A — process kernel** (a kernel stack per thread: Linux, Zircon, original L4). **Rejected:** a guarded - stack per thread in a kernel with no heap, and blocking in the kernel is how "hold a lock across a wait a - process controls" (O-7) becomes possible. + stack per thread with no heap, and blocking in the kernel is how "hold a lock across a wait a process + controls" (O-7) becomes possible. - **Option B — event kernel** (one stack per core: seL4, OKL4, NOVA; Fluke's "interrupt model"). **Verdict - sought.** The kernel is entered by exception, interrupt or syscall, runs to completion with interrupts masked, - and leaves by restoring *some* thread's user frame. A syscall that must wait records a state and the object - waited on; on wake the result is written into the waiter's frame, or the call restarts by rewinding the saved - PC one instruction (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & Elphinstone, TOCS 2016; Ford et al., - "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* - -**The cost:** every kernel path must be bounded. Long ones — destruction, RFC-0005 (proposed)'s `reissue` and -teardown, RFC-0007 (proposed)'s console write — take **preemption points**: on a pending interrupt -(`ISR_EL1.I`; the LAPIC IRR) they leave the operation restartable and exit. **A restarted operation -re-resolves every handle and re-checks every right**, and an object mid-teardown carries a *dying* mark so no -entry invokes it between the exit and the restart. The worst path between points is measured and published: it -is the interrupt latency. The boot stack becomes core 0's kernel stack, moved into RFC-0005 (proposed)'s -guarded window — **reversing `aarch64.ld`'s plan** to replace it "by per-thread stacks". - -**Idle is a kernel loop, not a thread,** billing nobody. It runs `wfi` with `PSTATE.I` set — a pending interrupt -still wakes it — then unmasks; Linux warns that masking at `ICC_PMR_EL1` would *not* wake the core -(`arch/arm64/kernel/idle.c`), so this kernel masks with DAIF only. x86_64 runs `sti; hlt`, whose STI shadow -issues the `hlt` before delivery. **An interrupt taken in idle abandons it:** the current-EL IRQ stub — this -RFC's; RFC-0007 owns the lower-EL and `syscall` entries — saves nothing, resets `SP` to the per-core stack top -and enters the dispatcher. Idle is the only place the kernel unmasks, and it drops the big lock (§12) first. + sought.** Entered by exception, interrupt or syscall, the kernel runs to completion with interrupts masked and + leaves by restoring *some* thread's frame. A syscall that must wait records a state; on wake its result is + written into the frame, or it restarts by rewinding the PC (4 bytes for `svc #0`, 2 for `syscall`). *(Heiser & + Elphinstone, TOCS 2016; Ford et al., "Interface and Execution Models in the Fluke Kernel", OSDI 1999.)* + +**The cost:** every path must be bounded. Long ones — destruction, RFC-0005's `reissue` and teardown, RFC-0007's +console write (both proposed) — take **preemption points**: on a pending interrupt (`ISR_EL1.I`; the LAPIC IRR) +they leave the operation restartable and exit. **A restart re-resolves every handle and re-checks every right**; +an object mid-teardown is marked *dying* so no entry invokes it meanwhile. The worst path between points is +measured and published as the interrupt latency. The boot stack becomes core 0's kernel stack in RFC-0005's +guarded window, **reversing `aarch64.ld`'s plan** to replace it "by per-thread stacks". + +**Idle is a kernel loop, not a thread,** billing nobody: `wfi` with `PSTATE.I` set (a pending interrupt still +wakes it), then unmask — DAIF only, since Linux warns that masking at `ICC_PMR_EL1` would *not* wake the core +(`arch/arm64/kernel/idle.c`); x86_64 runs `sti; hlt`, whose STI shadow issues the `hlt` first. **An interrupt in +idle abandons it:** the current-EL IRQ stub (this RFC's; RFC-0007 owns the lower-EL and `syscall` entries) saves +nothing, resets `SP` to the per-core stack top and enters the dispatcher. Idle alone unmasks, and it drops the +big lock (§12) first. ## 4. The Thread object (verdict 2) @@ -87,10 +85,6 @@ slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool prese | `fault_ep`, `timeout_ep` | **two kernel-held capability slots** *(seL4 MCS)*: moved in by the binder, generation-checked when a fault is raised, failing closed if the endpoint died; the thread cannot close them | | `fp_enabled`, `fp_state` | the FP/SIMD flag and save area (§10) | -**Every internal link survives slot reuse.** A queue link, `sc`, `home_sc`, a reply object's record of a lent -context, an `IrqHandler`'s notification and a core's FP owner are each a counted reference holding the slot -(RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare index. - | State | Meaning | Left by | |-------|---------|---------| | `Inactive` | created, suspended, or faulted with no handler; in no queue | `resume` | @@ -104,14 +98,16 @@ context, an `IrqHandler`'s notification and a core's FP owner are each a counted | `BlockedOnNotification` | waiting on a notification alone | signal, `suspend` | | `Exited` | finished; in no queue; the object lingers until destroyed | never | -Faults reuse IPC: the kernel `call`s `fault_ep` for the thread, which waits `BlockedOnReply`; RFC-0007 -(proposed) defines the message. Fault and timeout messages bypass `min_budget`. Destroying a thread bumps its -generation (RFC-0003 §7), unlinks it in O(1) from any doubly linked queue and destroys any reply object it waits -on (RFC-0004 §8); §7 governs a context it had lent. +Faults reuse IPC: the kernel `call`s `fault_ep` for the thread, which waits `BlockedOnReply` (RFC-0007 defines +the message); fault and timeout messages bypass `min_budget`. Destroying a thread bumps its generation (RFC-0003 +§7), unlinks it in O(1) and destroys any reply object it waits on (RFC-0004 §8); §7 governs a context it lent. +**Every internal link survives slot reuse:** queue links, `sc`, `home_sc`, a reply object's record of a lent +context, an `IrqHandler`'s notification and a core's FP owner are each a counted reference holding the slot +(RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare index. -Every operation is a method `call` on the **invoked** capability (RFC-0007 (proposed) verdict 1). Some also -name a **presented** capability: resolved in the caller's table and checked, never moved, needing no -`TRANSFER` — an encoding RFC-0007 (proposed) provides *(seL4's `extraCaps`)*. The list is closed: +Every operation is a method `call` on the **invoked** capability (RFC-0007 verdict 1). Some also name a +**presented** one — resolved and checked in the caller's table, never moved, needing no `TRANSFER` — an encoding +RFC-0007 (proposed) provides *(seL4's `extraCaps`)*. The list is closed: | Operation | Invoked: right | Presented: right | Effect | |-----------|----------------|------------------|--------| @@ -126,10 +122,10 @@ name a **presented** capability: resolved in the caller's table and checked, nev | `sc_stats(sc)` / `ctl_info(ctl)` | `sc` / `ctl`: `READ` | — | §5's counters; the exclusivity check (§12) | | `yield()` / `thread_exit()` | none: two of RFC-0003 §9's three authority-free exceptions | — | tail of own level / `Exited` | -No right joins RFC-0003's set. **The supervision cost:** a process holding `WRITE` on its own thread can rebind -that thread's `fault_ep`. A supervisor keeps the binding by granting no `WRITE` on threads it creates for a -child; threads the child creates are supervised through the Process (RFC-0007 open question 1). A TLS runtime -sets `FS_BASE` through `write_regs`, so it needs `WRITE` on its thread — or its spawner sets it before `resume`. +No right joins RFC-0003's set. **The supervision cost:** a process with `WRITE` on its own thread can rebind +its `fault_ep`, so a supervisor grants no `WRITE` on threads it creates for a child; threads the child creates +are supervised through the Process (RFC-0007 open question 1). A TLS runtime sets `FS_BASE` by `write_regs`, +needing `WRITE` on its thread, or its spawner sets it before `resume`. ## 5. Scheduling contexts and `SchedControl` — time as a capability (verdicts 3, 4) @@ -164,117 +160,104 @@ with released budget*: a spent partial context leaves its core idle by design. N **256 levels**, 255 most urgent, meaningful within a core. `set_priority(t, auth, p)` needs `WRITE` on `t`, any capability to the Thread `auth`, and `p ≤ mcp(auth)`; MCP is set alike, so authority over urgency only narrows -as it is delegated — O-2's shape *(seL4 `TCB_SetPriority`, whose authority is any TCB capability)*. **Every -thread is born at priority 0 and MCP 0**, so a fresh thread is no authority; the root's thread starts at 255 and -255 with a full context on core 0, the one bootstrap act (RFC-0007 (proposed) §9). To delegate priority -authority alone, derive a **rightless** capability to a thread holding the MCP wanted. - -**Rejected: `READ` or `WRITE` on `auth`** — either makes priority authority and register access the same token. -**Rejected: a `SCHEDULE` or `PRIORITY` right** — the catch-all right in embryo, one bit whose meaning grows with -every scheduling operation (the CAP_SYS_ADMIN grave). **Rejected: dynamic priorities** (CFS, BSD decay): -kernel-resident policies churned while accounting and enforcement endured (research/0002 Part 1). **The -cost:** any capability to a high-MCP thread is priority authority, so such capabilities are handed out sparingly. +as it is delegated — O-2's shape *(seL4 `TCB_SetPriority`, whose authority is any TCB capability)*. **Threads are +born at priority 0 and MCP 0**, so a fresh thread is no authority; the root's starts at 255 and 255 with a full +context on core 0, the one bootstrap act (RFC-0007 §9). A **rightless** capability to a thread holding the MCP +wanted delegates priority authority alone. **Rejected: `READ` or `WRITE` on `auth`**, which would make priority +authority and register access one token; **a `SCHEDULE` right**, the catch-all right in embryo (the +CAP_SYS_ADMIN grave); **dynamic priorities** (CFS, BSD decay), kernel-resident policies that churned while +accounting and enforcement endured (research/0002 Part 1). **The cost:** any capability to a high-MCP thread is +priority authority, to be handed out sparingly. - **Run queues.** Per core, 256 intrusive FIFO lists and a 256-bit bitmap: the next thread is a count-leading-zeros over four `u64` words — O(1). A preempted thread re-enters at the head of its level; one - whose timeslice ended, at the tail. The running thread is never queued, nor is a thread switched to - directly. *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* **Release queue:** per core, sorted by next + whose timeslice ended, at the tail; the running thread and one switched to directly are never queued. + *(Elphinstone & Heiser, "From L3 to seL4", SOSP 2013.)* The per-core **release queue** is sorted by next refill, O(n) in contexts only the `SchedControl` holder adds. - **Priority-aware direct switch** (RFC-0004 §4). When *C* wakes *W* by IPC on the same core and *W* has released budget: if *C* blocks (`call`, `recv`, `reply_recv`), switch to *W* when its effective priority is at least the - highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also strictly more urgent than - *C*. Otherwise *W* is queued; without released budget, it goes to the release queue; on another core, it is - enqueued there and sent an IPI (§12). + highest ready; if *C* stays runnable (`send`, `reply`, `notify`), only when *W* is also strictly more urgent. + Otherwise *W* is queued — on the release queue without budget, on its own core with an IPI (§12). - **Endpoint queues are priority-ordered, first-come within a level** — seL4 MCS's `tcbAppend` - (`include/object/tcb.h`, "priority ordered endpoint or notification queue"), which walks back from the tail; - the FIFO `tcbEPAppend` is compiled only without MCS. The walk is O(n) with interrupts masked, *n* bounded by - the thread array's fixed capacity (RFC-0005 §5) and counted in the worst path; `set_priority` repositions a - queued thread. **Rejected: FIFO** *(classic L4, non-MCS seL4)*: O(1), but a process could create threads from - its own Pool and queue *N* calls ahead of a more urgent client of a shared server, which cannot reorder them. + (`include/object/tcb.h`, "priority ordered endpoint or notification queue"), walking back from the tail; the + FIFO `tcbEPAppend` is compiled only without MCS. The walk is O(n) with interrupts masked, *n* bounded by the + thread array's fixed capacity (RFC-0005 §5) and counted in the worst path; `set_priority` repositions a queued + thread. **Rejected: FIFO** *(classic L4, non-MCS seL4)*: O(1), but a process could create threads from its own + Pool and queue *N* calls ahead of a more urgent client of a shared server, which cannot reorder them. ## 7. Donation, inheritance and budget expiry (verdicts 7, 8) -**Donation.** When *C* `call`s and the receiver *S* is **passive**, the kernel moves *C*'s current context to -*S*, recorded in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves it back. *S* runs -at `max(prio(S), eff(C))`, **fixed when the donation is made**, not propagated if *C*'s priority later changes — -O(1), one hop at a time. Lending priority is safe because time is lent with it. *(Ford & Lepreau, migrating -threads, 1994; QNX priority inheritance; seL4 MCS passive servers.)* - -**An active receiver gets no donation (amendment A1).** It runs on its own context at its own priority, as in -seL4 MCS (`reply_push`, `src/object/reply.c`, donates only to a receiver with none): a thread holds one context, -and an active server has chosen to pay for itself. RFC-0004 §4 and §8 say `call` lends unconditionally, so this -amends them. **The cost:** an active server inherits nothing, so a low-priority one blocks urgent clients. -Constitution §3's inheritance holds for passive servers; **a server shared across trust domains runs passive, or -active at no lower priority than its most urgent client.** - -**Inversion, bounded.** Priority-ordered queues serve the most urgent waiter next, but it still waits for the -request in service, which runs on its client's context and can be preempted by a middle-priority thread. -`prio(S)` is therefore a **ceiling floor**; set to the highest client priority it bounds inversion to one -request — required, not optional, for shared servers *(the priority-ceiling bound seL4 MCS relies on)*. -**Rejected:** boosting a busy server to a waiting sender's priority (QNX), where the kernel would guess which -thread serves an endpoint — policy — and walk unbounded chains with interrupts masked. - -**Caller death never unfunds a server.** If *C* exits or is killed mid-request, *R* is destroyed but the -context stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails `PeerGone`); then it -detaches, and only its capability holder may rebind or destroy it. A client's `thread_exit` never pulls time -from a server *(seL4 `reply_remove_tcb` likewise leaves the context with the server)*. +**Donation.** When *C* `call`s a **passive** receiver *S*, the kernel moves *C*'s current context to *S*, recorded +in the single-use reply object *R* (RFC-0004 §8); `reply` or `reply_recv` moves it back. *S* runs at +`max(prio(S), eff(C))`, **fixed at donation**, not propagated if *C*'s priority later changes — O(1), one hop at +a time. Lending priority is safe because time is lent with it. *(Ford & Lepreau, migrating threads, 1994; QNX +priority inheritance; seL4 MCS passive servers.)* + +**An active receiver gets no donation (amendment A1).** It runs on its own context and priority, as in seL4 MCS +(`reply_push`, `src/object/reply.c`, donates only to a receiver with none): a thread holds one context, and an +active server chose to pay for itself. RFC-0004 §4 and §8 lend unconditionally, so this amends them. **The +cost:** an active server inherits nothing, and a low-priority one blocks urgent clients; Constitution §3's +inheritance holds for passive servers only. **Inversion, bounded.** Priority-ordered queues serve the most +urgent waiter next, but the request in service runs on its client's context and a middle-priority thread can +preempt it. `prio(S)` is therefore a **ceiling floor**, bounding inversion to one request when set to the +highest client priority *(the priority-ceiling bound seL4 MCS relies on)* — **required, not optional, for a +server shared across trust domains**, passive or active. **Rejected:** boosting a busy server to a waiting +sender's priority (QNX): the kernel would guess which thread serves an endpoint and walk unbounded chains. + +**Caller death never unfunds a server.** If *C* exits or is killed mid-request, *R* is destroyed but the context +stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails `PeerGone`); it then detaches, +and only its capability holder may rebind or destroy it. A client's `thread_exit` never pulls time from a server +*(seL4 `reply_remove_tcb` likewise leaves the context with the server)*. **The 2026-08-01 amendment, discharged as amended,** after seL4 RFC-14 (Mitchell Johnston, "Budget limit thresholds on endpoints for SC Donation", seL4/rfcs pull request 24, open): -- **Prevention — `min_budget`,** fixed at endpoint creation (0 for none; RFC-0007 (proposed) encodes it). A - `call` needs released budget of at least `min_budget + MIN_BUDGET`, the margin paying for the `call` and - `reply` themselves — **this RFC's rule**; RFC-14's exact comparison is to be quoted before acceptance. A - context whose `B` can never meet it fails `BudgetRefused`; a non-donating `send` to such an endpoint is - invalid. With `min_budget` at the server's WCET, expiry inside it is a true error. -- **A caller short *now* waits (amendment A2).** The amendment says donation below the threshold is "refused up - front". Proposed: a partial context short now has its refills merged and deferred, waits `Throttled`, and the - call restarts when the head refill suffices — free in an event kernel, O(refills) ≤ 8, the deferral the - RFC-14 discussion describes. A **full** context short of its slice ends its timeslice and the call restarts - on the next. Nothing is donated below the bar either way; refusal would make clients spin. -- **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* runs on - it, the kernel returns it to *S*'s immediate caller *C*, whose wait on *R* completes with `BudgetExpired` - (seen when the context next refills); *R* is consumed, so a late `reply` fails closed. *S* is left - `Unfunded`, registers intact; the expiry counter rises. If `timeout_ep` is live, the kernel sends a **timeout - fault** carrying consumed time and donating nothing (seL4 MCS sends timeout faults without donation); the - handler, active on its own time, abandons the request (`write_regs`) or lends *S* a context to finish. It - carries **no identity** — badges are reserved zero until RFC-0003a — so a handler binds one endpoint per server. -- **Why one layer.** In A → B → C, expiry in C returns time to B, which gets `BudgetExpired` and can reply to A - with an error; unwinding to A would walk the reply stack — O(n) with interrupts masked. -- **`min_budget` 0 prevents nothing:** one client can strand a shared server for all its clients until a - handler funds it (O-26, A5). Zero suits a server with one client, or one under a supervisor. +- **Prevention — `min_budget`,** fixed at endpoint creation (0 for none; RFC-0007 encodes it). A `call` needs + released budget of at least `min_budget + MIN_BUDGET`, the margin paying for the `call` and `reply` — **this + RFC's rule**; RFC-14's exact comparison is to be quoted before acceptance. A context whose `B` can never meet + it fails `BudgetRefused`; a non-donating `send` to such an endpoint is invalid. +- **A caller short *now* waits (amendment A2),** where the amendment says "refused up front": a partial context + has its refills merged and deferred, waits `Throttled`, and the call restarts when the head refill suffices — + free in an event kernel, O(refills) ≤ 8, the deferral the RFC-14 discussion describes; a **full** context ends + its timeslice and retries on the next. Nothing is donated below the bar either way; refusal makes clients spin. +- **Recovery — one layer down.** If a donated context is exhausted with no refill due while passive *S* runs, the + kernel returns it to *S*'s immediate caller, whose wait on *R* completes with `BudgetExpired`; *R* is consumed, + so a late `reply` fails closed. *S* is left `Unfunded`, registers intact. If `timeout_ep` is live, a **timeout + fault** carries consumed time and donates nothing (seL4 MCS likewise); the handler, on its own time, abandons + the request (`write_regs`) or lends *S* a context. It carries **no identity** — badges are reserved zero until + RFC-0003a — so a handler binds one endpoint per server. Unwinding further (A → B → C) would walk the reply + stack, O(n) with interrupts masked; B gets `BudgetExpired` and can fail A itself. +- **`min_budget` 0 prevents nothing:** one client can strand a shared server for all until a handler funds it + (O-26, A5). Zero suits a server with one client, or one under a supervisor. **Rejected: the unwind without timeout faults** — a supervisor woken by a message beats one polling counters. -**Deferred: RFC-14's budget limits** (capping what a server spends of a donated context): the RFC-14 -discussion reports 22% extra fastpath cost; the per-operation figures are to be re-checked (Open question 3). +**Deferred: RFC-14's budget limits** (capping a server's spend of a donated context): the RFC-14 discussion +reports 22% extra fastpath cost; the per-operation figures are to be re-checked (Open question 3). ## 8. The timer and preemption (verdict 9) One HAL surface: `now() -> Ticks`, `frequency() -> Hz`, `set_deadline(Ticks)`, `cancel()`, and a `timer_fired()` -up-call. The kernel is **tickless**: each exit programs one per-core deadline — the earliest of a partial -context's exhaustion, the timeslice end if an equal-priority thread is ready, and the next refill due. A lone -full context arms nothing; a lone partial one still arms its exhaustion. - -- **AArch64: the EL1 virtual timer** — `CNTV_CVAL_EL0`, `CNTV_CTL_EL0` (`ENABLE`, `IMASK`, `ISTATUS`), - `CNTVCT_EL0`, and `CNTFRQ_EL0` always read, never assumed; PPI INTID 27, the Arm BSA assignment `virt` uses - (QEMU `include/hw/arm/bsa.h`). Preferred to the physical timer, whose EL1 access an EL2 beneath can trap - (`CNTHCTL_EL2`); any EL2-to-EL1 descent grants EL1 timer access and zeroes `CNTVOFF_EL2`. *(Arm ARM DDI - 0487, "The Generic Timer".)* +up-call. **Tickless:** each exit programs one per-core deadline — the earliest of a partial context's +exhaustion, the timeslice end if an equal-priority thread is ready, and the next refill. A lone full context +arms nothing; a lone partial one still arms its exhaustion. + +- **AArch64: the EL1 virtual timer** — `CNTV_CVAL_EL0`, `CNTV_CTL_EL0`, `CNTVCT_EL0`, and `CNTFRQ_EL0` always + read; PPI INTID 27, the Arm BSA assignment `virt` uses (QEMU `include/hw/arm/bsa.h`). Preferred to the + physical timer, whose EL1 access an EL2 beneath can trap (`CNTHCTL_EL2`); any EL2-to-EL1 descent grants EL1 + timer access and zeroes `CNTVOFF_EL2`. *(Arm ARM DDI 0487, "The Generic Timer".)* - **x86_64: the local APIC timer**, x2APIC where `CPUID.01H:ECX[21]` reports it, enabled by - `IA32_APIC_BASE.EXTD` (bit 10). TSC-deadline mode (LVT mode `10b`, bits 18:17; `IA32_TSC_DEADLINE`, MSR - `0x6E0`) only when `CPUID.01H:ECX[24]` **and** invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; otherwise - one-shot through the initial-count register, calibrated at boot, a past deadline clamped to a count of 1 - since 0 stops the timer. QEMU's TCG lacks TSC-deadline (`target/i386/cpu.c`), so one-shot is what the - emulator runs. The console prints the mode. *(Intel SDM Vol. 3A.)* -- **Userspace timers (RFC-0004 §9.3), named now, built later.** RFC-0004 §8 calls the watchdog load-bearing, and - the timer is kernel-reserved, so the kernel provides the source: a **deadline object**, bound to a - notification, armed with an absolute time in nanoseconds, kept in the per-core release queue beside refills, - charged to a Pool, signalled on expiry *(Zircon's timer object)*; its sorted insert is bounded by the deadline - array's capacity. **Rejected:** minting a second hardware timer — AArch64 has the EL1 physical timer (PPI 30) - per core, but x86_64 has no second per-core timer short of the HPET, so the source would differ per target. - -**Kernel-reserved; this RFC owns these bits.** RFC-0005 (proposed) writes `SCTLR_EL1` whole and must carry -them; its §13 step 5 says `nTWE = 0` today and needs correcting. + `IA32_APIC_BASE.EXTD` (bit 10). TSC-deadline mode (LVT mode `10b`; MSR `0x6E0`) only when `CPUID.01H:ECX[24]` + **and** invariant TSC (`CPUID.80000007H:EDX[8]`) are reported; else one-shot, calibrated at boot, a past + deadline clamped to an initial count of 1, since 0 stops the timer. QEMU's TCG lacks TSC-deadline + (`target/i386/cpu.c`), so one-shot is what it runs. *(Intel SDM Vol. 3A.)* `CR4.TSD` set; `hlt` faults at CPL 3. +- **Userspace timers (RFC-0004 §9.3), named now, built later.** The watchdog is load-bearing (RFC-0004 §8) and + the timer kernel-reserved, so the kernel provides the source: a **deadline object**, bound to a notification, + armed with an absolute time in nanoseconds, kept in the release queue beside refills, charged to a Pool, its + insert bounded by the deadline array's capacity *(Zircon's timer object)*. **Rejected:** minting a second + hardware timer — AArch64 has the EL1 physical timer (PPI 30), x86_64 nothing per core short of the HPET. + +**Kernel-reserved bits, owned here.** RFC-0005 (proposed) writes `SCTLR_EL1` whole and must carry them; its §13 +step 5 says `nTWE = 0` today and needs correcting. | Register | Value | Why | |----------|-------|-----| @@ -282,7 +265,6 @@ them; its §13 step 5 says `nTWE = 0` today and needs correcting. | `SCTLR_EL1.nTWE` (bit 18) | 1 | EL0 `wfe` does not trap, so spin-lock backoff costs no kernel entry; the `sev` residual is in §15's O-9 row | | `SCTLR_EL1.UMA` (bit 9), `SA` (bit 3) | 0, 1 | no EL0 DAIF access; `SP` alignment checked at EL1 (§9) | | `CNTKCTL_EL1`, `PMUSERENR_EL0` | 0 | no EL0 counter, timer or PMU; written at boot because their reset values are not architecturally fixed | -| x86_64 `CR4.TSD`; `hlt` | set; faults at CPL 3 | the same denials on the second target | ## 9. The context switch — what is saved, where @@ -406,16 +388,12 @@ Each, if accepted, is appended to its target as a dated amendment and logged in | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl`; MCP with any-capability authority; sporadic refills; `MIN_BUDGET`; timeout faults; handlers held in the TCB; priority-ordered endpoint queues; no donation to an active receiver | the WCET scale factor | +| **seL4 MCS** (Lyons et al. 2018) | contexts as objects; passive servers; `SchedControl`; any-capability MCP; sporadic refills; `MIN_BUDGET`; timeout faults and TCB-held handlers; priority-ordered endpoint queues; no donation to active receivers | the WCET scale factor | | **seL4 RFC-14** (Johnston) | the endpoint threshold, deferral, one-layer return | budget limits, for now | -| **Heiser & Elphinstone 2016; Fluke 1999** | the event kernel | per-thread kernel stacks | -| **Elphinstone & Heiser 2013** | the running thread unqueued; priority-aware direct switch | lazy scheduling; L4's unconditional switch | -| **QNX Neutrino** | priority inheritance across the message; partitions built on budgets | inheritance to busy servers | -| **Ford & Lepreau 1994** | the migrating-thread picture of `call` | Mach | -| **Sprunt, Sha & Lehoczky 1989** | the sporadic server | an unbounded replenishment list | -| **Zircon** | the timer object, for deadline objects | per-thread timeslices | -| **Jailhouse** | refusing to share: parked cores, exclusive assignment | static-only partitioning | -| **Peters et al. 2015; Linux** | one big lock; the idle wake-up rule; PSI-style stall counters | fine-grained locking before measurement; dynamic priorities | +| **Heiser & Elphinstone 2016; Fluke 1999; Elphinstone & Heiser 2013** | the event kernel; the running thread unqueued; priority-aware direct switch | per-thread kernel stacks; lazy scheduling; L4's unconditional switch | +| **QNX Neutrino; Ford & Lepreau 1994** | inheritance across the message; partitions on budgets; the migrating-thread `call` | inheritance to busy servers; Mach | +| **Sprunt, Sha & Lehoczky 1989; Zircon** | the sporadic server; the timer object | an unbounded replenishment list; per-thread timeslices | +| **Jailhouse; Peters et al. 2015; Linux** | parked cores, exclusive assignment; one big lock; the idle rule; PSI-style counters | static-only partitioning; fine-grained locking before measurement; dynamic priorities | ## 15. Obligations From eedc4d65e0f546f61730f07f279bde019335b448 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:44:52 +0000 Subject: [PATCH 15/20] wip(docs): snapshot RFC revisions in progress Backup only; the final revision agents for RFC-0006 and RFC-0007 are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 256 +++---- ...0007-system-calls-and-process-bootstrap.md | 694 +++++++++++------- 2 files changed, 540 insertions(+), 410 deletions(-) diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 00bcf05..95458f5 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -13,17 +13,17 @@ > **Proposed verdicts.** > -> 1. **Event kernel:** one kernel stack per core; long paths take preemption points; every wait is a thread state (§3). +> 1. **Event kernel:** one stack per core; long paths take preemption points; every wait is a state (§3). > 2. **Thread:** ten states, one of them `Unfunded`; born at priority 0; fault handlers held in the thread (§4). -> 3. **Time is a capability:** a `SchedContext`, set only through its core's one-method `SchedControl` (§5). -> 4. **Budgets are caps, not minima:** a guaranteed minimum is admission policy — a departure from research/0002 (§5). -> 5. **Priorities:** 256 fixed levels; any capability to a thread lends its priority ceiling; no scheduling right (§6). +> 3. **Time is a capability:** a `SchedContext`, configured only through its core's `SchedControl` (§5). +> 4. **Budgets are caps, not minima:** a guaranteed minimum is admission policy, unlike research/0002 (§5). +> 5. **Priorities:** 256 fixed levels; any capability to a thread lends its priority ceiling; no new right (§6). > 6. **Endpoint queues are priority-ordered,** first come first served within a level, as in seL4 MCS (§6). -> 7. **Donation:** `call` lends time to a passive server; a caller's death never takes it back mid-request (§7). -> 8. **Budget guard:** an endpoint's `min_budget` is checked at `call`; expiry inside a server fails one hop back (§7). -> 9. **Timer:** tickless per-core deadlines; userspace timers will be deadline objects on the same queue (§8). +> 7. **Donation:** `call` lends time to a *passive* server; a caller's death never takes it back mid-request (§7). +> 8. **Budget guard:** endpoints carry a `min_budget` checked at `call`; expiry fails one hop back (§7). +> 9. **Timer:** tickless per-core deadlines; userspace timers become deadline objects on the same queue (§8). > 10. **FP/SIMD:** off by default, enabled per thread, saved lazily; the kernel touches it only to save it (§10). -> 11. **Interrupts and cores:** a device line signals a notification; a big lock; cores start when first funded (§11, §12). +> 11. **Interrupts and cores:** a line signals a notification; one big lock; a core starts when funded (§11–§12). > 12. **Amendments:** three to RFC-0004, a note on RFC-0003 §14.2, a correction to research/0002 (§13). ## 1. The question @@ -32,18 +32,16 @@ not granted, a server never runs on time nobody paid for, and the kernel never waits on anything a process controls?** -That is O-7's scheduler half plus RFC-0004's three scheduling hazards: priority-aware direct switch, donation, -and budget expiry inside a server. This RFC designs the mechanism. It chooses no priorities, admits no -workloads and balances no load; that is for whoever holds the capabilities defined here. +That is O-7's scheduler half plus RFC-0004's three hazards: direct switch, donation, and budget expiry inside a +server. This RFC designs the mechanism; priorities, admission and load are for whoever holds its capabilities. ## 2. Which pillar **The kernel doctrine (§3) directly, and pillar 2 by consequence.** "IPC is the product" holds only if a `call` hands over the CPU and the budget without a scheduler round trip (RFC-0004). Time is the one resource every -thread consumes; if it is not a capability, pillar 2 has a hole exactly where availability (A5) lives. seL4 MCS -made time a capability, and this RFC adopts that shape. **Borrow Ledger:** scheduler, timer and -interrupt-controller code are the microkernel-core row — *write ourselves*; Redox is reference only, since the -kernel masks and signals but never services a device. +thread consumes; if it is not a capability, pillar 2 has a hole exactly where A5 lives — so, as seL4 MCS did, +it becomes one. **Borrow Ledger:** scheduler, timer and interrupt-controller code are the microkernel-core row, +*write ourselves*; Redox is reference only, since the kernel masks and signals but never services a device. ## 3. The execution model — one kernel stack per core (verdict 1) @@ -99,8 +97,9 @@ slot in RFC-0005 (proposed)'s typed array for threads, charged to the Pool prese | `Exited` | finished; in no queue; the object lingers until destroyed | never | Faults reuse IPC: the kernel `call`s `fault_ep` for the thread, which waits `BlockedOnReply` (RFC-0007 defines -the message); fault and timeout messages bypass `min_budget`. Destroying a thread bumps its generation (RFC-0003 -§7), unlinks it in O(1) and destroys any reply object it waits on (RFC-0004 §8); §7 governs a context it lent. +the message); a fault donates as `call` does, a timeout fault never, and both bypass `min_budget`. Destroying +a thread bumps its generation (RFC-0003 §7), unlinks it in O(1) and destroys any reply object it waits on +(RFC-0004 §8); §7 governs a context it lent. **Every internal link survives slot reuse:** queue links, `sc`, `home_sc`, a reply object's record of a lent context, an `IrqHandler`'s notification and a core's FP owner are each a counted reference holding the slot (RFC-0005 §5) or an `(index, generation)` pair checked on use — never a bare index. @@ -130,21 +129,19 @@ needing `WRITE` on its thread, or its spawner sets it before `resume`. ## 5. Scheduling contexts and `SchedControl` — time as a capability (verdicts 3, 4) **Options.** A timeslice on the thread (classic L4, Zircon) makes creating a thread *acquiring CPU*, with nothing -to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are **subsumed**: a -partition is what a `SchedControl` holder builds from contexts. **Verdict sought:** contexts as objects *(seL4 -MCS: Lyons, McLeod, Almatary & Heiser, "Scheduling-context capabilities", EuroSys 2018)*. - -A **SchedContext** holds budget `B` and period `T` (`B ≤ T`), at most eight refills, its core and its bound -thread. A thread with a bound context is **active**; one without is **passive** and runs only on donated time -(§7). **The ABI unit is the nanosecond** (`u64`) for `B`, `T` and `min_budget`, converted to counter ticks once, -in `sc_configure` and at endpoint creation — budgets round down, periods up — so no per-machine frequency -(62.5 MHz on QEMU's `cortex-a72`; a TSC rate on x86_64) enters the ABI. +to grant or withhold — **rejected** on O-4. Partition budgets (QNX adaptive partitioning) are **subsumed**, as +what a `SchedControl` holder builds from contexts. **Verdict sought:** contexts as objects *(seL4 MCS: Lyons, +McLeod, Almatary & Heiser, "Scheduling-context capabilities", EuroSys 2018)*. A **SchedContext** holds budget +`B` and period `T` (`B ≤ T`), at most eight refills, its core and its bound thread; a thread with one is +**active**, one without **passive**, running only on donated time (§7). **The ABI unit is the nanosecond** +(`u64`) for `B`, `T` and `min_budget`, converted to ticks once, in `sc_configure` and at endpoint creation — +budgets round down, periods up — so no per-machine frequency enters the ABI. **What a budget means — caps, not minima (verdict 4).** research/0002 Part 6 asks for guaranteed reservations with capping left to the broker. This kernel enforces the **cap**, because a sporadic cap is what makes fixed-priority admission sound: a holder can promise a minimum only if every more urgent context is capped. The **minimum** is admission, the `SchedControl` holder's policy. Dispatch is work-conserving *only among threads -with released budget*: a spent partial context leaves its core idle by design. No budget carries past its window. +with released budget*: a spent partial context leaves its core idle by design, and no budget carries over. | Aspect | Rule | |--------|------| @@ -197,7 +194,9 @@ priority inheritance; seL4 MCS passive servers.)* (`reply_push`, `src/object/reply.c`, donates only to a receiver with none): a thread holds one context, and an active server chose to pay for itself. RFC-0004 §4 and §8 lend unconditionally, so this amends them. **The cost:** an active server inherits nothing, and a low-priority one blocks urgent clients; Constitution §3's -inheritance holds for passive servers only. **Inversion, bounded.** Priority-ordered queues serve the most +inheritance holds for passive servers only. + +**Inversion, bounded.** Priority-ordered queues serve the most urgent waiter next, but the request in service runs on its client's context and a middle-priority thread can preempt it. `prio(S)` is therefore a **ceiling floor**, bounding inversion to one request when set to the highest client priority *(the priority-ceiling bound seL4 MCS relies on)* — **required, not optional, for a @@ -230,9 +229,8 @@ thresholds on endpoints for SC Donation", seL4/rfcs pull request 24, open): - **`min_budget` 0 prevents nothing:** one client can strand a shared server for all until a handler funds it (O-26, A5). Zero suits a server with one client, or one under a supervisor. -**Rejected: the unwind without timeout faults** — a supervisor woken by a message beats one polling counters. -**Deferred: RFC-14's budget limits** (capping a server's spend of a donated context): the RFC-14 discussion -reports 22% extra fastpath cost; the per-operation figures are to be re-checked (Open question 3). +**Rejected:** the unwind without timeout faults — a woken supervisor beats a polling one. **Deferred:** RFC-14's +budget limits, at a reported 22% extra fastpath cost, per-operation figures to be re-checked (Open question 3). ## 8. The timer and preemption (verdict 9) @@ -268,16 +266,15 @@ step 5 says `nTWE = 0` today and needs correcting. ## 9. The context switch — what is saved, where -In an event kernel a context switch is *which frame the exit path restores*; assembly is confined to the entry and -exit stubs in `kernel/src/arch/**`. **The user frame lives in the TCB.** On AArch64, while a thread runs at EL0, -`SP_EL1` points at the end of its TCB frame, so the entry stub's stores land there before it loads the per-core -kernel stack from the block `TPIDR_EL1` addresses; with `SA` on and the frame 16-byte aligned, the current-EL -`SP_ELx` synchronous vector reports a frame overwrite if `SP` lies inside a TCB. On x86_64 an interrupt from -ring 3 loads `RSP` from `TSS.RSP0`, the TCB frame; **`syscall` does not switch `RSP`**, so its stub runs -`swapgs`, parks the user `rsp` in per-CPU scratch, loads the TCB frame pointer, saves into it, then loads the -kernel stack. RFC-0007 (proposed) owns both entry sequences; this RFC owns the layout and §3's current-EL IRQ -stub. The address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005 (proposed)'s, made on exit -when the process differs — where a time-protection flush would go (Ge et al., EuroSys 2019). +A context switch is *which frame the exit path restores*; assembly stays in the entry and exit stubs in +`kernel/src/arch/**`. **The user frame lives in the TCB.** At EL0, `SP_EL1` points at the end of the thread's TCB +frame, so the entry stub's stores land there before it loads the per-core stack from the `TPIDR_EL1` block; with +`SA` on and the frame 16-byte aligned, the current-EL `SP_ELx` synchronous vector reports a frame overwrite if +`SP` lies inside a TCB. On x86_64 an interrupt from ring 3 loads `RSP` from `TSS.RSP0`, the TCB frame; **`syscall` +does not switch `RSP`**, so its stub runs `swapgs`, parks the user `rsp` in per-CPU scratch, loads the frame +pointer, saves, then loads the kernel stack. RFC-0007 (proposed) owns both entry sequences; this RFC owns the +layout and §3's IRQ stub. The address-space switch (`TTBR0_EL1` with ASID; `CR3` with PCID) is RFC-0005's, made +on exit when the process differs — where a time-protection flush would go (Ge et al., EuroSys 2019). | What | AArch64 | x86_64 | |------|---------|--------| @@ -289,93 +286,87 @@ when the process differs — where a time-protection flush would go (Ge et al., | **TCB size** | 36-word frame (288 B) + FP area padded to 528 B + ~180 B of fields: **1 KiB** | 23-word frame (184 B, padded to 192) + 832 B XSAVE + ~180 B: **1.25 KiB** | **O-5 on frames.** `write_regs` never lets userspace choose its privilege: the kernel builds `SPSR_EL1` (EL0t, -DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only status flags -(IOPL 0, IF set), and a non-canonical `RIP` or base is refused. It writes `TPIDR_EL0` and `FS_BASE` only: -`TPIDRRO_EL0` and the user GS base are the kernel's IPC-buffer discovery registers (RFC-0007 §5). NMI, `#DB`, -`#DF` and `#MC` run on IST stacks (RFC-0007 asks): they can arrive before `syscall`'s stack switch. +DAIF clear, only NZCV from the caller); on x86_64 `CS`/`SS` are constants, `RFLAGS` takes only status flags (IOPL +0, IF set), and a non-canonical `RIP` or base is refused. It writes `TPIDR_EL0` and `FS_BASE` only; `TPIDRRO_EL0` +and the user GS base are the kernel's IPC-buffer discovery registers (RFC-0007 §5). NMI, `#DB`, `#DF` and `#MC` +use IST stacks, as they can arrive before `syscall`'s stack switch. ## 10. FP/SIMD — owned by userspace, enabled per thread, saved lazily (verdict 10) - **Enabled per thread** by `set_fp`, off by default. `CLAUDE.md` says "per process"; per thread is finer, so an - FP-off server thread in an FP-using process never forces a save — its § Architectures wording changes with - this RFC. A thread with FP off that touches FP takes an ordinary fault. + FP-off server thread in an FP-using process never forces a save, and § Architectures changes with this RFC. A + thread with FP off that touches FP takes an ordinary fault. - **Loaded on first use, saved lazily.** Each core records an FP *owner* as an `(index, generation)` pair; every - other thread runs trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`, which the reporter already names) or - `CR0.TS = 1` (`#NM`, vector 7). `0b00` traps EL1 too and `XSAVE` raises `#NM` while `TS` is set, so the - handler first grants itself access — `FPEN = 0b01` (EL1 permitted, EL0 still trapped) and `isb`, or `clts` — - then saves a live owner's registers, loads the new owner's or zeros, sets `FPEN = 0b11`, `isb`, and returns. - A `call` to an FP-off server saves nothing. -- **The owner is cleared,** its registers unsaved, when the owner thread is destroyed or `set_fp(off)`; a thread - reusing its slot then fails the generation check and loads zeros, never a dead thread's registers (O-8, O-9). -- **The kernel's only FP instructions** are those save and restore stubs in `kernel/src/arch/**`, run only with - access granted; the soft-float target is the guarantee for generated code. No `FPEN` value traps EL1 while - allowing EL0, so while an owner runs, a stray kernel FP instruction would not trap. -- **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format area (512 - legacy + 64 header + 256 AVX). AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. -- **LazyFP** (CVE-2018-3665; Stecklina & Prescher, 2018): on affected Intel cores a non-owner can read the - owner's registers speculatively before `#NM` resolves. Out of scope (threat model §7); Open question 2. + other thread runs trapped — `CPACR_EL1.FPEN = 0b00` (EC `0x07`) or `CR0.TS = 1` (`#NM`). Since `0b00` traps EL1 + too and `XSAVE` raises `#NM` under `TS`, the handler first grants itself access (`FPEN = 0b01`, EL0 still + trapped, and `isb`; or `clts`), saves a live owner, loads the new owner's registers or zeros, sets `FPEN = + 0b11`, `isb`, and returns. A `call` to an FP-off server saves nothing. +- **The owner is cleared,** unsaved, when it is destroyed or `set_fp(off)`; a thread reusing its slot fails the + generation check and loads zeros, never a dead thread's registers (O-8, O-9). +- **The kernel's only FP instructions** are those stubs in `kernel/src/arch/**`, run with access granted; the + soft-float target guarantees generated code. No `FPEN` value traps EL1 while allowing EL0. +- **Extent.** x86_64 sets `CR4.OSFXSR`, `CR4.OSXSAVE` and `XCR0 = 0b111` — the 832-byte standard-format area. + AVX-512, AMX and SVE (`CPACR_EL1.ZEN`) stay disabled and fault. **LazyFP** (CVE-2018-3665; Stecklina & + Prescher, 2018) lets a non-owner read the owner's registers speculatively on affected Intel cores: out of + scope (threat model §7), Open question 2. ## 11. Interrupts — the controller, and device IRQs as notifications (verdict 11) The kernel owns the controller and nothing behind it *(seL4 `IRQControl`/`IRQHandler`)*. -- **Objects.** One `IrqControl` (the root task's) mints at most one `IrqHandler` per device line. Kernel-reserved - lines — the timer PPI, the IPI SGIs or vector, spurious INTID 1023 — are never minted. `bind(irq, - notification, bits)` invokes the `IrqHandler` (`WRITE`) and presents the notification (`WRITE`); `ack(irq)` - needs `WRITE`. **Phase 1 routes every line to the boot core**; SMP routing is `IrqControl`'s, never a - `SchedControl` method (Open question 6). Shared PCI INTx lines on `q35` are **unsupported** until MSI. +- **Objects.** One `IrqControl` (the root's) mints at most one `IrqHandler` per device line; the timer PPI, IPI + SGIs or vector and spurious INTID 1023 are never minted. `bind(irq, notification, bits)` invokes the + `IrqHandler` (`WRITE`) and presents the notification (`WRITE`); `ack(irq)` needs `WRITE`. **Phase 1 routes + every line to the boot core**; SMP routing is `IrqControl`'s, never `SchedControl`'s (Open question 6). Shared + PCI INTx lines on `q35` are **unsupported** until MSI. - **Delivery.** The kernel acknowledges, **masks the line**, signals end of interrupt, ORs `bits` into the - notification and wakes a waiter through §6's dispatcher; the driver's `ack` unmasks. A level-triggered line - cannot storm, though it fires once per `ack` (§17). A destroyed notification leaves the line masked (fail - closed); the `IrqHandler` outlives a crashed driver, and a restarted one re-binds (O-17). -- **AArch64: GICv3 first.** `GICD_CTLR` gets `ARE` and `EnableGrp1` (`virt` without `secure=on` has one security - state, `DS = 1`); SPIs through `GICD_ISENABLER`, `GICD_IPRIORITYR`, `GICD_IROUTER`; the redistributor - woken by `GICR_WAKER`, SGIs and PPIs through `GICR_ISENABLER0` and `GICR_IPRIORITYR`. Each enabled line - goes in Group 1 explicitly (Group 0 arrives as FIQ), at a priority numerically below `ICC_PMR_EL1`'s mask. CPU - interface: `ICC_SRE_EL1.SRE`, `ICC_IGRPEN1_EL1`, `ICC_IAR1_EL1`, `ICC_EOIR1_EL1`. On `virt`: distributor - `0x0800_0000`, redistributors from `0x080A_0000` (QEMU `hw/arm/virt.c`), through RFC-0005 (proposed)'s MMIO - window. *(Arm IHI 0069.)* **Cost:** `virt` picks GICv2 at eight CPUs or fewer (`finalize_gic_version_do`), - so xtask gains `gic-version=3`; the Raspberry Pi 4 has a GIC-400 (GICv2) — a **known** second driver. -- **x86_64: LAPIC and I/O APIC.** The 8259s are masked and never driven; I/O APIC entries carry the mask bit; end - of interrupt goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI/MSI-X wait for the driver RFC:** without IOMMU - interrupt remapping a device can aim any vector at any core — O-18's precondition. -- **Latency** is §3's worst path between preemption points. RFC-0007 (proposed)'s console write must take one - per FIFO-full poll, restartable by byte count; until then it holds the kernel about 44 ms per 504 bytes on - real hardware, and nothing on QEMU, which has no baud delay. + notification and wakes a waiter; the driver's `ack` unmasks. A level-triggered line cannot storm, though it + fires once per `ack` (§17). A destroyed notification leaves the line masked; the `IrqHandler` outlives a + crashed driver, and a restarted one re-binds (O-17). +- **AArch64: GICv3 first.** `GICD_CTLR.ARE` and `EnableGrp1` (`virt` without `secure=on` has one security state, + `DS = 1`); SPIs through `GICD_ISENABLER`, `GICD_IPRIORITYR`, `GICD_IROUTER`; the redistributor woken + by `GICR_WAKER`, SGIs and PPIs through `GICR_ISENABLER0` and `GICR_IPRIORITYR`; every line in Group 1 (Group + 0 arrives as FIQ) at a priority numerically below `ICC_PMR_EL1`'s mask; `ICC_SRE_EL1.SRE`, `ICC_IGRPEN1_EL1`, + `ICC_IAR1_EL1`, `ICC_EOIR1_EL1`. On `virt`: distributor `0x0800_0000`, redistributors from `0x080A_0000` (QEMU + `hw/arm/virt.c`), via RFC-0005's MMIO window. *(Arm IHI 0069.)* **Cost:** `virt` picks GICv2 at eight CPUs or + fewer, so xtask adds `gic-version=3`; the Raspberry Pi 4's GIC-400 (GICv2) is a **known** second driver. +- **x86_64: LAPIC and I/O APIC.** The 8259s are masked; I/O APIC entries carry the mask bit; end of interrupt + goes to the LAPIC (x2APIC EOI MSR `0x80B`). **MSI waits for the driver RFC:** without interrupt remapping a + device can aim any vector at any core — O-18's precondition. +- **Latency** is §3's worst path between preemption points. RFC-0007's console write must take one per + FIFO-full poll, restartable by byte count; until then it holds the kernel about 44 ms per 504 bytes on + hardware (nothing on QEMU, which has no baud delay). ## 12. Multi-core shape, placement and exclusive cores (verdict 11) Phase 1 runs `-smp 1`; the shape is fixed now so multicore is not bolted on. -- **Everything scheduling-related is per core.** A thread runs on the core of the context it holds, so donation - can move a server thread between cores. **The kernel never load-balances.** Cross-core wake-ups (RFC-0004 - §9.2) enqueue *W* on its core and send an IPI — an SGI through `ICC_SGI1R_EL1`; a fixed vector through the - LAPIC ICR. Direct switch is same-core only. -- **One big kernel lock when SMP lands**, taken on every entry, the IPC fast path included *(Peters, Danis, - Elphinstone & Heiser, "For a Microkernel, a Big Lock Is Fine", APSys 2015)*. This **rejects the premise** of - RFC-0003 §14.2, which asks for sharing without a global lock on the fast path; within one entry the - resolve→check→act window holds under the lock, and §3 covers restarts (amendment A4). -- **A core waiting for the lock services IPIs while it spins** *(as seL4's CLH lock does)*, because two operations - need a synchronous answer from another core: RFC-0005 (proposed)'s x86_64 TLB shootdown, which has no - broadcast, and **FP state live on another core** — a thread whose registers core X owns lazily, run next on - core Y, has Y ask X by IPI to save them. Cost: a cross-core FP handoff is an IPI round trip. +- **Everything scheduling-related is per core.** A thread runs on its context's core, so donation can move a + server between cores; **the kernel never load-balances.** Cross-core wake-ups (RFC-0004 §9.2) enqueue *W* on + its core and send an IPI (`ICC_SGI1R_EL1`; the LAPIC ICR). Direct switch is same-core only. +- **One big kernel lock when SMP lands**, on every entry, the fast path included *(Peters, Danis, Elphinstone & + Heiser, "For a Microkernel, a Big Lock Is Fine", APSys 2015)*. This **rejects the premise** of RFC-0003 + §14.2, sharing without a global lock on the fast path; resolve→check→act holds within an entry, and §3 covers + restarts (A4). +- **A core waiting for the lock services IPIs while it spins** *(as seL4's CLH lock does)*: two operations need + a synchronous answer from another core — RFC-0005's x86_64 TLB shootdown, lacking a broadcast, and **FP state + live elsewhere**: a thread whose registers core X owns lazily, run next on core Y, has Y ask X by IPI to save + them. Cost: a cross-core FP handoff is an IPI round trip. - **Cores are parked until funded (default-deny).** On QEMU `virt` with `-kernel` the DTB advertises PSCI and - secondaries start powered off — checked on QEMU 8.2.2: CPU 1 never leaves the entry address — so `boot.s`'s - `wfe` loop matters only on spin-table firmware. The first `sc_configure` onto a core starts it: PSCI `CPU_ON`, - conduit from the DTB's `/psci` node; INIT-SIPI-SIPI on x86_64. Deferred to increment 14, `boot.s`'s comment - with it. + secondaries start powered off (checked on QEMU 8.2.2: CPU 1 never leaves the entry address), so `boot.s`'s + `wfe` loop matters only on spin-table firmware. The first `sc_configure` onto a core starts it — PSCI + `CPU_ON`, conduit from `/psci`; INIT-SIPI-SIPI on x86_64 — in increment 14, `boot.s`'s comment with it. - **SMT placement, one sentence now** (research/0002 Part 6, after L1TF and MDS): a context will carry a class - set only by the `SchedControl` holder; SMT siblings never run different classes, and donation never crosses - one. The field is reserved zero until increment 14, so no thread labels itself; SMT-off is "never fund the - sibling". + set only by the `SchedControl` holder; SMT siblings never run different classes, donation never crosses one. + The field is reserved zero until increment 14, so no thread labels itself; SMT-off is "never fund the sibling". - **Exclusive assignment is a degenerate case, not a mode** (research/0002 Part 6, after Jailhouse): one thread, - a full context, nothing else on the core. The bitmap has one bit, no timer is armed, no IPI arrives unless the - `SchedControl` holder acts, lines are routed elsewhere, and counters advance only at the next kernel entry. A - recipient checks the promise with `ctl_info`: one context, reference count 1. + a full context, nothing else on the core — one bitmap bit, no timer armed, no IPI unless the `SchedControl` + holder acts, lines routed elsewhere, counters updated only at the next entry. A recipient checks the promise + with `ctl_info`: one context, reference count 1. ## 13. Amendments proposed to accepted documents (verdict 12) -Each, if accepted, is appended to its target as a dated amendment and logged in `docs/CHANGELOG.md` under Changed. +Each, if accepted, is appended to its target as a dated amendment and logged in `docs/CHANGELOG.md` (Changed). | # | Target | Change | Why | |---|--------|--------|-----| @@ -409,35 +400,28 @@ Each, if accepted, is appended to its target as a dated amendment and logged in ## 16. Graves checked (§3) -- **Policy in the kernel.** No admission, balancing or budget choice. The constants the kernel does choose: 256 - levels, eight refills, `MIN_BUDGET`'s factor 2, the XCR0 extent, the thread and deadline array capacities - (RFC-0005), and §19's interim constants until measured. -- **The catch-all right.** Priority authority is a relation (MCP); `SchedControl` has one mutating method, selling - time on one core, and a read of its own state; topology is data, routing is `IrqControl`'s; §4's list is closed. -- **Bolted-on multicore.** Per-core state, IPIs, the lock and its IPI-servicing spin, FP handoff, shootdowns and - core start-up are stated before any SMP code. -- **Drivers in the kernel; unused device paths.** The kernel masks and signals, never services a device; one - controller and one timer per target; unclaimed lines never enabled. -- **Baroque hierarchies; multi-copy IPC.** Contexts are flat; donation moves a reference to time. +| Grave | How this RFC stays out of it | +|-------|------------------------------| +| **Policy in the kernel** | no admission, balancing or budget choice; the constants it does choose are 256 levels, eight refills, `MIN_BUDGET`'s factor 2, the XCR0 extent, the thread and deadline array capacities (RFC-0005), and §19's interim constants until measured | +| **The catch-all right** | priority authority is a relation (MCP); `SchedControl` has one mutating method, selling time on one core, and a read of its own state; topology is data, routing is `IrqControl`'s; §4's list is closed | +| **Bolted-on multicore** | per-core state, IPIs, the lock and its IPI-servicing spin, FP handoff, shootdowns and core start-up are stated before any SMP code | +| **Drivers in the kernel; unused device paths** | the kernel masks and signals, never services a device; one controller and one timer per target; unclaimed lines never enabled | +| **Baroque hierarchies; multi-copy IPC** | contexts are flat; donation moves a reference to time | ## 17. Costs — what this makes harder -- **Every server needs someone to think about time.** A passive server cannot run until called; an active one - needs a configured context and inherits nothing (A1). "Spawn a thread and it runs" is gone. -- **A stranded server is userspace's problem,** and `min_budget` 0 lets one client strand it for all: a - supervisor is required before any passive server on partial contexts is trusted. -- **Caps, not minima:** a spent partial context idles its core. **Priority-ordered queues** cost an O(n) walk - with interrupts masked, *n* up to the thread capacity. -- **Interrupt time is billed to whoever runs:** a driver `ack`ing a level line in a loop charges other - processes, one interrupt per `ack` — seL4's choice; passive drivers (Open question 4) could bill the driver. -- **No userspace timer yet:** RFC-0004 §8's load-bearing watchdog lacks a source until increment 15 — an unmet - RFC-0004 dependency, named. -- **Exclusivity is checkable, not revocable:** `ctl_info` shows whether a grantor kept a copy; if one did, it can - still configure onto the core. -- **MCS is seL4's least-verified part**, and sporadic refills are subtle — hence property tests. -- **About 1 KiB per TCB on AArch64 and 1.25 KiB on x86_64**, FP space included for threads that never use it. -- **Lazy FP's cure:** eager restore into an FP thread of another process moves 520 bytes or an 832-byte - `XRSTOR` (9 or 13 cache lines), beside a `call` round trip of roughly 380–630 cycles (research/0001). +| Cost | Detail | +|------|--------| +| **Every server needs someone to think about time** | a passive server cannot run until called; an active one needs a configured context and inherits nothing (A1); "spawn a thread and it runs" is gone | +| **A stranded server is userspace's problem** | `min_budget` 0 lets one client strand it for all; a supervisor is required before any passive server on partial contexts is trusted | +| **Caps, not minima** | a spent partial context idles its core even when nothing else would run | +| **Priority-ordered endpoint queues** | an O(n) walk with interrupts masked, *n* up to the thread capacity | +| **Interrupt time is billed to whoever runs** | a driver `ack`ing a level line in a loop charges other processes, one interrupt per `ack` — seL4's choice; passive drivers (Open question 4) could bill the driver | +| **No userspace timer yet** | RFC-0004 §8's load-bearing watchdog lacks a source until increment 15 — an unmet RFC-0004 dependency | +| **Exclusivity is checkable, not revocable** | `ctl_info` shows whether a grantor kept a copy; if one did, it can still configure onto the core | +| **MCS is seL4's least-verified part** | sporadic refills are subtle — hence property tests | +| **About 1 KiB per TCB (1.25 KiB on x86_64)** | FP space included for threads that never use it | +| **Lazy FP's cure** | eager restore into another process's FP thread moves 520 bytes or an 832-byte `XRSTOR` (9 or 13 cache lines), beside a `call` round trip of roughly 380–630 cycles (research/0001) | ## 18. Open questions @@ -446,8 +430,7 @@ Each, if accepted, is appended to its target as a dated amendment and logged in 2. **LazyFP on x86_64.** Eager restore for cross-process switches into FP threads (costed in §17)? 3. **RFC-14 budget limits.** Worth their fastpath cost, once re-measured, to stop a server overspending? 4. **Passive drivers** need a context bound to a notification (MCS); interrupt entry could then bill the driver. -5. **Numbers from measurement:** `MIN_BUDGET`, kernel stack size, the worst path; x86_64's clock without an - invariant TSC. +5. **Numbers from measurement:** `MIN_BUDGET`, stack size, the worst path; x86_64's clock without invariant TSC. 6. **Interrupt routing on SMP:** per-(line, core) handlers minted by `IrqControl`, or a per-core interrupt-target capability — decided with increment 14. @@ -487,9 +470,8 @@ counted boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The x86_6 - **Provisional constants** in `scheduler/src/config.rs` until measured: `MIN_BUDGET_INTERIM` = 100 µs; `ROOT_TIMESLICE` = 5 ms, a full context; root priority and MCP 255, the root putting itself and the client at 100 to share a level; `KERNEL_STACK_SIZE` = 16 KiB. Cost: the budget floor is a guess. -- **No fault handler:** a faulting thread goes `Inactive` and the reporter prints it (RFC-0007 §7). -- **A non-preemptible console write** until RFC-0007 adds its preemption points — free on QEMU, 44 ms on - hardware; **no userspace timer**, so RFC-0004's watchdog cannot be built; **x86_64 stubs** that halt, until 13. +- **No fault handler** (a faulting thread goes `Inactive`, RFC-0007 §7); **a non-preemptible console write** + (free on QEMU, 44 ms on hardware); **no userspace timer** for RFC-0004's watchdog; **x86_64 stubs** that halt. ## 20. What this unblocks diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index f1670fc..3709b28 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -7,23 +7,25 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 (capabilities are the only authority), pillar 2; `CLAUDE.md` § `unsafe` policy and § Layout; the workspace (`user/`); `xtask` and `kernel/build.rs`; the kernel's trap path; the accepted `capability` crate (the handle word, a slot-generation bound, one error variant split) | -| Depends on | RFC-0003 (accepted; §9 binds every syscall); RFC-0004 (accepted; §5, §6, §8, §9.4, §10 and the 2026-08-01 amendment); RFC-0005 and RFC-0006 (proposed, drafted alongside) | -| Discharges | O-5 (argument validation) for the syscall surface; O-27 (explicit authority at spawn); binds O-1, O-2 and O-4 at B1; contributes to O-7, O-9 and O-13 | +| Affects | Constitution §3 (capabilities are the only authority; the Language clause), §4 (a Borrow Ledger reading, §12); `CLAUDE.md` § `unsafe` policy and § Layout; threat model O-6's text; RFC-0003 (a dated amendment: the handle word, a slot-generation bound, `free_slots`, one error variant split); RFC-0004 §8 (a dated amendment: reply objects are unbound, not destroyed); `docs/CHANGELOG.md`; the workspace (`user/`), the comments in `Cargo.toml` and `xtask/Cargo.toml`, `kernel/src/main.rs`'s grep scope; `xtask`, `kernel/build.rs` and `aarch64.ld`; the kernel's trap path | +| Depends on | RFC-0003 (accepted; §9 binds every syscall); RFC-0004 (accepted; §5, §6, §8, §9.4, §10 and the 2026-08-01 amendment); RFC-0005 and RFC-0006 (proposed, drafted alongside); a future x86_64 boot RFC for the UEFI stub | +| Discharges | O-5 (argument validation) for the syscall surface; O-27 (explicit authority at spawn); binds O-1, O-2 and O-4 at B1; keeps O-6 Built with a third designated tree; contributes to O-7, O-9 and O-13; designs O-17's fault path | > **Proposed verdicts.** > > 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§4). -> 2. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§3). -> 3. **Messages:** four physical and sixty-four virtual message registers, three capabilities; a 1 KiB IPC buffer, every word fetched once (§5). -> 4. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§5). -> 5. **Handle word:** 20-bit index, 44-bit *slot* generation, `0` is null; the crate retires slots at 2^44 − 1, objects keep 64 bits (§6). -> 6. **Errors in two layers:** transport status in a register, method result in the reply label; `BudgetRefused` beside `BudgetExpired`; faults are not errors (§7). -> 7. **Process object:** table + address space + threads, born empty, grant window shut by the first `resume` of any of its threads (§8). -> 8. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§9). -> 9. **Boot images:** `xtask` parses ELF on the host and embeds a bundle of fixed-shape descriptors in the kernel ELF; the kernel never parses ELF (§10). -> 10. **Console as a capability:** a kernel object with one `WRITE` method, scheduled for deletion when the Phase-2 UART driver lands (§11). -> 11. **`user/abi`** is the microkernel-core row, not libc/runtime; its `src/arch/**` is the third designated `unsafe` tree (§12). +> 2. **Method arguments in registers:** a kernel method's handle words ride in MR0–MR3, *presented* — resolved and checked, never moved — unless its row says *moved* (§4). +> 3. **Reply objects are reusable:** `reply`, budget expiry and caller death unbind them, never destroy them — a dated RFC-0004 amendment (§4). +> 4. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§3). +> 5. **Messages:** four physical and sixty-four virtual message registers, three capabilities, an optional 1 KiB IPC buffer; transfers resolved at rendezvous, every word fetched once (§5). +> 6. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§5). +> 7. **Handle word:** 20-bit index, 44-bit slot generation; word `0` is null; recorded as a dated RFC-0003 amendment (§6). +> 8. **Errors in two layers:** numbered transport status in a register, method result in the reply label; faults are not errors (§7). +> 9. **Process object:** table + one AddressSpace for life + threads, born empty; `GRANT` only until the first `resume`; once started, `WRITE` on it means `KILL` (§8). +> 10. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§9). +> 11. **Boot images:** `arch::boot_modules()` yields fixed-shape descriptors a host-side ELF reader made; the kernel never parses ELF; AArch64 embeds them as an interim (§10). +> 12. **Console as a capability:** one `WRITE` method of at most 24 bytes that never waits; deleted when the Phase-2 UART driver lands (§11). +> 13. **`user/abi`** is the microkernel-core row; its `src/arch/**` — trap instructions and entry stubs only — is the third designated `unsafe` tree (§12). ## 1. The question @@ -44,26 +46,28 @@ traversal), and O-27 exists because of it. Boot images reach userspace read-only ## 3. Entering the kernel -**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` (lower EL, AArch64 — entry 8 of the -table `vectors.s` installs) with `ESR_EL1.EC = 0x15`, the immediate in `ISS[15:0]` and `ELR_EL1` past the -`svc`; entry masks DAIF and selects `SP_EL1` *(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, -`ELR_EL1` and `SPSR_EL1` into the thread's frame (RFC-0006 (proposed) provides its layout, in the TCB) and -dispatches on EC: `0x15` to the syscall dispatcher, anything else is a *fault* (§7). A non-zero immediate -answers `InvalidArgument`, so a later ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the -kernel builds: `M` = EL0t, DAIF clear, NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5), which -EL0 sets itself with `msr`, carried where implemented; every other bit (`SS`, `IL`, `PAN`, `UAO`, `TCO`, -`BTYPE`) zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. +**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` (lower EL, AArch64) with +`ESR_EL1.EC = 0x15`, the immediate in `ISS[15:0]` and `ELR_EL1` past the `svc`; entry masks DAIF and selects +`SP_EL1` *(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the +thread's frame (RFC-0006 (proposed) §9 owns its layout, in the TCB) and dispatches on EC: `0x15` to the +syscall dispatcher, anything else is a *fault* (§7). A non-zero immediate answers `InvalidArgument`, so a +later ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the kernel builds: `M` = EL0t, DAIF +clear, NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5) carried where implemented; every other bit +zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. **x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits, takes `CS`/`SS` from -`IA32_STAR`, and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. The -stub begins with `swapgs`, parks the user `rsp` in per-CPU scratch and loads the kernel stack, never -dereferencing the user's. `FMASK` clears IF, DF, TF and AC — user access closed where SMAP exists. A -**non-canonical `rcx`** makes Intel's `sysretq` raise #GP *in ring 0* on the user's stack (CVE-2012-0217, -which took Xen, FreeBSD, NetBSD and Windows): RFC-0005 (proposed) leaves the top user page unmapped and -RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, and the exit path checks anyway, falling back to -`iretq`. The window before the stack switch gets IST stacks from RFC-0006 (proposed). **Rejected:** -`sysenter`, which AMD does not implement in long mode; an `int` gate, a full IDT delivery per call. +`IA32_STAR`, and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. As +RFC-0006 (proposed) §9 requires, the stub runs `swapgs`, parks the user `rsp` in per-core scratch, loads the +TCB frame pointer, saves into the frame, and only then loads the per-core kernel stack; it never +dereferences the user stack. `FMASK` clears IF, DF, TF and AC. **The exit uses `sysretq` only when `rcx` +is canonical and the frame's `RFLAGS` has TF and RF clear and IOPL 0; otherwise `iretq`.** A non-canonical +`rcx` makes Intel's `sysretq` raise #GP *in ring 0* on the user's stack (CVE-2012-0217, which took Xen, +FreeBSD, NetBSD and Windows); `sysretq` loads `RFLAGS` from `r11 AND 3C7FD7H OR 2` (SDM Vol. 2B, SYSRET), +which passes TF, so a user-set TF would trap straight after it — Linux takes `iretq` for TF or RF likewise +(`arch/x86/entry/entry_64.S`). RFC-0005 leaves the top user page unmapped and RFC-0006's `write_regs` +sanitises `RIP` and `RFLAGS`; the exit checks anyway. **Rejected:** `sysenter`, which AMD does not +implement in long mode; an `int` gate, a full IDT delivery per call. | Role | AArch64 | x86_64 | |------|---------|--------| @@ -71,23 +75,26 @@ RFC-0006 (proposed)'s `write_regs` refuses such a `RIP`, and the exit path check | invoked handle (in) / status (out) | `x0` | `rdi` in, `rax` out | | message tag (in/out) | `x1` | `rsi` | | MR0–MR3 (in/out) | `x2`–`x5` | `rdx`, `r10`, `r8`, `r9` | -| second handle: reply object (in) / badge (out) | `x6` | `r12` | +| reply-object handle (in) | `x6` | `r12` | | error detail (out) | `x7` | `r13` | | clobbered by hardware | — | `rcx`, `r11` | -Every register outside the result set is restored from the frame; the eight result registers carry a -result or zero, so no kernel value leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux; the -x86_64 column avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and what `syscall` destroys. -seL4 has the same shape in other seats — `x7`; `rdx` *(libsel4 `sel4_arch/syscalls.h`)*. **No argument -is a pointer:** only the IPC buffer is touched, by frame (§5) — on the demo CPU the whole defence, since -the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). +Every register outside the result set is restored from the frame; the result registers carry a result or +zero, so no kernel value leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux; the x86_64 column +avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and what `syscall` destroys. **No argument +is a pointer** (RFC-0005 (proposed) §11 owns the rule): only the IPC buffer is touched, by frame (§5) — on +the demo CPU the whole defence, since the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). -**Entry state of every thread,** root included: PC = entry; SP as given; `a0`, `a1` — `x0`/`x1`, -`rdi`/`rsi`, so `_start` is an ordinary two-argument `extern "C"` function — the two start values; every -other general register zero; EL0t with DAIF clear (`SPSR_EL1` = 0), or ring 3 with `RFLAGS` = `0x202`; -FP/SIMD trapped (`CPACR_EL1.FPEN` = 0; `CR0.TS` set) until RFC-0006 (proposed) enables it; `TPIDR_EL0` -and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is created so; the spawner writes PC, SP, -`a0` and `a1` through RFC-0006 (proposed)'s `write_regs`, and the kernel writes the root's (§9). +**Entry state of every thread,** root included: PC = entry; SP as given, 16-byte aligned; `a0`, `a1` +(`x0`/`x1`, `rdi`/`rsi`) the two start values; every other general register zero; EL0t with DAIF clear, or +ring 3 with `RFLAGS` = `0x202`; FP/SIMD trapped until RFC-0006 enables it; `TPIDR_EL0` and `FS_BASE` zero; +the discovery register set if a buffer is bound (§5). The entry symbol is an assembly stub in `abi` that +calls (`bl`; `call`) a Rust `extern "C" fn`, so AAPCS64 and SysV each see their entry alignment. + +**The Language clause.** The constitution allows "the minimal assembly required for boot and context +switching". A trap is the user half of a switch into the kernel, and `svc` or `syscall` has no Rust +equivalent, so the wrappers are single-instruction `asm!` intrinsics — the reading RFC-0005 (proposed) +§17.7 asks the maintainer to confirm for `TLBI`; the same confirmation covers these (Open question 4). ## 4. The syscall set @@ -95,46 +102,84 @@ and `FS_BASE` zero; the IPC-buffer register set (§5). A Thread is created so; t each entry is a new path to audit, and no server can ever stand in for a kernel object. - **B — uniform invocation** *(seL4; KeyKOS)*. A handful of IPC syscalls; a kernel-object operation is a `call` on its capability, the method in the tag's label, answered as a server would answer. -- **C — B plus an `invoke` number.** The small table without the point: callers can tell the two apart. +- **C — B plus a separate `invoke` syscall for kernel objects.** Keeps B's small table, but callers can + tell a kernel object from a server, which removes B's only advantage. -**Verdict sought: B,** for interposition: a Console and a logging server's endpoint are invoked alike, so -Phase 2's UART driver replaces the Console with no client changing, and the broker can hand out a -forwarder for any kernel object (RFC-0003a option (a)'s precondition). Such a `call` donates nothing. +**Verdict sought: B,** for interposition. A Console and a logging server's endpoint are invoked alike, so +Phase 2's UART driver replaces the Console with no client changing. The claim is scoped: **a server can +stand in for a kernel object whose methods take no handle-word arguments** — the Console, and every +protocol a server defines. It cannot stand in for `map` or `bind_sc`, whose second handle word means +something only in the caller's table (below; §16). `cap_rights` therefore reports rights, never the object +kind. A `call` on a kernel object donates no scheduling context. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| | 0 | `yield` | — | authority-free (RFC-0003 §9) | -| 1 | `send` | object | endpoint: blocking send; notification: signal (RFC-0004's `notify`) | +| 1 | `send` | object | endpoint: blocking send; notification: signal | | 2 | `nb_send` | object | as `send`, failing `WouldBlock` rather than blocking | -| 3 | `call` | object | endpoint: send, await reply, donating (RFC-0004 §8); kernel object: invoke a method | +| 3 | `call` | object | endpoint: send and await the reply; kernel object: invoke a method | | 4 | `recv` | endpoint or notification; reply object | blocking receive; a bound notification may wake it | | 5 | `nb_recv` | as `recv` | poll | -| 6 | `reply` | reply object | reply once to a caller | +| 6 | `reply` | reply object | reply once to the bound caller | | 7 | `reply_recv` | endpoint; reply object | `reply` then `recv` in one entry — the server loop *(seL4 `ReplyRecv`)* | | 8 | `cap_derive` | object, with `DUPLICATE` | in: MR0 the rights wanted, MR1 a badge (reserved, zero); out: MR0 the new handle word | | 9 | `cap_close` | object | nothing; the handle is dead on return | -| 10 | `cap_identify` | object | out: MR0 the object kind (an `abi` enumeration), MR1 the rights bits | -| 11 | `thread_exit` | — | halt the calling thread; authority-free | - -`reply_recv` makes a server loop one entry, as direct switch needs. The `cap_*` calls take and return a -zero tag and touch only the caller's own table; the dispatcher builds `Rights` from the crate's named -constants, refusing undefined bits. Transfer is a move; a sender keeping a copy derives first (RFC-0003 §6). -**Creation** is a method on a Pool, which RFC-0005 (proposed) provides; per-type parameters are encoded -here, and an endpoint's carries **`min_budget`** in MR1 (RFC-0006 (proposed)'s time unit, 0 for none, -fixed for the endpoint's life). **Rejected:** a `debug_putc` (§11); anything taking a PID, path or -address; seL4's depth-addressed capability pointers, made for CNode trees this kernel does not have. +| 10 | `cap_rights` | object | out: MR0 the rights bits | +| 11 | `thread_exit` | — | the calling thread becomes `Exited`; authority-free | + +The `cap_*` calls take and return a zero tag and touch only the caller's own table; the dispatcher builds +`Rights` from the crate's named constants, refusing undefined bits. **A signal sets bit 0** of the +notification's word until badges exist (RFC-0003a), then the capability's badge, fixed at derivation, as +in seL4. No sender chooses the bits, so the signal stays RFC-0004 §8's doorbell with no payload; an +`IrqHandler` ORs the bits bound to it (RFC-0006 (proposed) §11). **Rejected:** a `debug_putc` (§11); +anything taking a PID, path or address; seL4's depth-addressed capability pointers, made for CNode trees +this kernel does not have. + +**Reply objects.** A Reply is created once from a Pool and reused: `recv` binds it to one caller, and +`reply`, budget expiry (RFC-0006 (proposed) §7, `BudgetExpired`) or the caller's death **unbind** it. It +lives until its holder closes it; `reply` on an unbound object fails `PeerGone` *(seL4 MCS reply +objects)*. RFC-0004 §8 says the object "is destroyed if the caller dies"; destruction would let a client +that calls and exits in a loop cost the server one Pool allocation per request — an O-7 drain of the +server's Pool — so this needs a dated RFC-0004 amendment, and RFC-0006's §4 and §7 read "unbinds". + +**Kernel-object methods.** A method is a `call` whose label names it and whose MR0–MR3 carry every +argument; a method that needs more is split. A handle word among them is **presented**: resolved in the +caller's table, checked for the right its row names, never moved, never needing `TRANSFER` *(seL4's +`extraCaps` on an invocation)*. A row marked **moved** instead takes the capability out of the caller's +table, which always needs `TRANSFER`, the one right that lets authority leave a table (RFC-0003 §6). +`tag.caps` must be zero on every method (`InvalidArgument`): `caps[]` belongs to endpoint messages, whose +receiver needs its fresh handle words delivered to memory. A reply carries new handle words from MR0. +Labels count from 1 per object kind; the `abi` data half (increment 2) is the one table of record. + +| Invoked (right) | Label: method | MR0 | MR1 | MR2 | MR3 | Reply | +|-----------------|---------------|-----|-----|-----|-----|-------| +| Pool (`WRITE`) | 1 `region_create` | pages | alignment, pages | max extents | rights | Region | +| Pool (`WRITE`) | 2 `as_create` · 3 `process_create` | rights · AddressSpace, presented `WRITE`, bound for life | — | — | — | AddressSpace · Process | +| Pool (`WRITE`) | 4 `thread_create` | Process, presented `WRITE` | its AddressSpace, presented `WRITE` | IPC-buffer address, or 0 for none | — | Thread | +| Pool (`WRITE`) | 5 `sc_create` · 6 `endpoint_create` · 7 `notification_create` · 8 `reply_create` | — · `min_budget` in ns (RFC-0006) · — · — | — | — | — | the new object | +| AddressSpace (`WRITE`) | 1 `map` · 2 `unmap` · 3 `protect` (RFC-0005 §8) | Region, presented · vaddr · vaddr | the space's Pool, presented `WRITE` · — · perms | vaddr | first page (bits 0–31), pages (32–41), perms (48–49) | — | +| Region (`WRITE`) | 1 `region_copy` (RFC-0005, proposed here) | source Region, presented `READ` | source page | destination page | pages, at most 16 | — | +| Process (`WRITE` / `READ`) | 1 `GRANT` · 2 `KILL` · 3 `INFO` (§8) | handle words, **moved**, count in `tag.length` | | | | `GRANT`: words in the child; `INFO`: counts | +| Thread (RFC-0006 §4) | 1 `write_regs` · 2 `read_regs`, a four-word frame group in label bits 8–15 · 3 `resume` · 4 `suspend` · 5 `bind_sc` · 6 `unbind_sc` · 7 `set_priority` · 8 `set_mcp` · 9 `bind_notification` · 10 `unbind_notification` · 11 `set_fault_ep` · 12 `set_timeout_ep` · 13 `set_fp` | the group's words · SchedContext presented `WRITE` · `auth` presented, no right · Notification presented `READ` · endpoint **moved** | priority or flag | | | `read_regs`: the group | +| SchedControl (RFC-0006 §5) | 1 `sc_configure` (`WRITE`) · 2 `ctl_info` (`READ`) | SchedContext, presented `WRITE` | budget, ns | period, ns | refills (bits 0–7), class (8–15, zero) | `ctl_info`: counts | +| SchedContext (`READ`) · Console (`WRITE`) | 1 `sc_stats` · 1 `WRITE` (§11) | — · byte count, at most 24 | · bytes | · bytes | · bytes | counters · bytes accepted | + +`IrqControl` and `IrqHandler` (RFC-0006 (proposed) §11) follow the same rule, with labels assigned when +they are built. Endpoints and notifications have no methods: they are what `send`, `call` and `recv` act on. ## 5. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities (0–3); bit 9 set by the -kernel when a bound notification woke a `recv` (MR0 then holds the bits); bits 10–15 reserved, zero; bits +kernel, on a returned tag only, when a bound notification woke a `recv` (MR0 then holds the bits); bits 16–63 a 48-bit label — user-defined for endpoints, the method for kernel objects, the method result on a -kernel reply. *(seL4's `seL4_MessageInfo`.)* **Four physical message registers on both architectures:** -seL4 uses four on both, research/0001 found the in-register payoff small and shrinking (10% on ARM11, -negative on modern x86), and a per-architecture count would make a message fast on one Tier-1 target only. -Up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds the -non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a 2-bit -field with no invalid encodings. **Rejected:** six physical registers on AArch64; seL4's 120 words. +kernel reply. **Bits 9–15 must be zero in every tag userspace supplies**, or `InvalidArgument`, so no +sender can forge the notification flag. *(seL4's `seL4_MessageInfo`.)* **Four physical message registers +on both architectures:** seL4 uses four on both, research/0001 found the in-register payoff small and +shrinking (10% on ARM11, negative on modern x86), and a per-architecture count would make a message fast on +one Tier-1 target only. Up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: +512 bytes bounds the non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three +capabilities**, a 2-bit field with no invalid encodings. **Rejected:** six physical registers on AArch64; +seL4's 120 words. ```rust /// 1 KiB and 1 KiB-aligned, so it never straddles a page. @@ -148,74 +193,93 @@ pub struct IpcBuffer { } ``` -**The buffer is named by capability and reached by frame.** It is a 1 KiB-aligned slice of one page of a -Region, bound at thread creation by the Region capability (`READ | WRITE`), the offset and the user address -the spawner mapped it at; RFC-0006 (proposed) records the binding, RFC-0005 (proposed) provides the Region -and the physmap through which alone the kernel reads and writes it. The user address serves word 0 and -the discovery register and is never dereferenced. Received capabilities take fresh slots (RFC-0004 §6). +**The buffer is optional and reached by frame.** `thread_create` takes the buffer's user address, or 0 +for none. The kernel checks it against the AddressSpace's mapping list (RFC-0005 (proposed) §7): 1 KiB +aligned and inside a read-write mapping, or `InvalidArgument`. It then binds that page of the backing +Region — a counted reference and a writable use for W^X — and reaches it only through the physmap. The +address serves word 0 and the discovery register and is never dereferenced. A syscall needing the buffer +(a length over four, or any capability) from a thread with none, or whose binding was severed, fails +`NoBuffer`. **Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's address into `TPIDRRO_EL0` (readable at EL0, writable only at EL1) and, on x86_64, into the user GS base, with -`CR4.FSGSBASE` off (RFC-0006 (proposed)) and `abi` reading the self-pointer at `gs:[0]` — the ELF TLS -convention of a block whose first word is its own address *(Drepper, "ELF Handling For Thread-Local -Storage")*. RFC-0006 (proposed) restores both on every switch, so `_start` has its buffer at its first -instruction with no TLS runtime; the kernel trusts neither, and a thread reloading GS misleads only -itself. **Rejected:** `TPIDR_EL0` and `FS_BASE`, the ELF TLS thread pointers a relibc port needs; and -seL4's current answer, a thread-local (`__sel4_ipc_buffer`, libsel4 `functions.h`), which needs TLS first. - -**Single fetch (O-5).** Once SMP exists another thread can rewrite the buffer mid-syscall, so every buffer -word the kernel acts on is read **exactly once** into kernel memory before any check depends on it. -Lengths and counts come from the tag register, bounded by 64 and 3; `caps[]` is copied into a kernel array -and every word resolved and checked for `TRANSFER` before any moves (RFC-0004 §6); payload goes frame to -frame through RFC-0005 (proposed)'s volatile-copy module, never interpreted, and no Rust reference is -formed into the buffer. *(research/0002 Part 5, virtio's double-fetch lesson.)* +`CR4.FSGSBASE` off and `abi` reading the self-pointer at `gs:[0]` — the ELF TLS convention of a block whose +first word is its own address *(Drepper, "ELF Handling For Thread-Local Storage")*. RFC-0006 (proposed) +restores both on every switch, so a thread has its buffer at its first instruction with no TLS runtime. +**Rejected:** `TPIDR_EL0` and `FS_BASE`, the ELF TLS thread pointers a relibc port needs; seL4's +thread-local `__sel4_ipc_buffer` (libsel4 `functions.h`), which needs TLS first. + +**Transfer and single fetch (O-5).** Another thread can rewrite a buffer mid-syscall, so every buffer word +the kernel acts on is read **exactly once**, into kernel memory, before any check depends on it; no Rust +reference is formed into it, and payload moves through RFC-0005 (proposed)'s volatile-copy module, never +interpreted *(research/0002 Part 5, virtio's double fetch)*. A blocked sender keeps its tag and MR0–MR3 in +its frame, which is kernel memory. **Everything else is fetched at rendezvous, in one kernel entry** +*(seL4)*: the kernel reads the sender's `caps[]` once, resolves every word and checks `TRANSFER`, counts +the receiver's free slots exactly, and only then moves; any failure moves nothing, completes the sender +with its status and leaves the receiver waiting. A sibling that rewrote the sender's buffer while it was +blocked changed only what the sender itself sends. `TableFull` is therefore decided at rendezvous, by the +crate's new `free_slots()`, since a retired slot is neither free nor live. **Rejected:** removing the +capabilities at entry into the sender's TCB — three capabilities of in-flight state per thread, and a +refused delivery could not return them to their original slots. ## 6. Handles in one register RFC-0003's `Handle` — a `u32` index and a 64-bit generation — fits no register. The ABI word is **`slot_generation << 20 | index`**: 2^20 slots, Linux's default `fs.nr_open` (1,048,576); 2^44 slot -generations, about two hundred days of one close-and-reopen per microsecond on one slot. As -`Generation::FIRST` is 1, word `0` is never live: it is null — the width RFC-0003's amendment asked for. -**Only the slot generation crosses the ABI;** object generations stay in kernel memory and keep 64 bits. -The crate gains `Handle::to_word`/`from_word` and `MAX_SLOT_GENERATION` (2^44 − 1); `remove` retires a -slot whose next generation would exceed it, through its existing `Retired` state; a const assertion bounds -`N` by 2^20; `Generation`'s doc comment gains the slot bound. Phase 1 tables have `N` = 64, failing closed. -**Rejected:** two registers per handle (halves the budget); 32/32 (seventy minutes at the same rate, and -the LIFO free list concentrates reuse on one slot); a 16-bit index (too few for server workloads). +generations, about two hundred days of one close-and-reopen per microsecond on one slot. **Word `0` is +never live**, because `Generation::FIRST` is 1: it is the null handle. **The width is the invariant +RFC-0003's amendment asked to be stated**: only the slot generation crosses the ABI, bounded at 2^44 − 1, +and a slot whose next generation would pass the bound retires through the crate's existing `Retired` state +— fail-closed, never wrapped. Object generations stay in kernel memory at 64 bits. Phase 1 tables have `N` += 64. The rights word is RFC-0003's bit order — `DUPLICATE` 0, `TRANSFER` 1, `READ` 2, `WRITE` 3, `REVOKE` +4, and RFC-0005 (proposed)'s `EXECUTE` 5 — pinned in `abi` by a `const` assertion against the crate. +**Rejected:** two registers per handle (halves the budget); 32/32 (seventy minutes at the same rate, and the +LIFO free list concentrates reuse on one slot); a 16-bit index (too few for server workloads). + +**Crate changes, as one dated RFC-0003 amendment** logged in `docs/CHANGELOG.md`, so the width lives in +RFC-0003 and this section cites it: `Handle::to_word`/`from_word` and `MAX_SLOT_GENERATION`, with a +`const` assertion bounding `N` by 2^20; `free_slots()` (§5); and, with destruction (increment 14), a split +of `StaleGeneration` so a destroyed object reports `ObjectDestroyed` rather than the holder's own +use-after-close. ## 7. The error model **Transport status** (`x0`/`rax`) is what the syscall layer found, identical for a kernel object or a -server. Every failure is atomic, and the detail register names the culprit: 0 the invoked handle, 1–3 an -attached capability, 4 the tag, 5 the reply handle. - -| Status | Meaning | -|--------|---------| -| `Ok` | done | -| `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total and needs no fault-handler policy | -| `InvalidArgument` | a reserved bit, a length over 64, a non-zero `svc` immediate, an undefined rights bit | -| `InvalidHandle` | word 0, index out of range, empty slot, stale slot generation — the handle names nothing *of yours* (the crate's `OutOfBounds`, `Empty`, `StaleGeneration`) | -| `Revoked` | the slot is live but its object was destroyed — the crate's new `ObjectDestroyed`; close the handle | -| `WrongType` / `InsufficientRights` | live handle, wrong kind of object; or lacking the right the operation needs | -| `TableFull` | a moved capability found no slot in the receiving table; nothing moved | -| `WouldBlock` | `nb_send` or `nb_recv` found no partner | -| `PeerGone` | the object or party waited on was destroyed mid-wait | -| `BudgetRefused` | a `call` whose donated context can never meet the endpoint's `min_budget`, refused up front (RFC-0004's amendment; seL4 RFC-14) | -| `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object (RFC-0004's amendment) | - -`Revoked` needs the second crate change: `StaleGeneration` today covers both the holder's own -use-after-close and a destroyed object; `resolve` checks them in that order, so the split is one variant -and one changed return, and a holder observes destruction without its own bug being reported as one. When -the budget statuses arise is RFC-0006 (proposed)'s to define; its draft calls the first -`BudgetBelowThreshold`, and the name here is proposed because the encoding is this RFC's. +server. Every failure is atomic. The detail register names the culprit: 0 the invoked handle, 1–4 a handle +word in MR0–MR3, 5–7 `caps[0..2]`, 8 the tag, 9 the reply handle, 10 the IPC buffer. + +| # | Status | Meaning | +|---|--------|---------| +| 0 | `Ok` | done | +| 1 | `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total and needs no fault-handler policy | +| 2 | `InvalidArgument` | a reserved bit, a length over 64, a non-zero `svc` immediate, an undefined rights bit, `tag.caps` on a method, a bad buffer address | +| 3 | `InvalidHandle` | word 0, index out of range, empty slot, stale slot generation: the handle names nothing *of yours* | +| 4 | `Revoked` | the slot is live but its object was destroyed (from increment 14; before it, nothing is destroyed) | +| 5 | `WrongType` | live handle, wrong kind of object | +| 6 | `InsufficientRights` | lacking the right the operation needs | +| 7 | `TableFull` | a moved capability found no free slot in the receiving table; nothing moved | +| 8 | `WouldBlock` | `nb_send` or `nb_recv` found no partner | +| 9 | `PeerGone` | the object or party waited on went away, or a `reply` found its object unbound | +| 10 | `BudgetRefused` | a `call` whose context can never meet the endpoint's `min_budget`, or a `send` — which donates nothing — to an endpoint that has one | +| 11 | `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object | +| 12 | `NoBuffer` | the syscall needs an IPC buffer the thread does not have | + +`BudgetRefused` follows RFC-0006 (proposed) §7, which refuses only a context that can *never* pass and +lets one short *now* wait — its amendment A2 to RFC-0004's, which says "refused up front". The status +stands under either reading; its meaning narrows if A2 is accepted. **Method result** (the reply label) is what the object said, in one vocabulary for kernel and server, so a -stand-in reports errors alike *(seL4's reply label)*: the status numbers where meanings coincide, plus -`InvalidMethod`, RFC-0005 (proposed)'s `PoolExhausted` and `Fragmented`, and §8's `WindowClosed`. +stand-in reports errors alike *(seL4's reply label)*: the numbers above where meanings coincide, then 13 +`InvalidMethod`, 14 `PoolExhausted` and 15 `Fragmented` (RFC-0005, proposed), 16 `WindowClosed` (§8). **Faults** — EL0 exceptions that are not syscalls — are not return values: the thread is suspended and a -fault message goes to a handler RFC-0006 (proposed) binds, so a supervisor can restart a driver (O-17). -**Interim:** the reporter prints it, prefixed by the thread; the thread stops and the core continues. +fault message goes to its `fault_ep` (RFC-0006 (proposed) §4), so a supervisor can restart a driver +(O-17). **Interim, until increment 11:** the kernel prints one line — the thread, `ESR`, `ELR`, `FAR` — and +leaves the thread `Inactive`; the core continues. **Its cost:** it is an **ambient output channel**. A +thread holding no Console can put two words of its own choosing (`ELR`, `FAR`) on the operator's line by +faulting on purpose, once per thread it can create, each line holding the kernel for about 100 bytes of +UART time. It is compiled only under a `report-user-faults` feature that `xtask` enables for Phase-1 +boot-tests and `run-qemu`, and it qualifies the O-4 and O-9 rows (§14). ## 8. The Process object and spawning (O-27) @@ -223,102 +287,135 @@ A **Process** holds one RFC-0003 table, one AddressSpace (RFC-0005 (proposed)) a (proposed)) — a real object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. First authority: - **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority across creation. -- **B — one bootstrap handle** *(Zircon: `zx_process_start` moves one handle; a loader sends the rest)*. +- **B — one bootstrap handle** *(Zircon: `zx_process_start` passes one handle; a loader sends the rest)*. Over an unbuffered rendezvous the parent must serve its own child a protocol before the child can act. - **C — explicit grants before start** *(seL4's root task writing a child's CSpace)*. **Verdict sought.** -A Process is created through RFC-0005 (proposed)'s Pool, naming an AddressSpace, with an **empty table**: +`process_create` names an AddressSpace, which it **binds for life**: one bound already is refused +(`InvalidArgument`), so two tables never share one memory (O-8). The Process is born with an **empty +table** and no thread. | Method | Right | Effect | |--------|-------|--------| -| `GRANT` | `WRITE` on the process; `TRANSFER` on each attached capability | move up to three attached capabilities into the child's table; the reply's MR0–MR2 carry their handle words *in the child* | +| `GRANT` | `WRITE`; `TRANSFER` on each moved capability | before the first `resume`: move up to three capabilities into the child, all or nothing; the reply carries their handle words *in the child* | | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | - -`GRANT` is a `call` whose capabilities come from `caps[]` as an endpoint send's do (§5): fetched once, -all resolved before any moves. **The grant window.** `GRANT` succeeds only until the first `resume` of any -thread bound to the process — a one-way event RFC-0006 (proposed) reports by setting a flag on the -Process; afterwards it answers `WindowClosed`, and authority arrives only by IPC, which the process -consents to by receiving. No parent reaches into a running child's table; a stolen Process capability -injects nothing into a live one. The window is **single-use** in research/0002's sense (Part 3, from -Vault's response-wrapping) but lacks its **prior-use detection**: the child is not told who granted what. -Here the channel is a Process capability minted to its creator alone; detection across imperfectly -trusted spawn channels is the broker RFC's. - -The spawner passes the child's handle words in `a0` and `a1` (the demo's console and endpoint), or points -`a0` at a start block — a runtime convention, not kernel ABI. No right joins RFC-0003's set: `GRANT`, -`KILL` and thread creation use `WRITE` on that one Process. A process dies with its last thread. +| `INFO` | `READ` | live capabilities in the table; threads not `Exited`; whether the window is open | + +**The grant window.** `GRANT` succeeds only until the first `resume` of any thread of the process — a +one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers +`WindowClosed`, and authority arrives only by IPC, which the process consents to by receiving. The window +is **single-use** in research/0002's sense (Part 3, from Vault's response-wrapping) but lacks its +**prior-use detection**: the child is not told who granted what. Detection across imperfectly trusted spawn +channels is the broker RFC's. + +**What the window closes, stated exactly.** It closes **capability injection**: after the first `resume` +no holder of the Process capability can add to its table, and `WRITE` on a started Process means `KILL` +alone — so a supervisor can hold kill-and-inspect authority without holding the child's power (O-17). It +does **not** close **control**. `WRITE` on the child's AddressSpace maps into it and, with `WRITE` on the +Process, creates threads in it; `WRITE` on a Thread rewrites that thread's registers (RFC-0006 §4). That is +authority over the child's code and data, which its loader always had. So thread creation presents the +AddressSpace as well as the Process: a Process capability alone cannot start code in a child. + +**The spawn protocol**, which increment 9's loader plan encodes and host-tests: the spawner never maps the +child's writable memory into its own space — data pages arrive by `region_copy` (§10); before `resume` it +grants the child whichever of its own AddressSpace, Thread and Region capabilities the child needs, closes +the rest, and keeps the Process capability alone. A spawner that derived copies first keeps control; the +kernel cannot tell, and the child cannot yet check (Open question 3; §16). + +**Lifetime.** A Process ends by `KILL`, or when no capability names it and none of its threads can run +again. One whose threads have all `Exited` is dead but intact: `INFO` reports it, and its table and space +remain for a supervisor to inspect until `KILL` or its last capability closes *(seL4: objects live until +destroyed)*. **Rejected:** ending with its last thread *(Zircon)* — undefined for a newborn with no thread, +and it destroys what a supervisor would inspect. + +The spawner passes the child's handle words in `a0` and `a1` (the demo's Console and endpoint), or `a0` +names a start Region — a runtime convention, not kernel ABI; the root alone receives BootInfo (§9). ## 9. The root task and boot info The one act the kernel performs unasked: at boot it builds **exactly one** process from the first boot module and gives it every boot capability *(seL4's root task; research/0002 Part 3, "the bootstrap seam is -a one-time kernel act")* — an AddressSpace; the descriptor's segments copied into fresh frames and mapped -as stated (§10); a stack; an IPC buffer; a read-only **BootInfo** page; a Process and a Thread in §3's -entry state, `a0` = BootInfo's address, `a1` = 0; `eret`. Its scheduling context (RFC-0006 (proposed)) -takes a named boot constant, as seL4's root starts at maximum priority with one configured timeslice. - -**BootInfo**, in `user/abi`, is a header (magic, version, length) and one `{ kind, flags, handle, base, -size }` entry per initial capability, with a string table for module names, pinned within a page by a host -test. Every initial capability is *listed*, none at a well-known slot, so the order can change without an -ABI break and a reviewer can enumerate the whole grant. Kinds here: the root's Process, the Console (§11), -a read-only Region per further module with its descriptor; RFC-0005 (proposed) adds the AddressSpace, the -root Pool and the DTB, RFC-0006 (proposed) the Thread, `SchedControl` and interrupt capabilities. +a one-time kernel act")* — an AddressSpace; the descriptor's segments mapped as stated (§10); a stack; a +read-only **BootInfo** page; a Process and a Thread in §3's entry state, `a0` = BootInfo's address, `a1` = +0; `eret`. Its priority, ceiling and full context are RFC-0006 (proposed) §6's one bootstrap act. + +**BootInfo**, in `user/abi`, is a header (magic, version, length) and one entry per initial capability, +with a string table for module names, pinned within a page by a host test: + +```rust +#[repr(C)] +pub struct BootCap { kind: u32, flags: u32, handle: u64, base: u64, size: u64 } // 32 bytes +``` + +Every initial capability is *listed*, none at a well-known slot, so the order can change without an ABI +break and a reviewer can enumerate the whole grant. Kinds here: the root's Process, the Console (§11), a +read-only Region per further module with its descriptor; RFC-0005 (proposed) adds the AddressSpace, the one +root Pool and the DTB; RFC-0006 (proposed) the Thread, one `SchedControl` per core and the `IrqControl`. +A kind that is not built yet is simply absent. **The root is not a superuser.** Its capabilities are ordinary, droppable and generation-checked, and the -rule is checkable: **no kernel code branches on "is this the root task"**. The bootstrap graph is acyclic, -and nothing the root needs waits on a userspace service. **Rejected:** a kernel boot manifest (policy in -the kernel); a `spawn_from_module` syscall (a loader as a kernel service); a `system:masters` superuser. +rule is checkable: **no kernel code branches on "is this the root task"**. They are **not revocable** until +RFC-0003a: research/0002 Part 3 asks for revocable init capabilities, and today only destruction revokes. +The bootstrap graph is acyclic; nothing the root needs waits on a userspace service. **Rejected:** a kernel +boot manifest (policy in the kernel); a `spawn_from_module` syscall (a loader as a kernel service); a +`system:masters` superuser. ## 10. Getting the first images into memory -QEMU's `-kernel` loads one ELF. The candidates for the rest: - -- **Embed them in the kernel ELF** *(verdict sought)*. One artefact boots; its hash pins every program. -- **QEMU's `-initrd` or `-device loader`.** Two artefacts, QEMU-only, and neither reaches a bare ELF. In - `hw/arm/boot.c` (read at QEMU 8.2.2), `arm_setup_direct_kernel_boot` loads `-initrd` only for images it - treats as Linux, never an ELF; for a non-Linux image `do_cpu_reset` sets the PC and nothing else, so no - register names the DTB either, which goes at the base of RAM only if the image leaves room — ours does - not, hence RFC-0005 (proposed)'s relink to `0x4020_0000`. A loader blob's address is the command line's. -- **Link the children into the root's ELF** *(seL4 practice, a CPIO archive)*. The root becomes specific - to each image, and the children stop being separate immutable objects it can hand on. - -**`xtask` parses ELF; the kernel does not.** A dependency-free, host-tested ELF64 reader in `xtask` emits -a fixed-shape **load descriptor** per program: entry point; a text segment (`R+X`), an optional read-only -segment (`R`), a data segment (`R+W`, with a `bss` size), page-aligned in the user half; stack, IPC-buffer -and BootInfo addresses. It **refuses** at build time any writable-and-executable, overlapping or unaligned -segment, dynamic linking, TLS or other program header — W^X checked before the image exists (O-13). The -kernel checks one fixed structure against fixed bounds and never sees an ELF: Nexen's finding that kernel -code validating structures built elsewhere is the bug farm (research/0002 Part 7), applied to our own boot -path. **Rejected:** an in-kernel `elf/` crate — small and testable, but a parser in the TCB for no gain. - -**The bundle.** `xtask` builds the user crates, writes `payload.bin` — a versioned `abi` header, then per -module a name, descriptor and segment bytes — and builds the kernel with `SETONIX_PAYLOAD` naming it; -`kernel/build.rs` places it page-aligned in a read-only `.payload` section (`KEEP`). **Unset, it emits an -empty bundle**, so a bare `cargo build` or `cargo clippy --package setonix-kernel` still links, boots and -greets, printing `boot modules: 0`. *(Hubris's `xtask dist`; seL4's elfloader, parsing before the -kernel runs.)* The kernel reaches modules through `arch::boot_modules()` — the bundle on AArch64, files -the UEFI stub loaded on x86_64 — so embedding is Phase 1's *source*, not the design. It loads **entry 0 -only**; the rest reach the root as **read-only Regions** (RFC-0005 (proposed) provides the frames) with -descriptors in BootInfo, so the root parses nothing and the images stay immutable. *(Genode's core hands -boot modules to init read-only.)* - -**The programs** — `user/demo/server`, `user/demo/client` — are soft-float `no_std` crates on `abi` alone, -with no `unsafe`, linked static by `user/link/.ld` in the user half; `x86_64-unknown-none` defaults -to PIE and the *kernel* code model, so `xtask` passes `-C relocation-model=static -C code-model=small`. +**Verdict sought — the mechanism:** the kernel reaches modules through `arch::boot_modules()`, which yields +fixed-shape **load descriptors** produced on the host; **the kernel never parses ELF**. A dependency-free, +host-tested ELF64 reader in `xtask` emits per program: entry point; a text segment (`R+X`), an optional +read-only segment (`R`), a data segment (`R+W`, zero-padded to a page, with a `bss` size), page-aligned in +the user half; stack and BootInfo addresses. It **refuses** a writable-and-executable, overlapping or +unaligned `PT_LOAD`, `PT_DYNAMIC`, `PT_INTERP`, `PT_TLS`, `PT_GNU_RELRO` and an executable `PT_GNU_STACK`; +it ignores `PT_NOTE` and a non-executable `PT_GNU_STACK`. `user/link/.ld` declares `PHDRS`, so lld +emits nothing else — W^X checked before the image exists (O-13). The kernel checks one fixed structure +against fixed bounds: Nexen's finding that kernel code validating structures built elsewhere is the bug farm +(research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` crate — small and +testable, but a parser in the TCB for no gain. + +**The source — AArch64 interim: embedded in the kernel image.** QEMU's `-kernel` loads one ELF. Its +`-initrd` and `-device loader` give a second, QEMU-only artefact: in `hw/arm/boot.c`, read at QEMU 8.2.2 +and to be re-read at the pinned 11.0.3 before acceptance, `-initrd` is loaded only for images treated as +Linux, and no register names it. Linking children into the root's ELF *(seL4 practice, a CPIO archive)* +would make the root specific to each image. So `xtask` builds the user crates, writes `payload.bin` — a +versioned `abi` header, then per module a name, descriptor and segment bytes — and passes its path in +`SETONIX_PAYLOAD`. `kernel/build.rs` re-runs on that variable and exports the path, writing an empty bundle +to `OUT_DIR` when it is unset, so a bare `cargo build` still links, boots and prints `boot modules: 0`. A +`global_asm!` `.incbin` under `kernel/src/arch/aarch64/` embeds it; `aarch64.ld` places it in a +page-aligned, read-only, `KEEP`ed `.payload` section. *(Hubris's `xtask dist`; seL4's elfloader, parsing +before the kernel runs.)* **The interim's cost:** a user change relinks the kernel, whose reproducibility +now includes the programs'. **On x86_64**, a future x86_64 boot RFC provides the UEFI stub that loads the +same bundle as a file; increment 12 depends on it. + +**Loading.** The kernel loads **module 0 only**; the rest reach the root as **read-only Regions** (RFC-0005 +(proposed) §7 mints them `READ | EXECUTE | DUPLICATE | TRANSFER`) with descriptors in BootInfo, so the root +parses nothing and the images stay immutable *(Genode's core hands boot modules to init read-only)*. Any +loader, kernel or root, maps text `RX` and read-only data `R` **in place** from the module Region; copies +data pages into a fresh Region with `region_copy`, which RFC-0005 (proposed) provides as a page-granular, +frame-to-frame copy through its volatile-copy module; and takes `bss` as fresh zeroed pages. No loader +ever maps a child's writable memory into itself. + +**The programs** — `user/demo/server` (the root, for now), `user/demo/client` — are soft-float `no_std` +crates on `abi` alone, with no `unsafe`; `x86_64-unknown-none` defaults to PIE and the *kernel* code model, +so `xtask` passes `-C relocation-model=static -C code-model=small`. ## 11. The console as a capability Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, by -`call`: the byte count in MR0, the bytes from MR1 on — up to 504, so a short line never leaves the -registers. The root receives it with `DUPLICATE | TRANSFER | WRITE` (no `READ`: there is no input path) -and derives `WRITE`-only copies for the children it chooses. A program not handed one cannot print. -**Rejected:** seL4's `DebugPutChar`, which any thread may issue in a `CONFIG_PRINTING` build — ambient -authority, however small — and a console at a well-known handle. **Adopted in spirit:** Zircon's debuglog, -write-only by default. Because invocation is uniform, once a userspace UART driver exists the root hands -out an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** (increment 13). -**Interim, named:** until then a device path in the kernel is reachable from EL0 — tiny, write-only, -exercised by every boot-test — and at 115,200 baud a 504-byte write holds the non-preemptible kernel about -44 ms on real hardware, a bound RFC-0006's latency budget carries. +`call`: the byte count in MR0, at most 24 bytes in MR1–MR3, sent verbatim (the `abi` writer supplies +`\r\n`). **It never waits.** The kernel stores bytes while the PL011's transmit-full flag is clear and +returns the count accepted in MR0; `abi` yields and retries the rest, so any waiting happens at EL0, where it +is preemptible. The kernel's hold is at most 24 flag reads and 24 stores, counted in RFC-0006 (proposed)'s +worst path, so the console sets no floor under `MIN_BUDGET` and needs no preemption point. The root +receives it with `DUPLICATE | TRANSFER | WRITE` (no `READ`: there is no input path) and derives +`WRITE`-only copies for the children it chooses. A program not handed one cannot print, except through +§7's fault-report interim. **Rejected:** seL4's `DebugPutChar`, which any thread may issue in a +`CONFIG_PRINTING` build — ambient authority, however small — and a console at a well-known handle. +**Adopted in spirit:** Zircon's debuglog, write-only by default. Once a userspace UART driver exists the +root hands out an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** +(increment 13). **Interim, named:** until then a device path in the kernel is reachable from EL0 — tiny, +write-only, bounded, exercised by every boot-test. ## 12. `user/abi` — the kernel's own binding @@ -327,30 +424,38 @@ allocation, formatting, start-up, POSIX-shaped calls — and issues no syscalls. `redox_syscall`, versioned with the Redox kernel; seL4 keeps libsel4 in the kernel's repository, generated from the kernel's own interface specifications. The binding is the user half of §3's table, with nothing to port it from. **Verdict sought: the microkernel-core row, write ourselves.** A relibc port adds a layer -*above* it; anything here that would read the same on another kernel moves out. The maintainer may append -"syscall ABI binding (`user/abi`)" to that row — constitution text, and his; if he reads it as the runtime -row instead, it is write-ourselves-by-necessity there, and that row's verdict needs a recorded exception. - -It holds the numbers, `Handle`, `Tag`, `Status`, the ABI layouts with `const` assertions, and safe -wrappers — no allocator, no formatting. **One source of truth:** the kernel (with the `syscalls` feature -gating `src/arch/**` off) and `xtask` compile the same data half as userspace. - -**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is proposed as -`user/abi/src/arch/**`: the `svc`/`syscall` assembly; `_start`, which turns `a0` into the root's -`&'static BootInfo` and calls `main`; and accessors resting on a kernel promise about this process's -memory — the IPC-buffer view, and the slice over a Region just mapped into the caller (the root's loader -writes child segments through it). User programs contain no `unsafe`. **The `CLAUDE.md` edit** adds to -§ `unsafe` policy "`user/abi/src/arch/**` — trap instructions, the process entry stub and the kernel-shared -memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no external deps"; the -`grep -rn "allow(unsafe_code)"` audit widens from `kernel/src/` to the repository. +*above* it; anything here that would read the same on another kernel moves out. Appending "syscall ABI +binding (`user/abi`)" to that row is constitution text and the maintainer's (Constitution §4, logged in +`docs/CHANGELOG.md`); increments 2 and 7 wait on that verdict. + +It holds the numbers, `Handle`, `Tag`, `Status`, method labels, `BootCap`, the ABI layouts with `const` +assertions, and safe wrappers — no allocator, no formatting. **One source of truth:** the kernel and +`xtask` build `abi` without its `syscalls` feature, the feature that compiles `src/arch/**`, and share the +data half with userspace. + +**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is `user/abi/src/arch/**`, +holding only: the `svc`/`syscall` wrappers; the entry stubs — `root_start` copies the BootInfo page with +volatile reads onto its stack and passes `main` a reference to the copy, sound because the kernel mapped the +page read-only before this process's first instruction and nothing else has yet run; `child_start` passes +`a0` and `a1` — and the discovery-register read. User programs contain no `unsafe`, and no Rust reference +is ever formed into memory another thread may write. **Separately sought, not demo:** volatile word +accessors for the IPC buffer, under `user/abi/src/buffer/**`, whose invariant — the buffer stays mapped at +its bound address while the thread runs — is upheld by the process itself; if it is broken, the thread +faults, and no other process is touched. + +**The edits it implies, all in increment 7:** `CLAUDE.md` § `unsafe` policy gains +"`user/abi/src/arch/**` — trap instructions and process entry stubs", and § Layout gains `user/`; threat +model O-6 names the three trees; the workspace `Cargo.toml` comment ("the two trees") and +`kernel/src/main.rs`'s grep widen from `kernel/src/` to the repository; `xtask/Cargo.toml`'s "no +dependencies" becomes "no external dependencies". ## 13. Lineage | Source | What is taken | What is left | |--------|---------------|--------------| -| **seL4** | the root task and boot info; kernel objects invoked as IPC, errors in the reply label; four physical message registers; `ReplyRecv`; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | -| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one start handle as the only channel; negative status integers | -| **Linux** | `svc #0` + `x8`; `syscall` + `r10`; the canonical-`rcx` check; `nr_open` as a sizing reference | everything the registers carry | +| **seL4** | the root task and boot info; kernel objects invoked as IPC, errors in the reply label; presented capabilities (`extraCaps`); four physical message registers; `ReplyRecv`; reusable reply objects; transfer at rendezvous; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | +| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one start handle as the only channel; a process ending with its last thread | +| **Linux** | `svc #0` + `x8`; `syscall` + `r10`; the `sysretq` guards; `nr_open` as a sizing reference | everything the registers carry | | **x86_64 ELF TLS** | a block whose first word holds its own address | `FS` itself, left to TLS | | **Hubris** | the build tool assembles one bootable image | a static task set with no spawn | | **Redox** | `redox_syscall` versioned with the kernel, relibc above it | ambient path syscalls (research/0001) | @@ -361,89 +466,132 @@ memory the ABI defines", adds `user/` to § Layout and makes `xtask` "no externa | Obligation | This RFC | Status after implementation | |-----------|----------|-----------------------------| | O-1 unforgeability | handles reach the kernel only as words resolved through the caller's table; word 0 and out-of-range indices fail closed (§6) | **binds at B1** — RFC-0003's mechanism made live | -| O-2 non-widenability | `cap_derive` is the only duplication and is subset-only; `GRANT` and transfer move, never widen (§4, §8) | **binds at B1** | -| O-4 no ambient authority | twelve syscalls, each handle-indexed or authority-free; console by capability (§4, §11) | **discharged at the ABI**; each new kernel method checked at review | -| O-5 argument validation | no pointer arguments; reserved bits zero; single fetch; caps all-or-nothing; kernel-built `SPSR_EL1`; canonical `rcx` (§3, §5, §7) | **discharged for the syscall surface**, with RFC-0005 for user memory | -| O-6 kernel memory safety | one new designated `unsafe` tree, named and bounded (§12) | **stays Built**, with a wider grep | -| O-7 no unprivileged exhaustion | 512-byte copy bound; creation paid by a named Pool; fixed tables fail closed; the console's hold bounded (§5, §6, §11) | **contributes** — RFC-0005 owns the accounting | -| O-9 no ambient side channel | result registers carry results or zero; no grant after start; no ambient console (§3, §8, §11) | **contributes** | -| O-13 W^X | the build refuses writable-and-executable segments; the kernel maps per descriptor (§10) | **contributes** — RFC-0005 owns enforcement | -| O-17 contained drivers | fault messages to a supervisor (§7) | **designed, deferred** — the interim stops the thread | -| O-27 explicit authority at spawn | empty table at birth; grants only by explicit move; window shut at first resume; root's grant enumerated; the demo proves a denial (§8, §9, §18) | **discharged** | +| O-2 non-widenability | `cap_derive` is the only duplication and is subset-only; presented handles move nothing; moves never widen (§4, §8) | **binds at B1** | +| O-4 no ambient authority | twelve syscalls, each handle-indexed or authority-free; methods take handles; console by capability (§4, §11) | **discharged at the ABI, except the fault-report interim** (§7) until increment 11; each new method checked at review | +| O-5 argument validation | no pointer arguments; reserved tag bits zero; single fetch at rendezvous; transfers all-or-nothing; kernel-built `SPSR_EL1`; `sysretq` guards (§3, §5, §7) | **discharged for the syscall surface**, with RFC-0005 §11 for user memory | +| O-6 kernel memory safety | a third designated tree, trap instructions and entry stubs only; threat-model O-6 names it (§12) | **stays Built**, with a repository-wide grep | +| O-7 no unprivileged exhaustion | 512-byte copy bound; creation paid by a named Pool; reusable reply objects; fixed tables fail closed; a console that never waits (§4, §5, §11) | **contributes** — RFC-0005 owns the accounting | +| O-9 no ambient side channel | result registers carry results or zero; no grant after start; no ambient console (§3, §8, §11) | **contributes**, except the fault-report interim (§7) | +| O-13 W^X | the build refuses writable-and-executable segments; loaders map per descriptor (§10) | **contributes** — RFC-0005 owns enforcement | +| O-17 contained drivers | fault messages to `fault_ep`; `KILL`-only authority over a started Process; dead processes inspectable (§7, §8) | **designed, deferred** — the interim stops the thread | +| O-27 explicit authority at spawn | empty table at birth; grants only by explicit move before start; root's grant enumerated; a host test and a boot-test count the child's table (§8, §9, §18) | **discharged** in its letter; the spawner's retained control is named (§8), not claimed away | ## 15. Graves checked (§3) - **Policy in the kernel.** The kernel runs one image and lists what it granted; the rest is the root's. - The manifest and `spawn_from_module` were this grave's doors; the root's boot constant is its residue. -- **The catch-all right.** No root identity survives `eret`; no right is added; `WRITE` on a Process is - authority over one object, and the root's many capabilities are droppable and enumerable. + The boot manifest and `spawn_from_module` were this grave's doors, and stay shut. +- **The catch-all right.** No root identity survives `eret`; no right is added. A Process capability is a + proxy for its table only through `GRANT`, and only before start; afterwards `WRITE` on it is `KILL`. + Control of a running child lives in separate AddressSpace and Thread capabilities, each droppable. - **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; one copy. -- **Bolted-on multicore.** Entry is per-CPU from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`); single - fetch is written for a second core; no global lock. Two cores on one table stays RFC-0003 §14.2's. +- **Bolted-on multicore.** Entry is per-core from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`). + RFC-0006 (proposed) §12 provides the big kernel lock taken on entry, answering RFC-0003 §14.2; single + fetch stays necessary under it, because other threads write the buffer from EL0 without the lock. - **Drivers pulled in; unused device paths.** The Console is the only device method reachable from EL0: - write-only, used by every boot-test, its deletion scheduled. + write-only, bounded, used by every boot-test, its deletion scheduled. ## 16. Costs — what this makes harder - **`call` is overloaded** — per-type method decoding and two error layers, the price of interposition. +- **Interposition is partial:** a forwarder for a method with handle-word arguments must first be handed + those capabilities, which costs the client a `TRANSFER` it may not hold — input to RFC-0003a. - **Four physical registers and 64 words:** past 32 bytes a message touches memory, and a 65–120-word message seL4 would copy needs a Region here; raising either later breaks binaries. -- **Handles are 20/44 for good,** the accepted `capability` crate changes twice, and `TPIDRRO_EL0` and - the user GS base belong to the ABI rather than a runtime. -- **The root task is the most sensitive process,** as in seL4; it must stay small. **No ambient - printing; late grants need a willing child;** the grant window detects no prior use. -- **Embedding couples builds:** a user change relinks the kernel, whose reproducibility now includes the - programs'; `xtask` gains its first, in-workspace dependency. **No PAN on the demo CPU.** +- **Handles are 20/44 for good,** the accepted `capability` crate changes by amendment, and `TPIDRRO_EL0` + and the user GS base belong to the ABI rather than a runtime. +- **The spawner keeps control if it wants to:** the grant window stops injection, not a loader that kept + derived copies of the child's AddressSpace or Threads; the child cannot yet check (Open question 3). +- **The root task is the most sensitive process,** as in seL4, and its capabilities are not revocable + until RFC-0003a; it must stay small. **No ambient printing; late grants need a willing child.** +- **The Console moves at most 24 bytes per entry** and returns short counts, so every writer loops. +- **No PAN on the demo CPU;** `user/abi` is a third `unsafe` tree to audit. ## 17. Open questions -1. **Fault messages and death** — their layout (this RFC's), the fault binding (RFC-0006's), and how a - supervisor learns a process died (O-17); likely a notification bound at creation. -2. **Badges** — `cap_derive`'s badge word and the badge register are reserved zero until RFC-0003a. -3. **The console after Phase 2** — whether any Console stays mintable once a driver owns the UART. -4. **The `unsafe` tree's name** — two of its items are memory, not architecture: `src/kernel_mem/`? +1. **Fault messages and death** — their layout (this RFC's), how a supervisor learns a process died, and + how it supervises threads a child creates itself (RFC-0006 §4); likely a notification bound at creation. +2. **Badges** — `cap_derive`'s badge word is reserved zero until RFC-0003a; notification signals then carry it. +3. **A sovereignty check** — should RFC-0005's AddressSpace query report its live-capability count, as + RFC-0006's `ctl_info` does for `SchedControl`, so a child can confirm it holds the only one? +4. **The Language clause** — do single-instruction `svc`/`syscall` wrappers in userspace fall within + "minimal assembly … for boot and context switching", as RFC-0005 §17.7 asks for `TLBI`? 5. **The 64-word budget** — a verdict now, to be revisited with the first measurements. ## 18. Implementation increments Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect -prefixed strings, since bare `Kaya!` matches the kernel's greeting. The demo's interims, costed above: -faults print and stop (§7); the in-kernel Console (§11); modules in the kernel image (§10); RFC-0005's -static object pools; the root's boot scheduling constant (§9). The root doubling as server is composition. - -1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, the slot bound, `ObjectDestroyed`. - Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot retires at the bound while an object - generation passes it; use-after-close and destruction told apart. -2. **[demo] `user/abi`, data half** — numbers, `Tag`, `Status`, the layouts and their assertions, no - `unsafe`. Host tests refuse every reserved-bit and over-length encoding; CI builds both targets. -3. **[demo] AArch64 trap path, MMU off** — lower-EL save, dispatch, restore, `eret`; `yield`, - `thread_exit`, `InvalidSyscall`; a lower-EL fault stops the thread, not the kernel. A feature like - `provoke-exception`, `syscall-selftest`, drops to EL0 with the MMU off (the softfloat target is - `+strict-align`, so Device-memory accesses are safe) and runs `svc #0`: `--expect "syscall round trip"`. -4. **[demo] `user/abi/src/arch/**`** — trap stubs, `_start`, the buffer-register read — **with the - `CLAUDE.md` edit in the same PR**. Built for both targets under clippy; first run by increment 7. -5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on W+X, overlapping, truncated and foreign - fixtures; `build.rs`. Boot-tests: bare `--expect "boot modules: 0"`, full `--expect "boot modules: 2"`. -6. **[demo] Objects, `cap_*` and the Console** on RFC-0005's static tables; the self-test prints from EL0 - through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. -7. **[demo] The root reaches EL0** on RFC-0005's spaces and RFC-0006's threads: `--expect "[server] up"`. -8. **[demo] The Process object** — grant window and teardown as pure logic in a host-tested `process/` - crate, then wired in. Host tests: born empty; `GRANT` all-or-nothing; `WindowClosed` after first - resume; `KILL` empties the table. -9. **[demo] The root spawns the client** with a `WRITE`-only Console and the endpoint. The client prints - `[client] up`, invokes a handle word it was never granted, gets `InvalidHandle` and prints - `[client] denied, as expected` — O-27 proved observably, not asserted. -10. **[demo] IPC syscalls** over RFC-0004 endpoints and RFC-0006's blocking states. Boot-test lines in - order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. RFC-0006's increments add the - preemption evidence and the `BudgetRefused` and `BudgetExpired` tests. -11. **Fault messages and death notification** (§17.1): a faulting child is reported to the root, - which prints `[server] child faulted`. +prefixed strings, since bare `Kaya!` matches the kernel's greeting; every demo string fits one 24-byte +`WRITE`. + +1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, `MAX_SLOT_GENERATION`, `free_slots`, + with RFC-0003's dated amendment. Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot + retires at the bound while an object generation passes it; `free_slots` exact beside retired slots. +2. **[demo] `user/abi`, data half**, after the ledger verdict — numbers, `Tag`, `Status`, labels, `BootCap`, + the rights-bit assertion, no `unsafe`. Host tests refuse every reserved-bit (9–15 included) and + over-length encoding; CI builds both targets. +3. **[demo] AArch64 trap path, MMU off** — lower-EL save into one kernel-held static frame (RFC-0006's + increment 4 moves frames into TCBs), dispatch, restore, `eret`; `yield`, `thread_exit`, + `InvalidSyscall`; a lower-EL fault stops the self-test, not the kernel. `--features syscall-selftest + --expect "syscall round trip"`. **Deleted** when RFC-0005's increment 9 turns the MMU on; 6 replaces it. +4. **[demo] `xtask`: repeated `--expect`, matched in order.** Host tests on the matcher. +5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on W+X, overlapping, truncated, `PT_TLS`, + `PT_DYNAMIC` and executable-stack fixtures; `build.rs`, the `.incbin` and `aarch64.ld`'s `.payload`. + Boot-test: bare `--expect "boot modules: 0"`. +6. **[demo] Objects, `cap_*`, the method dispatcher, the Console and the Process's kernel half** (create, + `INFO`) on RFC-0005's store and one root Pool, with its increment 14. An EL0 self-test in a + kernel-built process prints through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. +7. **[demo] The root reaches EL0** — `user/abi/src/arch/**` with every §12 edit in the same PR, after the + maintainer approves the tree; the kernel builds the root from module 0: `--expect "boot modules: 2" + --expect "[root] up"`. +8. **[demo] `GRANT`, the window, `INFO` and `KILL`** as pure logic in a host-tested `process/` crate, then + wired. Host tests: born empty; `GRANT` all-or-nothing on an exact free-slot count; `WindowClosed` after + the first resume; an AddressSpace bound once; `KILL` empties the table; a GRANT-free child's table is + empty. +9. **[demo] The root spawns the client,** in three PRs: + - **a.** The loader plan in a host-tested `no_std` `user/loader` crate: descriptor in, ordered method + calls out; W^X and overlaps refused; host test that the plan ends with the spawner holding the + child's Process capability and nothing naming its memory or threads. + - **b.** The root runs the plan, `WRITE`-only Console and send-only endpoint granted: + `--expect "[root] up" --expect "[client] up"`. + - **c.** O-27 evidence: the client probes indices 0–63 at generation 1 with `cap_rights` — exhaustive + for a table in which no slot was ever vacated — printing `[client] 2 handles`, and the root prints + `INFO`'s count, `[root] client: 2 caps`. Evidence, not proof: the proof is increment 8's host test. +10. **[demo] IPC syscalls** over RFC-0004 endpoints, reusable Reply objects and RFC-0006's blocking states. + Boot-test in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. Host test: calls + from clients that exit mid-call leave the server's Pool balance unchanged. +11. **Fault messages and death notification** (§17.1): a faulting child is reported to the root, which + prints `[root] child faulted`; `report-user-faults` leaves the boot-tests. 12. **x86_64 trap path and `user/abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the - canonical check. Proves §3's second column links; boots with the UEFI stub. -13. **Delete the Console object** when the Phase-2 UART driver lands — scheduled now so it stays interim. + `sysretq` guards — after the x86_64 boot RFC's UEFI stub; the same boot-tests on `q35`. +13. **Delete the Console object** when the Phase-2 UART driver lands. +14. **`Revoked`**: the crate's `ObjectDestroyed` split, with RFC-0005's destruction (its 17). +15. **The root spawns the server** and stops serving: removes the root-as-server interim. + +**Demo order across the three RFCs,** proposed for the maintainer to fix: RFC-0005 1–8, RFC-0006 1–2 and +this RFC's 1, 2 and 4 in any order → 3 → RFC-0006 3–4 → RFC-0005 9–10 (retiring the MMU-off self-tests) → +RFC-0005 11–14 → RFC-0006 5 → 5–6 → 7 → 8 → RFC-0006 6a–6b → 9 → RFC-0006 7a–7b → 10 with RFC-0006 7c. +RFC-0006 places its 3–4 before RFC-0005's MMU and RFC-0005 places them after; this order takes RFC-0006's, +with RFC-0005's 10 repointing the GIC, which RFC-0005 allows. + +**The demo's interims, each with its cost:** + +- **Faults print and stop** (§7): an ambient output channel and ~100 bytes of kernel UART time per fault. +- **The in-kernel Console** (§11): a device path reachable from EL0, bounded to 24 stores per entry. +- **Modules embedded in the AArch64 image** (§10): a user change relinks the kernel. +- **One root Pool and no destruction** (RFC-0005): charges are unattributed, `KILL` releases nothing, and + `Revoked` never arises. +- **Register-only messages** (RFC-0005's IPC-buffer module, its 16, is not demo): threads have no buffer, + longer messages fail `NoBuffer`, and no capability crosses an endpoint — the demo's grants use `GRANT`, + whose handle words travel in registers by design. +- **An active server** (RFC-0006's A1): the `call` donates nothing, so RFC-0004's donation is not exercised. +- **The root doubles as server:** attacker-facing code runs in the process holding every boot capability, + against §16's "it must stay small"; increment 15 removes it. +- **The MMU-off self-test** (increment 3), deleted when the MMU comes on; **RFC-0006's provisional + constants** for the root's priority and timeslice. ## 19. What this unblocks The first userspace, and every Phase-2 paper: the **scheme registry** and **broker** (processes, grants, `call`), the **driver framework** (faults, interrupt notifications, the console moving out), a **relibc -port** over `user/abi` with both TLS registers its own, and the **x86_64 bring-up**. RFC-0003a gains an ABI. +port** over `user/abi` with both TLS registers its own, and the **x86_64 bring-up**. RFC-0003a gains an ABI +and two inputs: partial interposition (§16) and the sovereignty check (§17.3). From ad537edc3b28eb503766f4b81a86d824163b0133 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:45:10 +0000 Subject: [PATCH 16/20] wip(docs): snapshot RFC revisions in progress Backup only; the final revision agents for RFC-0006 and RFC-0007 are still running. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 95458f5..9f3d5a9 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -196,12 +196,12 @@ active server chose to pay for itself. RFC-0004 §4 and §8 lend unconditionally cost:** an active server inherits nothing, and a low-priority one blocks urgent clients; Constitution §3's inheritance holds for passive servers only. -**Inversion, bounded.** Priority-ordered queues serve the most -urgent waiter next, but the request in service runs on its client's context and a middle-priority thread can -preempt it. `prio(S)` is therefore a **ceiling floor**, bounding inversion to one request when set to the -highest client priority *(the priority-ceiling bound seL4 MCS relies on)* — **required, not optional, for a -server shared across trust domains**, passive or active. **Rejected:** boosting a busy server to a waiting -sender's priority (QNX): the kernel would guess which thread serves an endpoint and walk unbounded chains. +**Inversion, bounded.** Priority-ordered queues serve the most urgent waiter next, but the request in service +runs on its client's context and a middle-priority thread can preempt it. `prio(S)` is therefore a **ceiling +floor**, bounding inversion to one request when set to the highest client priority *(the priority-ceiling bound +seL4 MCS relies on)* — **required, not optional, for a server shared across trust domains**, passive or active. +**Rejected:** boosting a busy server to a waiting sender's priority (QNX): the kernel would guess which thread +serves an endpoint and walk unbounded chains. **Caller death never unfunds a server.** If *C* exits or is killed mid-request, *R* is destroyed but the context stays with *S* until *S* next blocks in `recv` or `reply_recv` (its `reply` fails `PeerGone`); it then detaches, From 28c49a97ff2e5879a85cc016d615a72c965e4f0a Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:47:00 +0000 Subject: [PATCH 17/20] docs(rfc-0006): threads and scheduling, reviewed and revised Proposed RFC-0006 after the draft duel, synthesis, three-lens review (62 findings, 2 blockers) and revision. Awaiting the maintainer's verdict; not yet cross-checked against RFC-0005 and RFC-0007. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/rfcs/0006-threads-and-scheduling.md | 11 +- ...0007-system-calls-and-process-bootstrap.md | 618 ++++++++---------- 2 files changed, 287 insertions(+), 342 deletions(-) diff --git a/docs/rfcs/0006-threads-and-scheduling.md b/docs/rfcs/0006-threads-and-scheduling.md index 9f3d5a9..117b05e 100644 --- a/docs/rfcs/0006-threads-and-scheduling.md +++ b/docs/rfcs/0006-threads-and-scheduling.md @@ -445,8 +445,8 @@ counted boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The x86_6 |---|----|--------------------|-------------|---| | 1 | `scheduler` I: run queues, the ten states, MCP, the priority-ordered endpoint queue | exhaustive host transition table against §4's "Left by" column; "a new thread cannot serve as `auth` above 0"; built for both targets | — | D | | 2 | `scheduler` II: full contexts, nanosecond conversion and rounding, billing, stall counters, interim constants | host tests against a shadow model | 1 | D | -| 3 | GICv3 and the virtual timer, kernel-only; `CNTKCTL_EL1`, `PMUSERENR_EL0` zeroed; idle and its abandon-idle stub | xtask gains `gic-version=3`; `timer: tick 3` | 0005#9 (GIC in the MMIO window) | D | -| 4 | Event-kernel dispatch at EL0 with the MMU off: TCB frames and dispatch under RFC-0007#3's trap stub; §8's `SCTLR_EL1` bits | two EL0 self-test threads alternate under the timer: `sched: A B A B`; an EL0 `wfi` returns: `sched: wfi yields` | 3, 0007#3 | D | +| 3 | GICv3 and the virtual timer, kernel-only; `CNTKCTL_EL1`, `PMUSERENR_EL0` zeroed; idle and its abandon-idle stub | xtask gains `gic-version=3`; `timer: tick 3` | — (physical addresses; 0005#9 must repoint the GIC with the console) | D | +| 4 | Event-kernel dispatch at EL0, no address spaces yet: TCB frames and dispatch under RFC-0007#3's trap stub; §8's `SCTLR_EL1` bits | two EL0 self-test threads alternate under the timer: `sched: A B A B`; an EL0 `wfi` returns: `sched: wfi yields` | 3, 0007#3 | D | | 5 | Threads in address spaces; the boot stack moved into the guarded window, `aarch64.ld`'s comment with it | a spinning EL0 thread is preempted and resumes intact: `user: still here` | 4, 0005#10 | D | | 6a | Thread, SchedContext, `SchedControl` as RFC-0005 object-store slots; state machine wired; `(index, generation)` links | host: destroy a lent context, reuse the slot, `reply` — nothing moves | 2, 0005#7 | D | | 6b | Methods behind RFC-0007's dispatcher: `sc_configure`, `bind_sc`, `resume`, `set_priority`, `write_regs`, `thread_exit` | the root runs two EL0 threads at one level: `[a] 1 [b] 1 [a] 2` | 5, 6a, 0007#6 | D | @@ -475,7 +475,6 @@ counted boot-test strings. New `unsafe` stays in `kernel/src/arch/**`. The x86_6 ## 20. What this unblocks -RFC-0004's implementation gets threads to rendezvous between. The driver framework receives interrupts as -notifications (O-17). The broker gains levers to sell time without the kernel knowing why: `SchedControl`, MCP, -`min_budget` and stall counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment (14), and -§13's dated entries. +RFC-0004's implementation gets threads to rendezvous between; drivers get interrupts as notifications (O-17); +the broker gets levers to sell time without the kernel knowing why — `SchedControl`, MCP, `min_budget`, stall +counters. Next on paper: the x86_64 boot RFC (increment 13), an SMP amendment (14), and §13's dated entries. diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index 3709b28..1df5584 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -7,25 +7,25 @@ | Status | **Proposed** — 2026-09-27, awaiting the maintainer's verdict | | Author | Drafted by Claude Code as sparring partner; verdict the maintainer's | | Date | 2026-09-27 | -| Affects | Constitution §3 (capabilities are the only authority; the Language clause), §4 (a Borrow Ledger reading, §12); `CLAUDE.md` § `unsafe` policy and § Layout; threat model O-6's text; RFC-0003 (a dated amendment: the handle word, a slot-generation bound, `free_slots`, one error variant split); RFC-0004 §8 (a dated amendment: reply objects are unbound, not destroyed); `docs/CHANGELOG.md`; the workspace (`user/`), the comments in `Cargo.toml` and `xtask/Cargo.toml`, `kernel/src/main.rs`'s grep scope; `xtask`, `kernel/build.rs` and `aarch64.ld`; the kernel's trap path | +| Affects | Constitution §3 (capabilities are the only authority; the Language clause) and §4 (a Borrow Ledger reading, §12); `CLAUDE.md` § `unsafe` policy and § Layout; threat model O-6's text; RFC-0003 (a dated amendment: the handle word, a slot-generation bound, `free_slots`, one error variant split); RFC-0004 §8 (a dated amendment: reply objects are unbound, not destroyed); `docs/CHANGELOG.md`; the workspace (`user/`), the comments in `Cargo.toml` and `xtask/Cargo.toml`, `kernel/src/main.rs`'s grep scope; `xtask`, `kernel/build.rs` and `aarch64.ld`; the kernel's trap path | | Depends on | RFC-0003 (accepted; §9 binds every syscall); RFC-0004 (accepted; §5, §6, §8, §9.4, §10 and the 2026-08-01 amendment); RFC-0005 and RFC-0006 (proposed, drafted alongside); a future x86_64 boot RFC for the UEFI stub | | Discharges | O-5 (argument validation) for the syscall surface; O-27 (explicit authority at spawn); binds O-1, O-2 and O-4 at B1; keeps O-6 Built with a third designated tree; contributes to O-7, O-9 and O-13; designs O-17's fault path | > **Proposed verdicts.** > > 1. **Uniform invocation:** every kernel-object operation is a `call` on its capability; twelve syscalls in all (§4). -> 2. **Method arguments in registers:** a kernel method's handle words ride in MR0–MR3, *presented* — resolved and checked, never moved — unless its row says *moved* (§4). -> 3. **Reply objects are reusable:** `reply`, budget expiry and caller death unbind them, never destroy them — a dated RFC-0004 amendment (§4). -> 4. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; every thread starts with zeroed registers but four (§3). -> 5. **Messages:** four physical and sixty-four virtual message registers, three capabilities, an optional 1 KiB IPC buffer; transfers resolved at rendezvous, every word fetched once (§5). +> 2. **Method arguments in registers:** handle words in MR0–MR3 are *presented* — resolved, checked, never moved — unless marked *moved* (§4). +> 3. **Reply objects are reusable:** `reply`, budget expiry and caller death unbind them — a dated RFC-0004 amendment (§4). +> 4. **Registers:** AArch64 `svc #0`, number in `x8`; x86_64 `syscall`, number in `rax`; threads start with zeroed registers but four (§3). +> 5. **Messages:** four physical and 64 virtual message registers, three capabilities, an optional 1 KiB IPC buffer, fetched once at rendezvous (§5). > 6. **IPC-buffer discovery:** a kernel-set, user-read-only register — `TPIDRRO_EL0`; the user GS base with a self-pointer at `gs:[0]` (§5). > 7. **Handle word:** 20-bit index, 44-bit slot generation; word `0` is null; recorded as a dated RFC-0003 amendment (§6). > 8. **Errors in two layers:** numbered transport status in a register, method result in the reply label; faults are not errors (§7). -> 9. **Process object:** table + one AddressSpace for life + threads, born empty; `GRANT` only until the first `resume`; once started, `WRITE` on it means `KILL` (§8). +> 9. **Process object:** table, one AddressSpace for life, threads; born empty; `GRANT` only until the first `resume`; then `WRITE` means `KILL` (§8). > 10. **One root task,** built once from a load descriptor, every boot capability listed in BootInfo, no "is root" branch anywhere (§9). -> 11. **Boot images:** `arch::boot_modules()` yields fixed-shape descriptors a host-side ELF reader made; the kernel never parses ELF; AArch64 embeds them as an interim (§10). -> 12. **Console as a capability:** one `WRITE` method of at most 24 bytes that never waits; deleted when the Phase-2 UART driver lands (§11). -> 13. **`user/abi`** is the microkernel-core row; its `src/arch/**` — trap instructions and entry stubs only — is the third designated `unsafe` tree (§12). +> 11. **Boot images:** `arch::boot_modules()` yields descriptors a host-side ELF reader made; the kernel parses no ELF; AArch64 embeds, as an interim (§10). +> 12. **Console as a capability:** one `WRITE` of at most 24 bytes that never waits; deleted when the Phase-2 UART driver lands (§11). +> 13. **`user/abi`** is the microkernel-core row; its `src/arch/**` — trap instructions and entry stubs only — is the third `unsafe` tree (§12). ## 1. The question @@ -46,28 +46,25 @@ traversal), and O-27 exists because of it. Boot images reach userspace read-only ## 3. Entering the kernel -**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` (lower EL, AArch64) with -`ESR_EL1.EC = 0x15`, the immediate in `ISS[15:0]` and `ELR_EL1` past the `svc`; entry masks DAIF and selects -`SP_EL1` *(Arm ARM DDI 0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the -thread's frame (RFC-0006 (proposed) §9 owns its layout, in the TCB) and dispatches on EC: `0x15` to the -syscall dispatcher, anything else is a *fault* (§7). A non-zero immediate answers `InvalidArgument`, so a -later ABI revision is detectable. Return is `eret` with an `SPSR_EL1` the kernel builds: `M` = EL0t, DAIF -clear, NZCV from the frame; `DIT` (Armv8.4) and `SSBS` (Armv8.5) carried where implemented; every other bit -zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. - -**x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`; it loads `RIP` from `IA32_LSTAR`, -leaves the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits, takes `CS`/`SS` from -`IA32_STAR`, and does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. As -RFC-0006 (proposed) §9 requires, the stub runs `swapgs`, parks the user `rsp` in per-core scratch, loads the -TCB frame pointer, saves into the frame, and only then loads the per-core kernel stack; it never -dereferences the user stack. `FMASK` clears IF, DF, TF and AC. **The exit uses `sysretq` only when `rcx` -is canonical and the frame's `RFLAGS` has TF and RF clear and IOPL 0; otherwise `iretq`.** A non-canonical -`rcx` makes Intel's `sysretq` raise #GP *in ring 0* on the user's stack (CVE-2012-0217, which took Xen, -FreeBSD, NetBSD and Windows); `sysretq` loads `RFLAGS` from `r11 AND 3C7FD7H OR 2` (SDM Vol. 2B, SYSRET), -which passes TF, so a user-set TF would trap straight after it — Linux takes `iretq` for TF or RF likewise -(`arch/x86/entry/entry_64.S`). RFC-0005 leaves the top user page unmapped and RFC-0006's `write_regs` -sanitises `RIP` and `RFLAGS`; the exit checks anyway. **Rejected:** `sysenter`, which AMD does not -implement in long mode; an `int` gate, a full IDT delivery per call. +**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` with `ESR_EL1.EC = 0x15`, the +immediate in `ISS[15:0]` and `ELR_EL1` past the `svc`; entry masks DAIF and selects `SP_EL1` *(Arm ARM DDI +0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's frame (RFC-0006 +(proposed) §9 owns the layout) and dispatches on EC: `0x15` to the syscall dispatcher, anything else is a +*fault* (§7). A non-zero immediate answers `InvalidArgument`, so a later ABI revision is detectable. `eret` +uses an `SPSR_EL1` the kernel builds: EL0t, DAIF clear, NZCV from the frame, `DIT` and `SSBS` where +implemented, every other bit zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. + +**x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`, loads `RIP` from `IA32_LSTAR`, leaves +the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits, and does **not** switch +stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. As RFC-0006 (proposed) §9 requires, the +stub runs `swapgs`, parks the user `rsp` in per-core scratch, loads the TCB frame pointer, saves into the +frame, and only then loads the per-core kernel stack. `FMASK` clears IF, DF, TF and AC. **The exit uses +`sysretq` only when `rcx` is canonical and the frame's `RFLAGS` has TF and RF clear and IOPL 0; otherwise +`iretq`.** A non-canonical `rcx` makes Intel's `sysretq` fault *in ring 0* on the user's stack +(CVE-2012-0217), and `sysretq` loads `RFLAGS` as `r11 AND 3C7FD7H OR 2` (SDM Vol. 2B, SYSRET), which +passes TF; Linux takes `iretq` for TF or RF likewise (`arch/x86/entry/entry_64.S`). RFC-0005 leaves the top +user page unmapped and RFC-0006's `write_regs` sanitises `RIP` and `RFLAGS`; the exit checks anyway. +**Rejected:** `sysenter`, absent on AMD in long mode; an `int` gate, a full IDT delivery per call. | Role | AArch64 | x86_64 | |------|---------|--------| @@ -79,38 +76,34 @@ implement in long mode; an `int` gate, a full IDT delivery per call. | error detail (out) | `x7` | `r13` | | clobbered by hardware | — | `rcx`, `r11` | -Every register outside the result set is restored from the frame; the result registers carry a result or -zero, so no kernel value leaves in a register (O-9 hygiene). `x8`/`rax` follow Linux; the x86_64 column -avoids `rbx` and `rbp`, which Rust inline assembly cannot name, and what `syscall` destroys. **No argument -is a pointer** (RFC-0005 (proposed) §11 owns the rule): only the IPC buffer is touched, by frame (§5) — on -the demo CPU the whole defence, since the Cortex-A72 (Armv8.0) lacks PAN (Armv8.1). +Every other register is restored from the frame; result registers carry a result or zero, so no kernel +value leaves in a register (O-9). `x8`/`rax` follow Linux; the x86_64 column avoids `rbx` and `rbp`, which +Rust inline assembly cannot name. **No argument is a pointer** (RFC-0005 (proposed) §11 owns the rule): +only the IPC buffer is touched, by frame (§5) — on the Cortex-A72, which lacks PAN, the whole defence. **Entry state of every thread,** root included: PC = entry; SP as given, 16-byte aligned; `a0`, `a1` (`x0`/`x1`, `rdi`/`rsi`) the two start values; every other general register zero; EL0t with DAIF clear, or ring 3 with `RFLAGS` = `0x202`; FP/SIMD trapped until RFC-0006 enables it; `TPIDR_EL0` and `FS_BASE` zero; the discovery register set if a buffer is bound (§5). The entry symbol is an assembly stub in `abi` that -calls (`bl`; `call`) a Rust `extern "C" fn`, so AAPCS64 and SysV each see their entry alignment. - -**The Language clause.** The constitution allows "the minimal assembly required for boot and context -switching". A trap is the user half of a switch into the kernel, and `svc` or `syscall` has no Rust -equivalent, so the wrappers are single-instruction `asm!` intrinsics — the reading RFC-0005 (proposed) -§17.7 asks the maintainer to confirm for `TLBI`; the same confirmation covers these (Open question 4). +calls a Rust `extern "C" fn` (`bl`; `call`), so AAPCS64 and SysV each see their entry alignment. The trap +wrappers are single-instruction `asm!` intrinsics, a reading of the constitution's "minimal assembly" +clause the maintainer is asked to confirm (Open question 4). ## 4. The syscall set -- **A — one syscall per operation** *(Zircon, Linux)*. Legible, but every object type grows the table, - each entry is a new path to audit, and no server can ever stand in for a kernel object. +- **A — one syscall per operation** *(Zircon, Linux)*. Every object type grows the table, each entry is a + new path to audit, and no server can stand in for a kernel object. - **B — uniform invocation** *(seL4; KeyKOS)*. A handful of IPC syscalls; a kernel-object operation is a `call` on its capability, the method in the tag's label, answered as a server would answer. -- **C — B plus a separate `invoke` syscall for kernel objects.** Keeps B's small table, but callers can - tell a kernel object from a server, which removes B's only advantage. +- **C — B plus a separate `invoke` syscall.** Callers could tell a kernel object from a server, which + removes B's one advantage. -**Verdict sought: B,** for interposition. A Console and a logging server's endpoint are invoked alike, so -Phase 2's UART driver replaces the Console with no client changing. The claim is scoped: **a server can -stand in for a kernel object whose methods take no handle-word arguments** — the Console, and every -protocol a server defines. It cannot stand in for `map` or `bind_sc`, whose second handle word means -something only in the caller's table (below; §16). `cap_rights` therefore reports rights, never the object -kind. A `call` on a kernel object donates no scheduling context. +**Verdict sought: B,** for interposition: a Console and a logging server's endpoint are invoked alike, so +Phase 2's UART driver replaces the Console with no client changing. **The claim is scoped:** a server can +stand in for a kernel object whose methods take no handle-word arguments — the Console, and every protocol +a server defines — but not for `map` or `bind_sc`, whose extra handle word means something only in the +caller's table (§16). So `cap_rights` reports rights, never the object's kind. A kernel method donates no +scheduling context. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| @@ -127,29 +120,26 @@ kind. A `call` on a kernel object donates no scheduling context. | 10 | `cap_rights` | object | out: MR0 the rights bits | | 11 | `thread_exit` | — | the calling thread becomes `Exited`; authority-free | -The `cap_*` calls take and return a zero tag and touch only the caller's own table; the dispatcher builds -`Rights` from the crate's named constants, refusing undefined bits. **A signal sets bit 0** of the -notification's word until badges exist (RFC-0003a), then the capability's badge, fixed at derivation, as -in seL4. No sender chooses the bits, so the signal stays RFC-0004 §8's doorbell with no payload; an -`IrqHandler` ORs the bits bound to it (RFC-0006 (proposed) §11). **Rejected:** a `debug_putc` (§11); -anything taking a PID, path or address; seL4's depth-addressed capability pointers, made for CNode trees -this kernel does not have. - -**Reply objects.** A Reply is created once from a Pool and reused: `recv` binds it to one caller, and -`reply`, budget expiry (RFC-0006 (proposed) §7, `BudgetExpired`) or the caller's death **unbind** it. It -lives until its holder closes it; `reply` on an unbound object fails `PeerGone` *(seL4 MCS reply -objects)*. RFC-0004 §8 says the object "is destroyed if the caller dies"; destruction would let a client -that calls and exits in a loop cost the server one Pool allocation per request — an O-7 drain of the -server's Pool — so this needs a dated RFC-0004 amendment, and RFC-0006's §4 and §7 read "unbinds". +The `cap_*` calls touch only the caller's own table and build `Rights` from the crate's named constants, +refusing undefined bits. **A signal sets bit 0** of a notification until badges exist (RFC-0003a), then the +capability's badge, fixed at derivation *(seL4)*: no sender chooses bits, so the signal stays RFC-0004 §8's +payload-free doorbell; an `IrqHandler` ORs its bound bits (RFC-0006 §11). **Rejected:** a `debug_putc` +(§11); anything taking a PID, path or address; seL4's depth-addressed pointers, made for CNode trees. + +**Reply objects** are created once from a Pool and reused *(seL4 MCS)*: `recv` binds one to a caller; +`reply`, budget expiry (`BudgetExpired`) or the caller's death **unbind** it; it lives until its holder +closes it, and `reply` on an unbound object fails `PeerGone`. RFC-0004 §8 says the object "is destroyed if +the caller dies"; then a client calling and exiting in a loop would cost the server a Pool allocation per +request — an O-7 drain — so RFC-0004 needs a dated amendment and RFC-0006 §4 and §7 read "unbinds". **Kernel-object methods.** A method is a `call` whose label names it and whose MR0–MR3 carry every -argument; a method that needs more is split. A handle word among them is **presented**: resolved in the -caller's table, checked for the right its row names, never moved, never needing `TRANSFER` *(seL4's -`extraCaps` on an invocation)*. A row marked **moved** instead takes the capability out of the caller's -table, which always needs `TRANSFER`, the one right that lets authority leave a table (RFC-0003 §6). -`tag.caps` must be zero on every method (`InvalidArgument`): `caps[]` belongs to endpoint messages, whose -receiver needs its fresh handle words delivered to memory. A reply carries new handle words from MR0. -Labels count from 1 per object kind; the `abi` data half (increment 2) is the one table of record. +argument; one needing more is split. A handle word among them is **presented**: resolved in the caller's +table, checked for its row's right, never moved, never needing `TRANSFER` *(seL4's `extraCaps` on an +invocation)*. A row marked **moved** takes the capability out of the caller's table, which always needs +`TRANSFER`, the one right that lets authority leave a table (RFC-0003 §6). `tag.caps` is zero on every +method (`InvalidArgument`): `caps[]` belongs to endpoint messages, whose receiver needs its new handle +words delivered to memory. Replies carry new handle words from MR0; labels count from 1 per kind, and the +`abi` data half (increment 2) is the table of record. | Invoked (right) | Label: method | MR0 | MR1 | MR2 | MR3 | Reply | |-----------------|---------------|-----|-----|-----|-----|-------| @@ -164,22 +154,21 @@ Labels count from 1 per object kind; the `abi` data half (increment 2) is the on | SchedControl (RFC-0006 §5) | 1 `sc_configure` (`WRITE`) · 2 `ctl_info` (`READ`) | SchedContext, presented `WRITE` | budget, ns | period, ns | refills (bits 0–7), class (8–15, zero) | `ctl_info`: counts | | SchedContext (`READ`) · Console (`WRITE`) | 1 `sc_stats` · 1 `WRITE` (§11) | — · byte count, at most 24 | · bytes | · bytes | · bytes | counters · bytes accepted | -`IrqControl` and `IrqHandler` (RFC-0006 (proposed) §11) follow the same rule, with labels assigned when -they are built. Endpoints and notifications have no methods: they are what `send`, `call` and `recv` act on. +`IrqControl` and `IrqHandler` (RFC-0006 §11) follow the same rule, their labels assigned when built. +Endpoints and notifications have no methods: they are what `send`, `call` and `recv` act on. ## 5. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) The **tag** is one word: bits 0–6 length in words (0–64); bits 7–8 capabilities (0–3); bit 9 set by the kernel, on a returned tag only, when a bound notification woke a `recv` (MR0 then holds the bits); bits -16–63 a 48-bit label — user-defined for endpoints, the method for kernel objects, the method result on a -kernel reply. **Bits 9–15 must be zero in every tag userspace supplies**, or `InvalidArgument`, so no -sender can forge the notification flag. *(seL4's `seL4_MessageInfo`.)* **Four physical message registers -on both architectures:** seL4 uses four on both, research/0001 found the in-register payoff small and -shrinking (10% on ARM11, negative on modern x86), and a per-architecture count would make a message fast on -one Tier-1 target only. Up to **64 words** spill MR4–MR63 to the buffer — below seL4's 120 on purpose: -512 bytes bounds the non-preemptible copy, and RFC-0004 §5 sends bulk through shared regions. **Three -capabilities**, a 2-bit field with no invalid encodings. **Rejected:** six physical registers on AArch64; -seL4's 120 words. +16–63 a 48-bit label — user-defined for endpoints, the method for kernel objects, its result on a kernel +reply *(seL4's `seL4_MessageInfo`)*. **Bits 9–15 must be zero in every tag userspace supplies**, so no +sender can forge the notification flag. **Four physical message registers on both architectures:** seL4 +uses four, research/0001 found the in-register payoff small and shrinking (10% on ARM11, negative on modern +x86), and a per-architecture count would make a message fast on one Tier-1 target only. Up to **64 words** +spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds the non-preemptible copy, and +RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a 2-bit field with no invalid +encodings. **Rejected:** six physical registers on AArch64; seL4's 120 words. ```rust /// 1 KiB and 1 KiB-aligned, so it never straddles a page. @@ -193,54 +182,48 @@ pub struct IpcBuffer { } ``` -**The buffer is optional and reached by frame.** `thread_create` takes the buffer's user address, or 0 -for none. The kernel checks it against the AddressSpace's mapping list (RFC-0005 (proposed) §7): 1 KiB -aligned and inside a read-write mapping, or `InvalidArgument`. It then binds that page of the backing -Region — a counted reference and a writable use for W^X — and reaches it only through the physmap. The -address serves word 0 and the discovery register and is never dereferenced. A syscall needing the buffer -(a length over four, or any capability) from a thread with none, or whose binding was severed, fails -`NoBuffer`. +**The buffer is optional and reached by frame.** `thread_create` takes its user address, or 0 for none. +The kernel checks the address against the AddressSpace's mapping list (RFC-0005 (proposed) §7) — 1 KiB +aligned, inside a read-write mapping — then binds that page of the backing Region, a counted reference and +a writable use for W^X, and reaches it only through the physmap. The address serves word 0 and discovery +and is never dereferenced. A syscall needing a buffer (a length over four, or any capability) from a thread +with none, or whose binding was severed, fails `NoBuffer`. **Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's address into -`TPIDRRO_EL0` (readable at EL0, writable only at EL1) and, on x86_64, into the user GS base, with -`CR4.FSGSBASE` off and `abi` reading the self-pointer at `gs:[0]` — the ELF TLS convention of a block whose -first word is its own address *(Drepper, "ELF Handling For Thread-Local Storage")*. RFC-0006 (proposed) -restores both on every switch, so a thread has its buffer at its first instruction with no TLS runtime. -**Rejected:** `TPIDR_EL0` and `FS_BASE`, the ELF TLS thread pointers a relibc port needs; seL4's -thread-local `__sel4_ipc_buffer` (libsel4 `functions.h`), which needs TLS first. +`TPIDRRO_EL0` (EL0 reads, only EL1 writes) and, on x86_64, into the user GS base with `CR4.FSGSBASE` off, +`abi` reading the self-pointer at `gs:[0]` — the ELF TLS convention of a block whose first word is its own +address *(Drepper, "ELF Handling For Thread-Local Storage")*. RFC-0006 restores both on every switch. +**Rejected:** `TPIDR_EL0` and `FS_BASE`, which a relibc port needs for TLS; seL4's thread-local +`__sel4_ipc_buffer` (libsel4 `functions.h`), which needs TLS first. **Transfer and single fetch (O-5).** Another thread can rewrite a buffer mid-syscall, so every buffer word -the kernel acts on is read **exactly once**, into kernel memory, before any check depends on it; no Rust -reference is formed into it, and payload moves through RFC-0005 (proposed)'s volatile-copy module, never -interpreted *(research/0002 Part 5, virtio's double fetch)*. A blocked sender keeps its tag and MR0–MR3 in -its frame, which is kernel memory. **Everything else is fetched at rendezvous, in one kernel entry** -*(seL4)*: the kernel reads the sender's `caps[]` once, resolves every word and checks `TRANSFER`, counts -the receiver's free slots exactly, and only then moves; any failure moves nothing, completes the sender -with its status and leaves the receiver waiting. A sibling that rewrote the sender's buffer while it was -blocked changed only what the sender itself sends. `TableFull` is therefore decided at rendezvous, by the -crate's new `free_slots()`, since a retired slot is neither free nor live. **Rejected:** removing the -capabilities at entry into the sender's TCB — three capabilities of in-flight state per thread, and a -refused delivery could not return them to their original slots. +the kernel acts on is read **exactly once** into kernel memory before any check depends on it; no Rust +reference is formed into it, and payload moves through RFC-0005's volatile-copy module, never interpreted +*(research/0002 Part 5, virtio's double fetch)*. A blocked sender's tag and MR0–MR3 wait in its frame, in +kernel memory; **everything else is fetched at rendezvous, in one kernel entry** *(seL4)*: `caps[]` read +once, every word resolved and checked for `TRANSFER`, the receiver's free slots counted exactly by the +crate's new `free_slots()` (a retired slot is neither free nor live), and only then any move. A failure +moves nothing, completes the sender with its status, and leaves the receiver waiting. **Rejected:** +removing the capabilities at entry into the sender's TCB — per-thread in-flight state, and a refused +delivery could not return them to their original slots. ## 6. Handles in one register RFC-0003's `Handle` — a `u32` index and a 64-bit generation — fits no register. The ABI word is **`slot_generation << 20 | index`**: 2^20 slots, Linux's default `fs.nr_open` (1,048,576); 2^44 slot generations, about two hundred days of one close-and-reopen per microsecond on one slot. **Word `0` is -never live**, because `Generation::FIRST` is 1: it is the null handle. **The width is the invariant -RFC-0003's amendment asked to be stated**: only the slot generation crosses the ABI, bounded at 2^44 − 1, -and a slot whose next generation would pass the bound retires through the crate's existing `Retired` state -— fail-closed, never wrapped. Object generations stay in kernel memory at 64 bits. Phase 1 tables have `N` -= 64. The rights word is RFC-0003's bit order — `DUPLICATE` 0, `TRANSFER` 1, `READ` 2, `WRITE` 3, `REVOKE` -4, and RFC-0005 (proposed)'s `EXECUTE` 5 — pinned in `abi` by a `const` assertion against the crate. -**Rejected:** two registers per handle (halves the budget); 32/32 (seventy minutes at the same rate, and the -LIFO free list concentrates reuse on one slot); a 16-bit index (too few for server workloads). - -**Crate changes, as one dated RFC-0003 amendment** logged in `docs/CHANGELOG.md`, so the width lives in -RFC-0003 and this section cites it: `Handle::to_word`/`from_word` and `MAX_SLOT_GENERATION`, with a -`const` assertion bounding `N` by 2^20; `free_slots()` (§5); and, with destruction (increment 14), a split -of `StaleGeneration` so a destroyed object reports `ObjectDestroyed` rather than the holder's own -use-after-close. +never live**, because `Generation::FIRST` is 1: it is the null handle. Separately, **the width is the +invariant RFC-0003's amendment asked to be stated:** only the slot generation crosses the ABI, bounded at +2^44 − 1, and a slot whose next generation would pass the bound retires through the crate's `Retired` +state — fail-closed, never wrapped. Object generations stay in kernel memory at 64 bits. The rights word is +RFC-0003's bit order — `DUPLICATE` 0, `TRANSFER` 1, `READ` 2, `WRITE` 3, `REVOKE` 4 — and RFC-0005's +`EXECUTE` 5, pinned in `abi` by a `const` assertion. **Rejected:** two registers per handle (halves the +budget); 32/32 (seventy minutes at the same rate); a 16-bit index (too few for server workloads). + +**The crate changes, as one dated RFC-0003 amendment** logged in `docs/CHANGELOG.md`, so the width lives +in RFC-0003: `Handle::to_word`/`from_word`, `MAX_SLOT_GENERATION` and a `const` bound of `N` by 2^20; +`free_slots()` (§5); and, with destruction (increment 14), a split of `StaleGeneration` so a destroyed +object reports `ObjectDestroyed`, not the holder's own use-after-close. ## 7. The error model @@ -251,10 +234,10 @@ word in MR0–MR3, 5–7 `caps[0..2]`, 8 the tag, 9 the reply handle, 10 the IPC | # | Status | Meaning | |---|--------|---------| | 0 | `Ok` | done | -| 1 | `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total and needs no fault-handler policy | +| 1 | `InvalidSyscall` | no such number — a closed error, not a fault: the surface stays total | | 2 | `InvalidArgument` | a reserved bit, a length over 64, a non-zero `svc` immediate, an undefined rights bit, `tag.caps` on a method, a bad buffer address | | 3 | `InvalidHandle` | word 0, index out of range, empty slot, stale slot generation: the handle names nothing *of yours* | -| 4 | `Revoked` | the slot is live but its object was destroyed (from increment 14; before it, nothing is destroyed) | +| 4 | `Revoked` | the slot is live but its object was destroyed (from increment 14; nothing is destroyed before it) | | 5 | `WrongType` | live handle, wrong kind of object | | 6 | `InsufficientRights` | lacking the right the operation needs | | 7 | `TableFull` | a moved capability found no free slot in the receiving table; nothing moved | @@ -264,190 +247,162 @@ word in MR0–MR3, 5–7 `caps[0..2]`, 8 the tag, 9 the reply handle, 10 the IPC | 11 | `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object | | 12 | `NoBuffer` | the syscall needs an IPC buffer the thread does not have | -`BudgetRefused` follows RFC-0006 (proposed) §7, which refuses only a context that can *never* pass and -lets one short *now* wait — its amendment A2 to RFC-0004's, which says "refused up front". The status -stands under either reading; its meaning narrows if A2 is accepted. - -**Method result** (the reply label) is what the object said, in one vocabulary for kernel and server, so a -stand-in reports errors alike *(seL4's reply label)*: the numbers above where meanings coincide, then 13 -`InvalidMethod`, 14 `PoolExhausted` and 15 `Fragmented` (RFC-0005, proposed), 16 `WindowClosed` (§8). +`BudgetRefused` follows RFC-0006 (proposed) §7, which refuses only a context that can *never* pass and lets +one short *now* wait — its amendment A2 to RFC-0004's, which refuses "up front". The status stands either +way; A2 narrows when it arises. **Method results** (the reply label) share one vocabulary for kernel and +server, so a stand-in reports errors alike *(seL4's reply label)*: the numbers above, then 13 +`InvalidMethod`, 14 `PoolExhausted` and 15 `Fragmented` (RFC-0005), 16 `WindowClosed` (§8). **Faults** — EL0 exceptions that are not syscalls — are not return values: the thread is suspended and a -fault message goes to its `fault_ep` (RFC-0006 (proposed) §4), so a supervisor can restart a driver -(O-17). **Interim, until increment 11:** the kernel prints one line — the thread, `ESR`, `ELR`, `FAR` — and -leaves the thread `Inactive`; the core continues. **Its cost:** it is an **ambient output channel**. A -thread holding no Console can put two words of its own choosing (`ELR`, `FAR`) on the operator's line by -faulting on purpose, once per thread it can create, each line holding the kernel for about 100 bytes of -UART time. It is compiled only under a `report-user-faults` feature that `xtask` enables for Phase-1 -boot-tests and `run-qemu`, and it qualifies the O-4 and O-9 rows (§14). +fault message goes to its `fault_ep` (RFC-0006 §4), so a supervisor can restart a driver (O-17). +**Interim, until increment 11:** the kernel prints one line — thread, `ESR`, `ELR`, `FAR` — and leaves the +thread `Inactive`. **Its cost:** an **ambient output channel**. A thread holding no Console can put two +chosen words (`ELR`, `FAR`) on the operator's line by faulting, once per thread it can create, each line +holding the kernel for about 100 bytes of UART time. It builds only under a `report-user-faults` feature +that `xtask` enables for Phase-1 boot-tests and `run-qemu`, and it qualifies the O-4 and O-9 rows (§14). ## 8. The Process object and spawning (O-27) -A **Process** holds one RFC-0003 table, one AddressSpace (RFC-0005 (proposed)) and its threads (RFC-0006 -(proposed)) — a real object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. First authority: +A **Process** holds one RFC-0003 table, one AddressSpace (RFC-0005) and its threads (RFC-0006) — a real +object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. First authority: - **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority across creation. -- **B — one bootstrap handle** *(Zircon: `zx_process_start` passes one handle; a loader sends the rest)*. - Over an unbuffered rendezvous the parent must serve its own child a protocol before the child can act. +- **B — one bootstrap handle** *(Zircon's `zx_process_start`)*. Over an unbuffered rendezvous the parent + must serve its own child a protocol before the child can act. - **C — explicit grants before start** *(seL4's root task writing a child's CSpace)*. **Verdict sought.** -`process_create` names an AddressSpace, which it **binds for life**: one bound already is refused -(`InvalidArgument`), so two tables never share one memory (O-8). The Process is born with an **empty -table** and no thread. +`process_create` **binds its AddressSpace for life** — one already bound is refused, so two tables never +share one memory (O-8) — and the Process is born with an **empty table** and no thread. | Method | Right | Effect | |--------|-------|--------| -| `GRANT` | `WRITE`; `TRANSFER` on each moved capability | before the first `resume`: move up to three capabilities into the child, all or nothing; the reply carries their handle words *in the child* | +| `GRANT` | `WRITE`; `TRANSFER` on each moved capability | before the first `resume` only: move up to three capabilities in, all or nothing; the reply carries their handle words *in the child* | | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | | `INFO` | `READ` | live capabilities in the table; threads not `Exited`; whether the window is open | -**The grant window.** `GRANT` succeeds only until the first `resume` of any thread of the process — a -one-way event RFC-0006 (proposed) reports by setting a flag on the Process; afterwards it answers -`WindowClosed`, and authority arrives only by IPC, which the process consents to by receiving. The window -is **single-use** in research/0002's sense (Part 3, from Vault's response-wrapping) but lacks its -**prior-use detection**: the child is not told who granted what. Detection across imperfectly trusted spawn -channels is the broker RFC's. - -**What the window closes, stated exactly.** It closes **capability injection**: after the first `resume` -no holder of the Process capability can add to its table, and `WRITE` on a started Process means `KILL` -alone — so a supervisor can hold kill-and-inspect authority without holding the child's power (O-17). It -does **not** close **control**. `WRITE` on the child's AddressSpace maps into it and, with `WRITE` on the -Process, creates threads in it; `WRITE` on a Thread rewrites that thread's registers (RFC-0006 §4). That is -authority over the child's code and data, which its loader always had. So thread creation presents the -AddressSpace as well as the Process: a Process capability alone cannot start code in a child. - -**The spawn protocol**, which increment 9's loader plan encodes and host-tests: the spawner never maps the -child's writable memory into its own space — data pages arrive by `region_copy` (§10); before `resume` it -grants the child whichever of its own AddressSpace, Thread and Region capabilities the child needs, closes -the rest, and keeps the Process capability alone. A spawner that derived copies first keeps control; the -kernel cannot tell, and the child cannot yet check (Open question 3; §16). +**The grant window** closes at the first `resume` of any of its threads — a one-way event RFC-0006 +reports by flagging the Process; `GRANT` then answers `WindowClosed`, and authority arrives only by IPC, +which the process consents to by receiving. It is **single-use** in research/0002's sense (Part 3, from +Vault's response-wrapping) but lacks **prior-use detection**, which is the broker RFC's. + +**What the window closes, exactly.** It closes **capability injection**: once started, no holder of the +Process capability adds to its table, and `WRITE` on it means `KILL` alone, so a supervisor can hold +kill-and-inspect authority without the child's power (O-17). It does **not** close **control**: `WRITE` on +the child's AddressSpace maps into it and, beside `WRITE` on the Process, creates threads in it, and +`WRITE` on a Thread rewrites its registers (RFC-0006 §4) — authority over the child's code and data, which +its loader always had. Thread creation presents the AddressSpace so that a Process capability alone cannot +start code in a child. + +**The spawn protocol**, encoded and host-tested in increment 9's loader plan: the spawner never maps the +child's writable memory into itself — data arrives by `region_copy` (§10); before `resume` it grants the +child whichever of its AddressSpace, Thread and Region capabilities the child needs, closes the rest, and +keeps the Process capability alone. A spawner that derived copies first keeps control; the kernel cannot +tell, and the child cannot yet check (Open question 3). **Lifetime.** A Process ends by `KILL`, or when no capability names it and none of its threads can run -again. One whose threads have all `Exited` is dead but intact: `INFO` reports it, and its table and space -remain for a supervisor to inspect until `KILL` or its last capability closes *(seL4: objects live until -destroyed)*. **Rejected:** ending with its last thread *(Zircon)* — undefined for a newborn with no thread, -and it destroys what a supervisor would inspect. - -The spawner passes the child's handle words in `a0` and `a1` (the demo's Console and endpoint), or `a0` -names a start Region — a runtime convention, not kernel ABI; the root alone receives BootInfo (§9). +again. One whose threads have all `Exited` is dead but intact — `INFO` shows it, and a supervisor can +inspect it until `KILL` or its last capability closes *(seL4: objects live until destroyed)*. **Rejected:** +ending with the last thread *(Zircon)* — undefined for a newborn, and it destroys what a supervisor would +inspect. The spawner passes the child's handle words in `a0` and `a1` (the demo's Console and endpoint), or +names a start Region in `a0` — a runtime convention, not kernel ABI; only the root receives BootInfo (§9). ## 9. The root task and boot info The one act the kernel performs unasked: at boot it builds **exactly one** process from the first boot module and gives it every boot capability *(seL4's root task; research/0002 Part 3, "the bootstrap seam is -a one-time kernel act")* — an AddressSpace; the descriptor's segments mapped as stated (§10); a stack; a -read-only **BootInfo** page; a Process and a Thread in §3's entry state, `a0` = BootInfo's address, `a1` = -0; `eret`. Its priority, ceiling and full context are RFC-0006 (proposed) §6's one bootstrap act. - -**BootInfo**, in `user/abi`, is a header (magic, version, length) and one entry per initial capability, -with a string table for module names, pinned within a page by a host test: - -```rust -#[repr(C)] -pub struct BootCap { kind: u32, flags: u32, handle: u64, base: u64, size: u64 } // 32 bytes -``` - -Every initial capability is *listed*, none at a well-known slot, so the order can change without an ABI -break and a reviewer can enumerate the whole grant. Kinds here: the root's Process, the Console (§11), a -read-only Region per further module with its descriptor; RFC-0005 (proposed) adds the AddressSpace, the one -root Pool and the DTB; RFC-0006 (proposed) the Thread, one `SchedControl` per core and the `IrqControl`. -A kind that is not built yet is simply absent. +a one-time kernel act")* — an AddressSpace, the segments mapped as described (§10), a stack, a read-only +**BootInfo** page, a Process and a Thread in §3's entry state with `a0` = BootInfo's address and `a1` = 0. +Its priority, ceiling and full context are RFC-0006 (proposed) §6's one bootstrap act. + +**BootInfo**, in `user/abi`, is a header (magic, version, length), one `#[repr(C)] BootCap { kind: u32, +flags: u32, handle: u64, base: u64, size: u64 }` per initial capability, and a string table for module +names, pinned within a page by a host test. Every initial capability is *listed*, none at a well-known +slot, so the order can change without an ABI break and a reviewer can enumerate the whole grant. Kinds: the +root's Process, the Console (§11), a read-only Region per further module with its descriptor; RFC-0005 +adds the AddressSpace, the one root Pool and the DTB; RFC-0006 the Thread, a `SchedControl` per core and the +`IrqControl`. A kind not yet built is absent, so BootInfo grows with the increments. **The root is not a superuser.** Its capabilities are ordinary, droppable and generation-checked, and the rule is checkable: **no kernel code branches on "is this the root task"**. They are **not revocable** until -RFC-0003a: research/0002 Part 3 asks for revocable init capabilities, and today only destruction revokes. -The bootstrap graph is acyclic; nothing the root needs waits on a userspace service. **Rejected:** a kernel -boot manifest (policy in the kernel); a `spawn_from_module` syscall (a loader as a kernel service); a -`system:masters` superuser. +RFC-0003a, though research/0002 Part 3 asks for revocable init capabilities: today only destruction revokes. +Nothing the root needs waits on a userspace service. **Rejected:** a kernel boot manifest (policy in the +kernel); a `spawn_from_module` syscall (a loader as a kernel service); a `system:masters` superuser. ## 10. Getting the first images into memory **Verdict sought — the mechanism:** the kernel reaches modules through `arch::boot_modules()`, which yields -fixed-shape **load descriptors** produced on the host; **the kernel never parses ELF**. A dependency-free, -host-tested ELF64 reader in `xtask` emits per program: entry point; a text segment (`R+X`), an optional -read-only segment (`R`), a data segment (`R+W`, zero-padded to a page, with a `bss` size), page-aligned in -the user half; stack and BootInfo addresses. It **refuses** a writable-and-executable, overlapping or -unaligned `PT_LOAD`, `PT_DYNAMIC`, `PT_INTERP`, `PT_TLS`, `PT_GNU_RELRO` and an executable `PT_GNU_STACK`; -it ignores `PT_NOTE` and a non-executable `PT_GNU_STACK`. `user/link/.ld` declares `PHDRS`, so lld -emits nothing else — W^X checked before the image exists (O-13). The kernel checks one fixed structure -against fixed bounds: Nexen's finding that kernel code validating structures built elsewhere is the bug farm -(research/0002 Part 7), applied to our own boot path. **Rejected:** an in-kernel `elf/` crate — small and -testable, but a parser in the TCB for no gain. - -**The source — AArch64 interim: embedded in the kernel image.** QEMU's `-kernel` loads one ELF. Its -`-initrd` and `-device loader` give a second, QEMU-only artefact: in `hw/arm/boot.c`, read at QEMU 8.2.2 -and to be re-read at the pinned 11.0.3 before acceptance, `-initrd` is loaded only for images treated as -Linux, and no register names it. Linking children into the root's ELF *(seL4 practice, a CPIO archive)* -would make the root specific to each image. So `xtask` builds the user crates, writes `payload.bin` — a -versioned `abi` header, then per module a name, descriptor and segment bytes — and passes its path in -`SETONIX_PAYLOAD`. `kernel/build.rs` re-runs on that variable and exports the path, writing an empty bundle -to `OUT_DIR` when it is unset, so a bare `cargo build` still links, boots and prints `boot modules: 0`. A -`global_asm!` `.incbin` under `kernel/src/arch/aarch64/` embeds it; `aarch64.ld` places it in a -page-aligned, read-only, `KEEP`ed `.payload` section. *(Hubris's `xtask dist`; seL4's elfloader, parsing -before the kernel runs.)* **The interim's cost:** a user change relinks the kernel, whose reproducibility -now includes the programs'. **On x86_64**, a future x86_64 boot RFC provides the UEFI stub that loads the -same bundle as a file; increment 12 depends on it. - -**Loading.** The kernel loads **module 0 only**; the rest reach the root as **read-only Regions** (RFC-0005 -(proposed) §7 mints them `READ | EXECUTE | DUPLICATE | TRANSFER`) with descriptors in BootInfo, so the root -parses nothing and the images stay immutable *(Genode's core hands boot modules to init read-only)*. Any -loader, kernel or root, maps text `RX` and read-only data `R` **in place** from the module Region; copies -data pages into a fresh Region with `region_copy`, which RFC-0005 (proposed) provides as a page-granular, -frame-to-frame copy through its volatile-copy module; and takes `bss` as fresh zeroed pages. No loader -ever maps a child's writable memory into itself. - -**The programs** — `user/demo/server` (the root, for now), `user/demo/client` — are soft-float `no_std` -crates on `abi` alone, with no `unsafe`; `x86_64-unknown-none` defaults to PIE and the *kernel* code model, -so `xtask` passes `-C relocation-model=static -C code-model=small`. +fixed-shape **load descriptors** made on the host, and **never parses ELF**. A dependency-free, host-tested +ELF64 reader in `xtask` emits per program an entry point; a text segment (`R+X`), an optional read-only +segment (`R`) and a data segment (`R+W`, zero-padded to a page, with a `bss` size), page-aligned in the user +half; and stack and BootInfo addresses. It **refuses** a writable-and-executable, overlapping or unaligned +`PT_LOAD`, `PT_DYNAMIC`, `PT_INTERP`, `PT_TLS`, `PT_GNU_RELRO` and an executable `PT_GNU_STACK`, ignoring +`PT_NOTE` and a non-executable `PT_GNU_STACK`; `user/link/.ld` declares `PHDRS` so lld emits no other. +W^X is checked before the image exists (O-13), and the kernel validates one fixed structure — Nexen's +finding that kernel code validating structures built elsewhere is the bug farm (research/0002 Part 7). +**Rejected:** an in-kernel `elf/` crate — small and testable, but a parser in the TCB for no gain. + +**The source — an AArch64 interim: embedded in the kernel image.** QEMU's `-kernel` loads one ELF; +`-initrd` and `-device loader` add a second, QEMU-only artefact (and in `hw/arm/boot.c`, read at QEMU 8.2.2 +and to be re-read at the pinned 11.0.3 before acceptance, `-initrd` loads only for Linux images). Linking +children into the root's ELF *(seL4's CPIO archive)* would make the root specific to each image. So `xtask` +writes `payload.bin` — a versioned `abi` header, then per module a name, descriptor and segment bytes — and +names it in `SETONIX_PAYLOAD`; `kernel/build.rs` re-runs on that variable, substituting an empty bundle in +`OUT_DIR` when it is unset, so a bare `cargo build` links and prints `boot modules: 0`. A `global_asm!` +`.incbin` under `kernel/src/arch/aarch64/` embeds it, and `aarch64.ld` places it in a page-aligned, +read-only, `KEEP`ed `.payload` section *(Hubris's `xtask dist`)*. **Its cost:** a user change relinks the +kernel. **On x86_64**, a future x86_64 boot RFC provides the UEFI stub that loads the same bundle as a +file; increment 12 depends on it. + +**Loading.** The kernel loads **module 0 only**; the others reach the root as **read-only Regions** +(RFC-0005 §7 mints them `READ | EXECUTE | DUPLICATE | TRANSFER`) with descriptors in BootInfo *(Genode's +core hands boot modules to init read-only)*. Every loader, kernel or root, maps text `RX` and read-only +data `R` **in place** from the module Region, copies data pages into a fresh Region with `region_copy` — +which RFC-0005 (proposed) provides as a page-granular, frame-to-frame copy through its volatile-copy +module — and takes `bss` as fresh zeroed pages. No loader maps a child's writable memory into itself. The +programs, `user/demo/server` (the root, for now) and `user/demo/client`, are soft-float `no_std` crates on +`abi` alone with no `unsafe`; for `x86_64-unknown-none` `xtask` passes `-C relocation-model=static -C +code-model=small`, since the target defaults to PIE and the kernel code model. ## 11. The console as a capability -Userspace reaches the kernel's UART only through a **Console** kernel object with one method, `WRITE`, by -`call`: the byte count in MR0, at most 24 bytes in MR1–MR3, sent verbatim (the `abi` writer supplies -`\r\n`). **It never waits.** The kernel stores bytes while the PL011's transmit-full flag is clear and -returns the count accepted in MR0; `abi` yields and retries the rest, so any waiting happens at EL0, where it -is preemptible. The kernel's hold is at most 24 flag reads and 24 stores, counted in RFC-0006 (proposed)'s -worst path, so the console sets no floor under `MIN_BUDGET` and needs no preemption point. The root -receives it with `DUPLICATE | TRANSFER | WRITE` (no `READ`: there is no input path) and derives -`WRITE`-only copies for the children it chooses. A program not handed one cannot print, except through -§7's fault-report interim. **Rejected:** seL4's `DebugPutChar`, which any thread may issue in a -`CONFIG_PRINTING` build — ambient authority, however small — and a console at a well-known handle. -**Adopted in spirit:** Zircon's debuglog, write-only by default. Once a userspace UART driver exists the -root hands out an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** -(increment 13). **Interim, named:** until then a device path in the kernel is reachable from EL0 — tiny, -write-only, bounded, exercised by every boot-test. +Userspace reaches the kernel's UART only through a **Console** object with one method, `WRITE`, by `call`: +the byte count in MR0, at most 24 bytes in MR1–MR3, sent verbatim (`abi`'s writer supplies `\r\n`). **It +never waits:** the kernel stores bytes while the PL011's transmit-full flag is clear and returns the count +accepted; `abi` yields and retries the rest, so any waiting happens at EL0, preemptibly. The kernel's hold +is at most 24 flag reads and 24 stores, inside RFC-0006 (proposed)'s measured worst path, so the console +sets no floor under `MIN_BUDGET` and needs no preemption point. The root receives it with `DUPLICATE | +TRANSFER | WRITE` (no `READ`: there is no input path) and derives `WRITE`-only copies for the children it +chooses; a program not handed one cannot print, but for §7's interim. **Rejected:** seL4's `DebugPutChar`, +which any thread may issue in a `CONFIG_PRINTING` build, and a console at a well-known handle. **Adopted in +spirit:** Zircon's debuglog, write-only by default. Once a userspace UART driver exists the root hands out +an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** (increment 13). ## 12. `user/abi` — the kernel's own binding **Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment — -allocation, formatting, start-up, POSIX-shaped calls — and issues no syscalls. relibc sits on -`redox_syscall`, versioned with the Redox kernel; seL4 keeps libsel4 in the kernel's repository, generated -from the kernel's own interface specifications. The binding is the user half of §3's table, with nothing to -port it from. **Verdict sought: the microkernel-core row, write ourselves.** A relibc port adds a layer -*above* it; anything here that would read the same on another kernel moves out. Appending "syscall ABI -binding (`user/abi`)" to that row is constitution text and the maintainer's (Constitution §4, logged in -`docs/CHANGELOG.md`); increments 2 and 7 wait on that verdict. - -It holds the numbers, `Handle`, `Tag`, `Status`, method labels, `BootCap`, the ABI layouts with `const` -assertions, and safe wrappers — no allocator, no formatting. **One source of truth:** the kernel and -`xtask` build `abi` without its `syscalls` feature, the feature that compiles `src/arch/**`, and share the -data half with userspace. - -**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is `user/abi/src/arch/**`, -holding only: the `svc`/`syscall` wrappers; the entry stubs — `root_start` copies the BootInfo page with -volatile reads onto its stack and passes `main` a reference to the copy, sound because the kernel mapped the -page read-only before this process's first instruction and nothing else has yet run; `child_start` passes -`a0` and `a1` — and the discovery-register read. User programs contain no `unsafe`, and no Rust reference -is ever formed into memory another thread may write. **Separately sought, not demo:** volatile word -accessors for the IPC buffer, under `user/abi/src/buffer/**`, whose invariant — the buffer stays mapped at -its bound address while the thread runs — is upheld by the process itself; if it is broken, the thread -faults, and no other process is touched. - -**The edits it implies, all in increment 7:** `CLAUDE.md` § `unsafe` policy gains -"`user/abi/src/arch/**` — trap instructions and process entry stubs", and § Layout gains `user/`; threat -model O-6 names the three trees; the workspace `Cargo.toml` comment ("the two trees") and -`kernel/src/main.rs`'s grep widen from `kernel/src/` to the repository; `xtask/Cargo.toml`'s "no -dependencies" becomes "no external dependencies". +allocation, formatting, start-up — and issues no syscalls: relibc sits on `redox_syscall`, versioned with +the Redox kernel, and seL4 keeps libsel4 in its kernel's repository. The binding is the user half of §3's +table, with nothing to port. **Verdict sought: the microkernel-core row, write ourselves**; anything that +would read the same on another kernel moves out. Appending "syscall ABI binding (`user/abi`)" to that row +is constitution text and the maintainer's (Constitution §4, logged in `docs/CHANGELOG.md`); increments 2 +and 7 wait on it. `abi` holds the numbers, `Handle`, `Tag`, `Status`, labels, `BootCap`, the layouts with +`const` assertions, and safe wrappers — no allocator, no formatting. The kernel and `xtask` build it +without its `syscalls` feature, the one that compiles `src/arch/**`, so all three share one data half. + +**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is `user/abi/src/arch/**` +and holds only: the `svc`/`syscall` wrappers; the discovery-register read; and the entry stubs. +`root_start` copies the BootInfo page by volatile reads onto its stack and hands `main` a reference to the +copy — sound because the kernel mapped that page read-only before the process's first instruction and +nothing else has run; `child_start` passes `a0` and `a1`. User programs contain no `unsafe`, and no Rust +reference is formed into memory another thread may write. **Sought separately, not for the demo:** +volatile word accessors for the IPC buffer in `user/abi/src/buffer/**`, whose invariant — the buffer stays +mapped at its bound address while the thread runs — the process itself upholds; broken, it faults the +thread and touches no other process. + +**The edits, all in increment 7:** `CLAUDE.md` § `unsafe` policy gains "`user/abi/src/arch/**` — trap +instructions and process entry stubs" and § Layout gains `user/`; threat-model O-6 names three trees; the +workspace `Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep widen to the repository; +`xtask/Cargo.toml`'s "No dependencies" becomes "no external dependencies". ## 13. Lineage @@ -480,9 +435,9 @@ dependencies" becomes "no external dependencies". - **Policy in the kernel.** The kernel runs one image and lists what it granted; the rest is the root's. The boot manifest and `spawn_from_module` were this grave's doors, and stay shut. -- **The catch-all right.** No root identity survives `eret`; no right is added. A Process capability is a - proxy for its table only through `GRANT`, and only before start; afterwards `WRITE` on it is `KILL`. - Control of a running child lives in separate AddressSpace and Thread capabilities, each droppable. +- **The catch-all right.** No root identity survives `eret`; no right is added. A Process capability reaches + its table only through `GRANT` before start, then means `KILL`; control of a running child lives in + separate AddressSpace and Thread capabilities, each droppable. - **Baroque hierarchies; multi-copy IPC.** Flat handles, no receive window, a flat BootInfo; one copy. - **Bolted-on multicore.** Entry is per-core from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`). RFC-0006 (proposed) §12 provides the big kernel lock taken on entry, answering RFC-0003 §14.2; single @@ -493,16 +448,16 @@ dependencies" becomes "no external dependencies". ## 16. Costs — what this makes harder - **`call` is overloaded** — per-type method decoding and two error layers, the price of interposition. -- **Interposition is partial:** a forwarder for a method with handle-word arguments must first be handed - those capabilities, which costs the client a `TRANSFER` it may not hold — input to RFC-0003a. + **Interposition is partial:** a forwarder for a method with handle-word arguments must first be handed + those capabilities, costing the client a `TRANSFER` it may not hold — input to RFC-0003a. - **Four physical registers and 64 words:** past 32 bytes a message touches memory, and a 65–120-word message seL4 would copy needs a Region here; raising either later breaks binaries. -- **Handles are 20/44 for good,** the accepted `capability` crate changes by amendment, and `TPIDRRO_EL0` - and the user GS base belong to the ABI rather than a runtime. -- **The spawner keeps control if it wants to:** the grant window stops injection, not a loader that kept - derived copies of the child's AddressSpace or Threads; the child cannot yet check (Open question 3). -- **The root task is the most sensitive process,** as in seL4, and its capabilities are not revocable - until RFC-0003a; it must stay small. **No ambient printing; late grants need a willing child.** +- **Handles are 20/44 for good,** the accepted crate changes by amendment, and `TPIDRRO_EL0` and the user + GS base belong to the ABI rather than a runtime. +- **A spawner keeps control if it wants to:** the window stops injection, not a loader that kept derived + copies of the child's AddressSpace or Threads, and the child cannot yet check. +- **The root task is the most sensitive process,** as in seL4, its capabilities not revocable until + RFC-0003a; it must stay small. **No ambient printing; late grants need a willing child.** - **The Console moves at most 24 bytes per entry** and returns short counts, so every writer loops. - **No PAN on the demo CPU;** `user/abi` is a third `unsafe` tree to audit. @@ -513,81 +468,72 @@ dependencies" becomes "no external dependencies". 2. **Badges** — `cap_derive`'s badge word is reserved zero until RFC-0003a; notification signals then carry it. 3. **A sovereignty check** — should RFC-0005's AddressSpace query report its live-capability count, as RFC-0006's `ctl_info` does for `SchedControl`, so a child can confirm it holds the only one? -4. **The Language clause** — do single-instruction `svc`/`syscall` wrappers in userspace fall within - "minimal assembly … for boot and context switching", as RFC-0005 §17.7 asks for `TLBI`? +4. **The Language clause** — do single-instruction `svc`/`syscall` wrappers and entry stubs in userspace + fall within "minimal assembly … for boot and context switching", as RFC-0005 §17.7 asks for `TLBI`? 5. **The 64-word budget** — a verdict now, to be revisited with the first measurements. ## 18. Implementation increments Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect -prefixed strings, since bare `Kaya!` matches the kernel's greeting; every demo string fits one 24-byte -`WRITE`. +prefixed strings, since bare `Kaya!` matches the kernel's greeting; every demo line fits one `WRITE`. 1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, `MAX_SLOT_GENERATION`, `free_slots`, - with RFC-0003's dated amendment. Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot - retires at the bound while an object generation passes it; `free_slots` exact beside retired slots. + with RFC-0003's amendment. Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot retires at + the bound while an object generation passes it; `free_slots` exact beside retired slots. 2. **[demo] `user/abi`, data half**, after the ledger verdict — numbers, `Tag`, `Status`, labels, `BootCap`, - the rights-bit assertion, no `unsafe`. Host tests refuse every reserved-bit (9–15 included) and - over-length encoding; CI builds both targets. -3. **[demo] AArch64 trap path, MMU off** — lower-EL save into one kernel-held static frame (RFC-0006's - increment 4 moves frames into TCBs), dispatch, restore, `eret`; `yield`, `thread_exit`, - `InvalidSyscall`; a lower-EL fault stops the self-test, not the kernel. `--features syscall-selftest - --expect "syscall round trip"`. **Deleted** when RFC-0005's increment 9 turns the MMU on; 6 replaces it. -4. **[demo] `xtask`: repeated `--expect`, matched in order.** Host tests on the matcher. -5. **[demo] The bundle** — `xtask`'s ELF reader, host-tested on W+X, overlapping, truncated, `PT_TLS`, - `PT_DYNAMIC` and executable-stack fixtures; `build.rs`, the `.incbin` and `aarch64.ld`'s `.payload`. - Boot-test: bare `--expect "boot modules: 0"`. -6. **[demo] Objects, `cap_*`, the method dispatcher, the Console and the Process's kernel half** (create, - `INFO`) on RFC-0005's store and one root Pool, with its increment 14. An EL0 self-test in a - kernel-built process prints through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. -7. **[demo] The root reaches EL0** — `user/abi/src/arch/**` with every §12 edit in the same PR, after the - maintainer approves the tree; the kernel builds the root from module 0: `--expect "boot modules: 2" - --expect "[root] up"`. + the rights assertion, no `unsafe`. Host tests refuse every reserved-bit (9–15 included) and over-length + encoding; CI builds both targets. +3. **[demo] AArch64 trap path, MMU off** — save into one kernel-held frame (RFC-0006's increment 4 moves + frames into TCBs), dispatch, `eret`; `yield`, `thread_exit`, `InvalidSyscall`; an EL0 fault stops the + self-test, not the kernel: `--features syscall-selftest --expect "syscall round trip"`. **Deleted** when + RFC-0005's increment 9 turns the MMU on; 6 replaces it. +4. **[demo] `xtask`: repeated `--expect`, matched in order**, host-tested. +5. **[demo] The bundle** — the ELF reader, host-tested on W+X, overlapping, truncated, `PT_TLS`, + `PT_DYNAMIC` and executable-stack fixtures; `build.rs`, `.incbin`, `.payload`: `--expect "boot modules: 0"`. +6. **[demo] Objects, `cap_*`, the dispatcher, the Console and the Process's kernel half** (create, + `INFO`), on RFC-0005's store and root Pool with its increment 14. An EL0 self-test in a kernel-built + process prints through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. +7. **[demo] The root reaches EL0** — `user/abi/src/arch/**` and every §12 edit in one PR, once the + maintainer approves the tree; module 0 becomes the root: `--expect "boot modules: 2" --expect "[root] up"`. 8. **[demo] `GRANT`, the window, `INFO` and `KILL`** as pure logic in a host-tested `process/` crate, then wired. Host tests: born empty; `GRANT` all-or-nothing on an exact free-slot count; `WindowClosed` after - the first resume; an AddressSpace bound once; `KILL` empties the table; a GRANT-free child's table is - empty. + the first resume; an AddressSpace bound once; `KILL` empties the table. 9. **[demo] The root spawns the client,** in three PRs: - - **a.** The loader plan in a host-tested `no_std` `user/loader` crate: descriptor in, ordered method - calls out; W^X and overlaps refused; host test that the plan ends with the spawner holding the - child's Process capability and nothing naming its memory or threads. - - **b.** The root runs the plan, `WRITE`-only Console and send-only endpoint granted: + - **a.** The loader plan in a host-tested `no_std` `user/loader` crate: descriptor in, method calls + out, W^X and overlaps refused; a host test that the spawner ends holding only the Process capability. + - **b.** The root runs it, granting a `WRITE`-only Console and a send-only endpoint: `--expect "[root] up" --expect "[client] up"`. - **c.** O-27 evidence: the client probes indices 0–63 at generation 1 with `cap_rights` — exhaustive - for a table in which no slot was ever vacated — printing `[client] 2 handles`, and the root prints - `INFO`'s count, `[root] client: 2 caps`. Evidence, not proof: the proof is increment 8's host test. -10. **[demo] IPC syscalls** over RFC-0004 endpoints, reusable Reply objects and RFC-0006's blocking states. - Boot-test in order: `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!`. Host test: calls - from clients that exit mid-call leave the server's Pool balance unchanged. + while no slot has been vacated — printing `[client] 2 handles`; the root prints `INFO`'s count, + `[root] client: 2 caps`. Evidence, not proof: the proof is increment 8's host test. +10. **[demo] IPC syscalls** over RFC-0004 endpoints, reusable Reply objects and RFC-0006's blocking states: + `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!` in order. Host test: calls from clients + that exit mid-call leave the server's Pool balance unchanged. 11. **Fault messages and death notification** (§17.1): a faulting child is reported to the root, which prints `[root] child faulted`; `report-user-faults` leaves the boot-tests. -12. **x86_64 trap path and `user/abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the - `sysretq` guards — after the x86_64 boot RFC's UEFI stub; the same boot-tests on `q35`. +12. **x86_64 trap path and `abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the `sysretq` + guards — after the x86_64 boot RFC's UEFI stub; the demo boot-tests on `q35`. 13. **Delete the Console object** when the Phase-2 UART driver lands. 14. **`Revoked`**: the crate's `ObjectDestroyed` split, with RFC-0005's destruction (its 17). -15. **The root spawns the server** and stops serving: removes the root-as-server interim. +15. **The root spawns the server** and stops serving, removing that interim. -**Demo order across the three RFCs,** proposed for the maintainer to fix: RFC-0005 1–8, RFC-0006 1–2 and -this RFC's 1, 2 and 4 in any order → 3 → RFC-0006 3–4 → RFC-0005 9–10 (retiring the MMU-off self-tests) → +**Demo order across the three RFCs,** for the maintainer to fix: RFC-0005 1–8, RFC-0006 1–2 and this +RFC's 1, 2 and 4 in any order → 3 → RFC-0006 3–4 → RFC-0005 9–10, retiring the MMU-off self-tests → RFC-0005 11–14 → RFC-0006 5 → 5–6 → 7 → 8 → RFC-0006 6a–6b → 9 → RFC-0006 7a–7b → 10 with RFC-0006 7c. -RFC-0006 places its 3–4 before RFC-0005's MMU and RFC-0005 places them after; this order takes RFC-0006's, -with RFC-0005's 10 repointing the GIC, which RFC-0005 allows. +RFC-0006 places its 3–4 before RFC-0005's MMU and RFC-0005 after; this takes RFC-0006's, with RFC-0005's +10 repointing the GIC, as RFC-0005 allows. -**The demo's interims, each with its cost:** +**The demo's interims, with their costs:** - **Faults print and stop** (§7): an ambient output channel and ~100 bytes of kernel UART time per fault. -- **The in-kernel Console** (§11): a device path reachable from EL0, bounded to 24 stores per entry. -- **Modules embedded in the AArch64 image** (§10): a user change relinks the kernel. -- **One root Pool and no destruction** (RFC-0005): charges are unattributed, `KILL` releases nothing, and - `Revoked` never arises. -- **Register-only messages** (RFC-0005's IPC-buffer module, its 16, is not demo): threads have no buffer, - longer messages fail `NoBuffer`, and no capability crosses an endpoint — the demo's grants use `GRANT`, - whose handle words travel in registers by design. -- **An active server** (RFC-0006's A1): the `call` donates nothing, so RFC-0004's donation is not exercised. -- **The root doubles as server:** attacker-facing code runs in the process holding every boot capability, - against §16's "it must stay small"; increment 15 removes it. -- **The MMU-off self-test** (increment 3), deleted when the MMU comes on; **RFC-0006's provisional - constants** for the root's priority and timeslice. +- **The in-kernel Console** (§11), bounded to 24 stores per entry; **modules in the AArch64 image** (§10), + so a user change relinks the kernel. +- **One root Pool, no destruction** (RFC-0005): charges unattributed, `KILL` releases nothing, no `Revoked`. +- **Register-only messages** (RFC-0005's IPC-buffer module, its 16, is not demo): no thread has a buffer + and no capability crosses an endpoint; the demo's grants use `GRANT`, whose words are registers by design. +- **An active server** (RFC-0006's A1), so RFC-0004's donation is not exercised; and **the root doubles as + server**, running client-facing code in the process holding every boot capability, until increment 15. +- **The MMU-off self-test** (increment 3); **RFC-0006's provisional constants** for the root's scheduling. ## 19. What this unblocks From 4cbc926a239b1519c9c0382611da5c046683ba1d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:52:03 +0000 Subject: [PATCH 18/20] docs(rfc-0007): system calls and process bootstrap, reviewed and revised Proposed RFC-0007 after the draft duel, synthesis, three-lens review (48 findings, 3 blockers) and revision. Awaiting the maintainer's verdict; not yet cross-checked against RFC-0005 and RFC-0006. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- ...0007-system-calls-and-process-bootstrap.md | 469 ++++++++---------- 1 file changed, 213 insertions(+), 256 deletions(-) diff --git a/docs/rfcs/0007-system-calls-and-process-bootstrap.md b/docs/rfcs/0007-system-calls-and-process-bootstrap.md index 1df5584..c86f70a 100644 --- a/docs/rfcs/0007-system-calls-and-process-bootstrap.md +++ b/docs/rfcs/0007-system-calls-and-process-bootstrap.md @@ -46,25 +46,25 @@ traversal), and O-27 exists because of it. Boot images reach userspace read-only ## 3. Entering the kernel -**AArch64 — `svc #0`.** From EL0, `svc` traps to `VBAR_EL1 + 0x400` with `ESR_EL1.EC = 0x15`, the -immediate in `ISS[15:0]` and `ELR_EL1` past the `svc`; entry masks DAIF and selects `SP_EL1` *(Arm ARM DDI -0487)*. The stub saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's frame (RFC-0006 -(proposed) §9 owns the layout) and dispatches on EC: `0x15` to the syscall dispatcher, anything else is a -*fault* (§7). A non-zero immediate answers `InvalidArgument`, so a later ABI revision is detectable. `eret` -uses an `SPSR_EL1` the kernel builds: EL0t, DAIF clear, NZCV from the frame, `DIT` and `SSBS` where -implemented, every other bit zero. The Cortex-A72 implements neither, so on the demo CPU only NZCV crosses. +**AArch64 — `svc #0`** traps from EL0 to `VBAR_EL1 + 0x400` with `ESR_EL1.EC = 0x15`, the immediate in +`ISS[15:0]`, `ELR_EL1` past the `svc`, DAIF masked and `SP_EL1` selected *(Arm ARM DDI 0487)*. The stub +saves `x0`–`x30`, `SP_EL0`, `ELR_EL1` and `SPSR_EL1` into the thread's frame (RFC-0006 (proposed) §9 owns +the layout) and dispatches on EC: `0x15` is a syscall, anything else a *fault* (§7). A non-zero immediate +answers `InvalidArgument`, so a later ABI revision is detectable. `eret` uses an `SPSR_EL1` the kernel +builds: EL0t, DAIF clear, NZCV from the frame, `DIT` and `SSBS` where implemented (the Cortex-A72 has +neither), every other bit zero. **x86_64 — `syscall` / `sysretq`.** `syscall` needs `IA32_EFER.SCE`, loads `RIP` from `IA32_LSTAR`, leaves -the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits, and does **not** switch -stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. As RFC-0006 (proposed) §9 requires, the -stub runs `swapgs`, parks the user `rsp` in per-core scratch, loads the TCB frame pointer, saves into the -frame, and only then loads the per-core kernel stack. `FMASK` clears IF, DF, TF and AC. **The exit uses -`sysretq` only when `rcx` is canonical and the frame's `RFLAGS` has TF and RF clear and IOPL 0; otherwise -`iretq`.** A non-canonical `rcx` makes Intel's `sysretq` fault *in ring 0* on the user's stack -(CVE-2012-0217), and `sysretq` loads `RFLAGS` as `r11 AND 3C7FD7H OR 2` (SDM Vol. 2B, SYSRET), which -passes TF; Linux takes `iretq` for TF or RF likewise (`arch/x86/entry/entry_64.S`). RFC-0005 leaves the top -user page unmapped and RFC-0006's `write_regs` sanitises `RIP` and `RFLAGS`; the exit checks anyway. -**Rejected:** `sysenter`, absent on AMD in long mode; an `int` gate, a full IDT delivery per call. +the return `RIP` in `rcx` and `RFLAGS` in `r11`, clears the `IA32_FMASK` bits (here IF, DF, TF and AC) and +does **not** switch stacks *(Intel SDM Vol. 3A, "Fast System Calls in 64-Bit Mode")*. As RFC-0006 §9 +requires, the stub runs `swapgs`, parks the user `rsp` in per-core scratch, loads the TCB frame pointer, +saves into the frame, and only then loads the per-core kernel stack. **The exit uses `sysretq` only when +`rcx` is canonical and the frame's `RFLAGS` has TF and RF clear and IOPL 0, else `iretq`:** a +non-canonical `rcx` makes Intel's `sysretq` fault *in ring 0* on the user's stack (CVE-2012-0217), and +`sysretq` sets `RFLAGS` to `r11 AND 3C7FD7H OR 2` (SDM Vol. 2B, SYSRET), which passes TF — Linux takes +`iretq` for TF or RF likewise (`arch/x86/entry/entry_64.S`). RFC-0005's unmapped top user page and +RFC-0006's sanitising `write_regs` already prevent both; the exit checks anyway. **Rejected:** `sysenter`, +absent on AMD in long mode; an `int` gate, a full IDT delivery per call. | Role | AArch64 | x86_64 | |------|---------|--------| @@ -76,70 +76,61 @@ user page unmapped and RFC-0006's `write_regs` sanitises `RIP` and `RFLAGS`; the | error detail (out) | `x7` | `r13` | | clobbered by hardware | — | `rcx`, `r11` | -Every other register is restored from the frame; result registers carry a result or zero, so no kernel +Other registers are restored from the frame and result registers carry a result or zero, so no kernel value leaves in a register (O-9). `x8`/`rax` follow Linux; the x86_64 column avoids `rbx` and `rbp`, which -Rust inline assembly cannot name. **No argument is a pointer** (RFC-0005 (proposed) §11 owns the rule): -only the IPC buffer is touched, by frame (§5) — on the Cortex-A72, which lacks PAN, the whole defence. +Rust inline assembly cannot name. **No argument is a pointer** (RFC-0005 (proposed) §11 owns the rule): only +the IPC buffer is touched, by frame (§5) — on the Cortex-A72, which lacks PAN, the whole defence. -**Entry state of every thread,** root included: PC = entry; SP as given, 16-byte aligned; `a0`, `a1` -(`x0`/`x1`, `rdi`/`rsi`) the two start values; every other general register zero; EL0t with DAIF clear, or -ring 3 with `RFLAGS` = `0x202`; FP/SIMD trapped until RFC-0006 enables it; `TPIDR_EL0` and `FS_BASE` zero; -the discovery register set if a buffer is bound (§5). The entry symbol is an assembly stub in `abi` that -calls a Rust `extern "C" fn` (`bl`; `call`), so AAPCS64 and SysV each see their entry alignment. The trap -wrappers are single-instruction `asm!` intrinsics, a reading of the constitution's "minimal assembly" -clause the maintainer is asked to confirm (Open question 4). +**Every thread starts** at its entry PC with SP as given (16-byte aligned) and two start values in `a0`, +`a1` (`x0`/`x1`, `rdi`/`rsi`); every other general register zero; EL0t with DAIF clear, or ring 3 with +`RFLAGS` = `0x202`; FP/SIMD trapped until RFC-0006 enables it; `TPIDR_EL0` and `FS_BASE` zero. The entry +symbol is an `abi` assembly stub that calls a Rust `extern "C" fn`, so AAPCS64 and SysV each see their +entry alignment. The trap wrappers are single-instruction `asm!` intrinsics (Open question 4). ## 4. The syscall set -- **A — one syscall per operation** *(Zircon, Linux)*. Every object type grows the table, each entry is a - new path to audit, and no server can stand in for a kernel object. -- **B — uniform invocation** *(seL4; KeyKOS)*. A handful of IPC syscalls; a kernel-object operation is a - `call` on its capability, the method in the tag's label, answered as a server would answer. -- **C — B plus a separate `invoke` syscall.** Callers could tell a kernel object from a server, which - removes B's one advantage. +**Options.** **A — one syscall per operation** *(Zircon, Linux)*: every object type grows the table and no +server can stand in for a kernel object. **B — uniform invocation** *(seL4; KeyKOS)*: a handful of IPC +syscalls, a kernel-object operation being a `call` on its capability with the method in the tag's label, +answered as a server would answer. **C — B plus a separate `invoke`**: callers could tell a kernel object +from a server, which removes B's one advantage. **Verdict sought: B,** for interposition: a Console and a logging server's endpoint are invoked alike, so Phase 2's UART driver replaces the Console with no client changing. **The claim is scoped:** a server can stand in for a kernel object whose methods take no handle-word arguments — the Console, and every protocol a server defines — but not for `map` or `bind_sc`, whose extra handle word means something only in the -caller's table (§16). So `cap_rights` reports rights, never the object's kind. A kernel method donates no -scheduling context. +caller's table (§16). So `cap_rights` reports rights, never the object's kind. | # | Syscall | Handles | Beyond the tag and message registers | |---|---------|---------|--------------------------------------| -| 0 | `yield` | — | authority-free (RFC-0003 §9) | -| 1 | `send` | object | endpoint: blocking send; notification: signal | -| 2 | `nb_send` | object | as `send`, failing `WouldBlock` rather than blocking | -| 3 | `call` | object | endpoint: send and await the reply; kernel object: invoke a method | -| 4 | `recv` | endpoint or notification; reply object | blocking receive; a bound notification may wake it | -| 5 | `nb_recv` | as `recv` | poll | +| 0, 11 | `yield`, `thread_exit` | — | authority-free (RFC-0003 §9); `thread_exit` leaves the caller `Exited` | +| 1, 2 | `send`, `nb_send` | object | endpoint: send, blocking or failing `WouldBlock`; notification: signal | +| 3 | `call` | object | endpoint: send and await the reply; kernel object: invoke a method, donating nothing | +| 4, 5 | `recv`, `nb_recv` | endpoint or notification; reply object | receive, blocking or polling; a bound notification may wake it | | 6 | `reply` | reply object | reply once to the bound caller | | 7 | `reply_recv` | endpoint; reply object | `reply` then `recv` in one entry — the server loop *(seL4 `ReplyRecv`)* | | 8 | `cap_derive` | object, with `DUPLICATE` | in: MR0 the rights wanted, MR1 a badge (reserved, zero); out: MR0 the new handle word | -| 9 | `cap_close` | object | nothing; the handle is dead on return | -| 10 | `cap_rights` | object | out: MR0 the rights bits | -| 11 | `thread_exit` | — | the calling thread becomes `Exited`; authority-free | +| 9, 10 | `cap_close`, `cap_rights` | object | the handle is dead on return; out: MR0 the rights bits | -The `cap_*` calls touch only the caller's own table and build `Rights` from the crate's named constants, +The `cap_*` calls touch only the caller's table and build `Rights` from the crate's named constants, refusing undefined bits. **A signal sets bit 0** of a notification until badges exist (RFC-0003a), then the -capability's badge, fixed at derivation *(seL4)*: no sender chooses bits, so the signal stays RFC-0004 §8's +capability's badge, fixed at derivation *(seL4)*: no sender chooses bits, so it stays RFC-0004 §8's payload-free doorbell; an `IrqHandler` ORs its bound bits (RFC-0006 §11). **Rejected:** a `debug_putc` (§11); anything taking a PID, path or address; seL4's depth-addressed pointers, made for CNode trees. -**Reply objects** are created once from a Pool and reused *(seL4 MCS)*: `recv` binds one to a caller; -`reply`, budget expiry (`BudgetExpired`) or the caller's death **unbind** it; it lives until its holder -closes it, and `reply` on an unbound object fails `PeerGone`. RFC-0004 §8 says the object "is destroyed if -the caller dies"; then a client calling and exiting in a loop would cost the server a Pool allocation per -request — an O-7 drain — so RFC-0004 needs a dated amendment and RFC-0006 §4 and §7 read "unbinds". +**Reply objects** are created once from a Pool and reused *(seL4 MCS)*: `recv` binds one to a caller, and +`reply`, budget expiry or the caller's death **unbinds** it; `reply` on an unbound one fails `PeerGone`. +RFC-0004 §8 says it "is destroyed if the caller dies", which would let a client that calls and exits in a +loop cost the server a Pool allocation per request — an O-7 drain — so RFC-0004 needs a dated amendment and +RFC-0006 §4 and §7 read "unbinds". **Kernel-object methods.** A method is a `call` whose label names it and whose MR0–MR3 carry every -argument; one needing more is split. A handle word among them is **presented**: resolved in the caller's -table, checked for its row's right, never moved, never needing `TRANSFER` *(seL4's `extraCaps` on an -invocation)*. A row marked **moved** takes the capability out of the caller's table, which always needs -`TRANSFER`, the one right that lets authority leave a table (RFC-0003 §6). `tag.caps` is zero on every -method (`InvalidArgument`): `caps[]` belongs to endpoint messages, whose receiver needs its new handle -words delivered to memory. Replies carry new handle words from MR0; labels count from 1 per kind, and the -`abi` data half (increment 2) is the table of record. +argument; one needing more is split. A handle word there is **presented**: resolved in the caller's table, +checked for its row's right, never moved, never needing `TRANSFER` *(seL4's `extraCaps` on an +invocation)*. A handle word marked **moved** leaves the caller's table, which always needs `TRANSFER` +(RFC-0003 §6). `tag.caps` is zero on every method (`InvalidArgument`): `caps[]` belongs to endpoint +messages, whose receiver needs its new handle words delivered to memory. Replies carry new handle words +from MR0; labels count from 1 per kind; the `abi` data half (increment 2) is the table of record. | Invoked (right) | Label: method | MR0 | MR1 | MR2 | MR3 | Reply | |-----------------|---------------|-----|-----|-----|-----|-------| @@ -150,12 +141,13 @@ words delivered to memory. Replies carry new handle words from MR0; labels count | AddressSpace (`WRITE`) | 1 `map` · 2 `unmap` · 3 `protect` (RFC-0005 §8) | Region, presented · vaddr · vaddr | the space's Pool, presented `WRITE` · — · perms | vaddr | first page (bits 0–31), pages (32–41), perms (48–49) | — | | Region (`WRITE`) | 1 `region_copy` (RFC-0005, proposed here) | source Region, presented `READ` | source page | destination page | pages, at most 16 | — | | Process (`WRITE` / `READ`) | 1 `GRANT` · 2 `KILL` · 3 `INFO` (§8) | handle words, **moved**, count in `tag.length` | | | | `GRANT`: words in the child; `INFO`: counts | -| Thread (RFC-0006 §4) | 1 `write_regs` · 2 `read_regs`, a four-word frame group in label bits 8–15 · 3 `resume` · 4 `suspend` · 5 `bind_sc` · 6 `unbind_sc` · 7 `set_priority` · 8 `set_mcp` · 9 `bind_notification` · 10 `unbind_notification` · 11 `set_fault_ep` · 12 `set_timeout_ep` · 13 `set_fp` | the group's words · SchedContext presented `WRITE` · `auth` presented, no right · Notification presented `READ` · endpoint **moved** | priority or flag | | | `read_regs`: the group | +| Thread (`WRITE` / `READ`; RFC-0006 §4) | 1 `write_regs` · 2 `read_regs`, a four-word group of RFC-0006's frame in label bits 8–15, group 0 being PC, SP, `a0`, `a1` | `write_regs`: the group's four words in MR0–MR3 | | | | `read_regs`: the group | +| Thread (`WRITE`; RFC-0006 §4) | 3 `resume` · 4 `suspend` · 5 `bind_sc` · 6 `unbind_sc` · 7 `set_priority` · 8 `set_mcp` · 9 `bind_notification` · 10 `unbind_notification` · 11 `set_fault_ep` · 12 `set_timeout_ep` · 13 `set_fp` | 5: SchedContext, presented `WRITE` · 7, 8: `auth`, presented, no right checked · 9: Notification, presented `READ` · 11, 12: endpoint, **moved** · 13: on or off | 7, 8: the value | — | — | — | | SchedControl (RFC-0006 §5) | 1 `sc_configure` (`WRITE`) · 2 `ctl_info` (`READ`) | SchedContext, presented `WRITE` | budget, ns | period, ns | refills (bits 0–7), class (8–15, zero) | `ctl_info`: counts | | SchedContext (`READ`) · Console (`WRITE`) | 1 `sc_stats` · 1 `WRITE` (§11) | — · byte count, at most 24 | · bytes | · bytes | · bytes | counters · bytes accepted | -`IrqControl` and `IrqHandler` (RFC-0006 §11) follow the same rule, their labels assigned when built. -Endpoints and notifications have no methods: they are what `send`, `call` and `recv` act on. +`IrqControl` and `IrqHandler` (RFC-0006 §11) follow the same rule, labelled when built. Endpoints and +notifications have no methods: they are what `send`, `call` and `recv` act on. ## 5. Messages, the register budget and the IPC buffer (RFC-0004 §9.4) @@ -170,24 +162,13 @@ spill MR4–MR63 to the buffer — below seL4's 120 on purpose: 512 bytes bounds RFC-0004 §5 sends bulk through shared regions. **Three capabilities**, a 2-bit field with no invalid encodings. **Rejected:** six physical registers on AArch64; seL4's 120 words. -```rust -/// 1 KiB and 1 KiB-aligned, so it never straddles a page. -#[repr(C, align(1024))] -pub struct IpcBuffer { - pub self_ptr: u64, // this buffer's user address, written by the kernel at binding - _reserved: [u64; 3], - pub msg: [u64; 64], // MR0..MR63; msg[0..4] never read, those travel in registers - pub caps: [u64; 3], // handle words sent or received, `tag.caps` of them - _tail: [u64; 57], // neither read nor written by the kernel -} -``` - -**The buffer is optional and reached by frame.** `thread_create` takes its user address, or 0 for none. -The kernel checks the address against the AddressSpace's mapping list (RFC-0005 (proposed) §7) — 1 KiB -aligned, inside a read-write mapping — then binds that page of the backing Region, a counted reference and -a writable use for W^X, and reaches it only through the physmap. The address serves word 0 and discovery -and is never dereferenced. A syscall needing a buffer (a length over four, or any capability) from a thread -with none, or whose binding was severed, fails `NoBuffer`. +**The IPC buffer** is 1 KiB and 1 KiB-aligned, so it never straddles a page: `#[repr(C)]` words 0 (its own +user address, written by the kernel), 4–67 (MR0–MR63, the first four never read) and 68–70 (`caps[]`), the +rest untouched. **It is optional and reached by frame.** `thread_create` takes its user address, or 0; the +kernel checks it against the AddressSpace's mapping list (RFC-0005 (proposed) §7) — aligned, inside a +read-write mapping — then binds that page of the backing Region, a counted reference and a writable use for +W^X, and reaches it only through the physmap. The address is never dereferenced. A syscall needing a buffer +(a length over four, or any capability) from a thread with none, or a severed one, fails `NoBuffer`. **Discovery: a kernel-set, user-read-only register.** The kernel loads the buffer's address into `TPIDRRO_EL0` (EL0 reads, only EL1 writes) and, on x86_64, into the user GS base with `CR4.FSGSBASE` off, @@ -199,13 +180,12 @@ address *(Drepper, "ELF Handling For Thread-Local Storage")*. RFC-0006 restores **Transfer and single fetch (O-5).** Another thread can rewrite a buffer mid-syscall, so every buffer word the kernel acts on is read **exactly once** into kernel memory before any check depends on it; no Rust reference is formed into it, and payload moves through RFC-0005's volatile-copy module, never interpreted -*(research/0002 Part 5, virtio's double fetch)*. A blocked sender's tag and MR0–MR3 wait in its frame, in -kernel memory; **everything else is fetched at rendezvous, in one kernel entry** *(seL4)*: `caps[]` read -once, every word resolved and checked for `TRANSFER`, the receiver's free slots counted exactly by the -crate's new `free_slots()` (a retired slot is neither free nor live), and only then any move. A failure -moves nothing, completes the sender with its status, and leaves the receiver waiting. **Rejected:** -removing the capabilities at entry into the sender's TCB — per-thread in-flight state, and a refused -delivery could not return them to their original slots. +*(research/0002 Part 5, virtio's double fetch)*. A blocked sender's tag and MR0–MR3 wait in its frame; +**the rest is fetched at rendezvous, in one kernel entry** *(seL4)*: `caps[]` read once, every word resolved +and checked for `TRANSFER`, the receiver's free slots counted exactly by the crate's new `free_slots()` (a +retired slot is neither free nor live), and only then any move. A failure moves nothing, completes the +sender with its status and leaves the receiver waiting. **Rejected:** removing the capabilities at entry +into the sender's TCB — in-flight state per thread, and a refused delivery could not restore their slots. ## 6. Handles in one register @@ -214,16 +194,16 @@ RFC-0003's `Handle` — a `u32` index and a 64-bit generation — fits no regist generations, about two hundred days of one close-and-reopen per microsecond on one slot. **Word `0` is never live**, because `Generation::FIRST` is 1: it is the null handle. Separately, **the width is the invariant RFC-0003's amendment asked to be stated:** only the slot generation crosses the ABI, bounded at -2^44 − 1, and a slot whose next generation would pass the bound retires through the crate's `Retired` -state — fail-closed, never wrapped. Object generations stay in kernel memory at 64 bits. The rights word is -RFC-0003's bit order — `DUPLICATE` 0, `TRANSFER` 1, `READ` 2, `WRITE` 3, `REVOKE` 4 — and RFC-0005's -`EXECUTE` 5, pinned in `abi` by a `const` assertion. **Rejected:** two registers per handle (halves the -budget); 32/32 (seventy minutes at the same rate); a 16-bit index (too few for server workloads). +2^44 − 1; a slot whose next generation would pass it retires through the crate's `Retired` state, never +wrapping. Object generations stay in kernel memory at 64 bits. Rights travel in RFC-0003's bit order — +`DUPLICATE` 0, `TRANSFER` 1, `READ` 2, `WRITE` 3, `REVOKE` 4 — and RFC-0005's `EXECUTE` 5, pinned in `abi` +by a `const` assertion. **Rejected:** two registers per handle (halves the budget); 32/32 (seventy minutes +at the same rate); a 16-bit index (too few for server workloads). **The crate changes, as one dated RFC-0003 amendment** logged in `docs/CHANGELOG.md`, so the width lives -in RFC-0003: `Handle::to_word`/`from_word`, `MAX_SLOT_GENERATION` and a `const` bound of `N` by 2^20; -`free_slots()` (§5); and, with destruction (increment 14), a split of `StaleGeneration` so a destroyed -object reports `ObjectDestroyed`, not the holder's own use-after-close. +in RFC-0003: `Handle::to_word`/`from_word`, `MAX_SLOT_GENERATION`, a `const` bound of `N` by 2^20 and +`free_slots()` (§5); later, with destruction (increment 14), a split so a destroyed object reports +`ObjectDestroyed` rather than the holder's own use-after-close as `StaleGeneration`. ## 7. The error model @@ -247,32 +227,29 @@ word in MR0–MR3, 5–7 `caps[0..2]`, 8 the tag, 9 the reply handle, 10 the IPC | 11 | `BudgetExpired` | the donated budget expired inside the server; the kernel completed the wait on the reply object | | 12 | `NoBuffer` | the syscall needs an IPC buffer the thread does not have | -`BudgetRefused` follows RFC-0006 (proposed) §7, which refuses only a context that can *never* pass and lets -one short *now* wait — its amendment A2 to RFC-0004's, which refuses "up front". The status stands either -way; A2 narrows when it arises. **Method results** (the reply label) share one vocabulary for kernel and -server, so a stand-in reports errors alike *(seL4's reply label)*: the numbers above, then 13 -`InvalidMethod`, 14 `PoolExhausted` and 15 `Fragmented` (RFC-0005), 16 `WindowClosed` (§8). +`BudgetRefused` follows RFC-0006 (proposed) §7, which refuses only a context that can *never* pass — its +amendment A2 to RFC-0004's "refused up front"; the status stands either way. **Method results** (the reply +label) share one vocabulary for kernel and server, so a stand-in reports errors alike *(seL4's reply +label)*: the numbers above, then 13 `InvalidMethod`, 14 `PoolExhausted`, 15 `Fragmented` (RFC-0005) and 16 +`WindowClosed` (§8). **Faults** — EL0 exceptions that are not syscalls — are not return values: the thread is suspended and a fault message goes to its `fault_ep` (RFC-0006 §4), so a supervisor can restart a driver (O-17). -**Interim, until increment 11:** the kernel prints one line — thread, `ESR`, `ELR`, `FAR` — and leaves the -thread `Inactive`. **Its cost:** an **ambient output channel**. A thread holding no Console can put two -chosen words (`ELR`, `FAR`) on the operator's line by faulting, once per thread it can create, each line -holding the kernel for about 100 bytes of UART time. It builds only under a `report-user-faults` feature -that `xtask` enables for Phase-1 boot-tests and `run-qemu`, and it qualifies the O-4 and O-9 rows (§14). +**Interim, until increment 11:** the kernel prints one line (thread, `ESR`, `ELR`, `FAR`) and leaves the +thread `Inactive`. **Its cost:** an **ambient output channel** — a thread holding no Console can put two +chosen words on the operator's line by faulting, once per thread it can create, each line about 100 bytes +of kernel UART time. It builds only under a `report-user-faults` feature `xtask` enables for Phase-1 +boot-tests and `run-qemu`, and it qualifies the O-4 and O-9 rows (§14). ## 8. The Process object and spawning (O-27) A **Process** holds one RFC-0003 table, one AddressSpace (RFC-0005) and its threads (RFC-0006) — a real -object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. First authority: - -- **A — inherit** *(Unix `fork`/`exec`)*. **Rejected** by O-27 by name: ambient authority across creation. -- **B — one bootstrap handle** *(Zircon's `zx_process_start`)*. Over an unbuffered rendezvous the parent - must serve its own child a protocol before the child can act. -- **C — explicit grants before start** *(seL4's root task writing a child's CSpace)*. **Verdict sought.** - -`process_create` **binds its AddressSpace for life** — one already bound is refused, so two tables never -share one memory (O-8) — and the Process is born with an **empty table** and no thread. +object *(Zircon)*, not seL4's thread naming a CSpace and VSpace. For first authority, **A — inherit** +*(Unix `fork`/`exec`)* is **rejected** by O-27 by name; **B — one bootstrap handle** *(Zircon's +`zx_process_start`)* makes a parent serve its child a protocol over an unbuffered rendezvous before the +child can act; **C — explicit grants before start** *(seL4's root task writing a child's CSpace)* is the +**verdict sought**. `process_create` **binds its AddressSpace for life** — one already bound is refused, so +two tables never share one memory (O-8) — and the Process is born with an **empty table** and no thread. | Method | Right | Effect | |--------|-------|--------| @@ -280,141 +257,128 @@ share one memory (O-8) — and the Process is born with an **empty table** and n | `KILL` | `WRITE` | stop every thread, drop the table, release the address space; the generation bump makes every capability to the process inert | | `INFO` | `READ` | live capabilities in the table; threads not `Exited`; whether the window is open | -**The grant window** closes at the first `resume` of any of its threads — a one-way event RFC-0006 -reports by flagging the Process; `GRANT` then answers `WindowClosed`, and authority arrives only by IPC, -which the process consents to by receiving. It is **single-use** in research/0002's sense (Part 3, from -Vault's response-wrapping) but lacks **prior-use detection**, which is the broker RFC's. - -**What the window closes, exactly.** It closes **capability injection**: once started, no holder of the -Process capability adds to its table, and `WRITE` on it means `KILL` alone, so a supervisor can hold -kill-and-inspect authority without the child's power (O-17). It does **not** close **control**: `WRITE` on -the child's AddressSpace maps into it and, beside `WRITE` on the Process, creates threads in it, and -`WRITE` on a Thread rewrites its registers (RFC-0006 §4) — authority over the child's code and data, which -its loader always had. Thread creation presents the AddressSpace so that a Process capability alone cannot +**The grant window** closes at the first `resume` of any of its threads, which RFC-0006 reports by +flagging the Process; `GRANT` then answers `WindowClosed`, and authority arrives only by IPC, which the +process consents to by receiving. It is **single-use** in research/0002's sense (Part 3, after Vault's +response-wrapping) but lacks **prior-use detection**, which is the broker RFC's. + +**What the window closes, exactly: capability injection.** Once started, no holder of the Process +capability adds to its table, and `WRITE` on it means `KILL` alone, so a supervisor can hold kill-and- +inspect authority without the child's power (O-17). It does **not** close **control**: `WRITE` on the +child's AddressSpace maps into it and, beside `WRITE` on the Process, creates threads in it; `WRITE` on a +Thread rewrites its registers (RFC-0006 §4). That is authority over the child's code and data, which its +loader always had — and why thread creation presents the AddressSpace: a Process capability alone cannot start code in a child. **The spawn protocol**, encoded and host-tested in increment 9's loader plan: the spawner never maps the child's writable memory into itself — data arrives by `region_copy` (§10); before `resume` it grants the child whichever of its AddressSpace, Thread and Region capabilities the child needs, closes the rest, and -keeps the Process capability alone. A spawner that derived copies first keeps control; the kernel cannot +keeps only the Process capability. A spawner that derived copies first keeps control; the kernel cannot tell, and the child cannot yet check (Open question 3). **Lifetime.** A Process ends by `KILL`, or when no capability names it and none of its threads can run -again. One whose threads have all `Exited` is dead but intact — `INFO` shows it, and a supervisor can -inspect it until `KILL` or its last capability closes *(seL4: objects live until destroyed)*. **Rejected:** -ending with the last thread *(Zircon)* — undefined for a newborn, and it destroys what a supervisor would -inspect. The spawner passes the child's handle words in `a0` and `a1` (the demo's Console and endpoint), or -names a start Region in `a0` — a runtime convention, not kernel ABI; only the root receives BootInfo (§9). +again. One whose threads have all `Exited` is dead but intact, inspectable through `INFO` until then +*(seL4: objects live until destroyed)*. **Rejected:** ending with the last thread *(Zircon)* — undefined for +a newborn, and it destroys what a supervisor would inspect. The spawner passes handle words in `a0` and +`a1` (the demo's Console and endpoint), or a start Region in `a0` — a runtime convention, not kernel ABI. ## 9. The root task and boot info The one act the kernel performs unasked: at boot it builds **exactly one** process from the first boot module and gives it every boot capability *(seL4's root task; research/0002 Part 3, "the bootstrap seam is -a one-time kernel act")* — an AddressSpace, the segments mapped as described (§10), a stack, a read-only +a one-time kernel act")* — an AddressSpace, the segments mapped as in §10, a stack, a read-only **BootInfo** page, a Process and a Thread in §3's entry state with `a0` = BootInfo's address and `a1` = 0. Its priority, ceiling and full context are RFC-0006 (proposed) §6's one bootstrap act. **BootInfo**, in `user/abi`, is a header (magic, version, length), one `#[repr(C)] BootCap { kind: u32, flags: u32, handle: u64, base: u64, size: u64 }` per initial capability, and a string table for module names, pinned within a page by a host test. Every initial capability is *listed*, none at a well-known -slot, so the order can change without an ABI break and a reviewer can enumerate the whole grant. Kinds: the -root's Process, the Console (§11), a read-only Region per further module with its descriptor; RFC-0005 -adds the AddressSpace, the one root Pool and the DTB; RFC-0006 the Thread, a `SchedControl` per core and the -`IrqControl`. A kind not yet built is absent, so BootInfo grows with the increments. +slot, so a reviewer can enumerate the whole grant. Kinds: the root's Process, the Console (§11), a +read-only Region per further module with its descriptor; RFC-0005 adds the AddressSpace, the one root Pool +and the DTB; RFC-0006 the Thread, a `SchedControl` per core and the `IrqControl`. Kinds appear as built. -**The root is not a superuser.** Its capabilities are ordinary, droppable and generation-checked, and the -rule is checkable: **no kernel code branches on "is this the root task"**. They are **not revocable** until -RFC-0003a, though research/0002 Part 3 asks for revocable init capabilities: today only destruction revokes. -Nothing the root needs waits on a userspace service. **Rejected:** a kernel boot manifest (policy in the -kernel); a `spawn_from_module` syscall (a loader as a kernel service); a `system:masters` superuser. +**The root is not a superuser.** Its capabilities are ordinary, droppable and generation-checked, and **no +kernel code branches on "is this the root task"**. They are **not revocable** until RFC-0003a, though +research/0002 Part 3 asks for revocable init capabilities. **Rejected:** a kernel boot manifest (policy in +the kernel); a `spawn_from_module` syscall (a loader as a kernel service); a `system:masters` superuser. ## 10. Getting the first images into memory **Verdict sought — the mechanism:** the kernel reaches modules through `arch::boot_modules()`, which yields fixed-shape **load descriptors** made on the host, and **never parses ELF**. A dependency-free, host-tested -ELF64 reader in `xtask` emits per program an entry point; a text segment (`R+X`), an optional read-only -segment (`R`) and a data segment (`R+W`, zero-padded to a page, with a `bss` size), page-aligned in the user -half; and stack and BootInfo addresses. It **refuses** a writable-and-executable, overlapping or unaligned -`PT_LOAD`, `PT_DYNAMIC`, `PT_INTERP`, `PT_TLS`, `PT_GNU_RELRO` and an executable `PT_GNU_STACK`, ignoring -`PT_NOTE` and a non-executable `PT_GNU_STACK`; `user/link/.ld` declares `PHDRS` so lld emits no other. -W^X is checked before the image exists (O-13), and the kernel validates one fixed structure — Nexen's -finding that kernel code validating structures built elsewhere is the bug farm (research/0002 Part 7). -**Rejected:** an in-kernel `elf/` crate — small and testable, but a parser in the TCB for no gain. - -**The source — an AArch64 interim: embedded in the kernel image.** QEMU's `-kernel` loads one ELF; -`-initrd` and `-device loader` add a second, QEMU-only artefact (and in `hw/arm/boot.c`, read at QEMU 8.2.2 -and to be re-read at the pinned 11.0.3 before acceptance, `-initrd` loads only for Linux images). Linking -children into the root's ELF *(seL4's CPIO archive)* would make the root specific to each image. So `xtask` -writes `payload.bin` — a versioned `abi` header, then per module a name, descriptor and segment bytes — and -names it in `SETONIX_PAYLOAD`; `kernel/build.rs` re-runs on that variable, substituting an empty bundle in -`OUT_DIR` when it is unset, so a bare `cargo build` links and prints `boot modules: 0`. A `global_asm!` -`.incbin` under `kernel/src/arch/aarch64/` embeds it, and `aarch64.ld` places it in a page-aligned, -read-only, `KEEP`ed `.payload` section *(Hubris's `xtask dist`)*. **Its cost:** a user change relinks the -kernel. **On x86_64**, a future x86_64 boot RFC provides the UEFI stub that loads the same bundle as a -file; increment 12 depends on it. +ELF64 reader in `xtask` emits an entry point; text (`R+X`), optional read-only (`R`) and data (`R+W`, +zero-padded to a page, with a `bss` size) segments, page-aligned in the user half; and stack and BootInfo +addresses. It **refuses** a writable-and-executable, overlapping or unaligned `PT_LOAD`, `PT_DYNAMIC`, +`PT_INTERP`, `PT_TLS`, `PT_GNU_RELRO` and an executable `PT_GNU_STACK`, ignoring `PT_NOTE` and a +non-executable `PT_GNU_STACK`; `user/link/.ld` declares `PHDRS` so lld emits no other. W^X is checked +before the image exists (O-13), and the kernel validates one fixed structure — Nexen's finding that kernel +code validating structures built elsewhere is the bug farm (research/0002 Part 7). **Rejected:** an +in-kernel `elf/` crate, a parser in the TCB for no gain. + +**The source — an AArch64 interim: embedded in the kernel image.** QEMU's `-initrd` and `-device loader` +add a second, QEMU-only artefact (`hw/arm/boot.c`, read at QEMU 8.2.2 and to be re-read at the pinned +11.0.3, loads `-initrd` only for Linux images); children linked into the root's ELF *(seL4's CPIO +archive)* would tie the root to each image. So `xtask` writes `payload.bin` — a versioned header, then per +module a name, descriptor and segment bytes — named by `SETONIX_PAYLOAD`; `kernel/build.rs` re-runs on that +variable and substitutes an empty bundle when it is unset, so a bare `cargo build` prints `boot modules: +0`. A `global_asm!` `.incbin` under `kernel/src/arch/aarch64/` embeds it; `aarch64.ld` places it in a +page-aligned, read-only, `KEEP`ed `.payload` section *(Hubris's `xtask dist`)*. **Cost:** a user change +relinks the kernel. **On x86_64**, a future x86_64 boot RFC provides the UEFI stub that loads the same +bundle as a file; increment 12 depends on it. **Loading.** The kernel loads **module 0 only**; the others reach the root as **read-only Regions** -(RFC-0005 §7 mints them `READ | EXECUTE | DUPLICATE | TRANSFER`) with descriptors in BootInfo *(Genode's -core hands boot modules to init read-only)*. Every loader, kernel or root, maps text `RX` and read-only -data `R` **in place** from the module Region, copies data pages into a fresh Region with `region_copy` — -which RFC-0005 (proposed) provides as a page-granular, frame-to-frame copy through its volatile-copy -module — and takes `bss` as fresh zeroed pages. No loader maps a child's writable memory into itself. The -programs, `user/demo/server` (the root, for now) and `user/demo/client`, are soft-float `no_std` crates on -`abi` alone with no `unsafe`; for `x86_64-unknown-none` `xtask` passes `-C relocation-model=static -C -code-model=small`, since the target defaults to PIE and the kernel code model. +(RFC-0005 §7) with descriptors in BootInfo *(Genode's core hands init boot modules read-only)*. Every +loader, kernel or root, maps text `RX` and read-only data `R` **in place**, copies data pages into a fresh +Region with `region_copy` — which RFC-0005 (proposed) provides as a page-granular, frame-to-frame copy, the +one it already makes for the root (§7 there), so the demo needs that half of its copy module — and takes +`bss` as fresh zeroed pages, so no loader maps a child's writable memory into itself. The programs are soft-float `no_std` crates on `abi` alone with no `unsafe`; +on `x86_64-unknown-none` `xtask` passes `-C relocation-model=static -C code-model=small`. ## 11. The console as a capability -Userspace reaches the kernel's UART only through a **Console** object with one method, `WRITE`, by `call`: -the byte count in MR0, at most 24 bytes in MR1–MR3, sent verbatim (`abi`'s writer supplies `\r\n`). **It -never waits:** the kernel stores bytes while the PL011's transmit-full flag is clear and returns the count -accepted; `abi` yields and retries the rest, so any waiting happens at EL0, preemptibly. The kernel's hold -is at most 24 flag reads and 24 stores, inside RFC-0006 (proposed)'s measured worst path, so the console -sets no floor under `MIN_BUDGET` and needs no preemption point. The root receives it with `DUPLICATE | -TRANSFER | WRITE` (no `READ`: there is no input path) and derives `WRITE`-only copies for the children it -chooses; a program not handed one cannot print, but for §7's interim. **Rejected:** seL4's `DebugPutChar`, -which any thread may issue in a `CONFIG_PRINTING` build, and a console at a well-known handle. **Adopted in -spirit:** Zircon's debuglog, write-only by default. Once a userspace UART driver exists the root hands out -an endpoint speaking `WRITE`, no client changes, and the Console object is **deleted** (increment 13). +Userspace reaches the kernel's UART only through a **Console** object with one method, `WRITE`: the byte +count in MR0, at most 24 bytes in MR1–MR3, sent verbatim (`abi`'s writer supplies `\r\n`). **It never +waits:** the kernel stores bytes while the PL011's transmit-full flag is clear and returns the count +accepted, and `abi` yields and retries the rest, so any waiting happens at EL0, preemptibly. The hold is at +most 24 flag reads and 24 stores, inside RFC-0006 (proposed)'s measured worst path, so the console sets no +floor under `MIN_BUDGET` and needs no preemption point. The root receives it with `DUPLICATE | TRANSFER | +WRITE` (no input path, so no `READ`) and derives `WRITE`-only copies for chosen children; a program not +handed one cannot print, but for §7's interim. **Rejected:** seL4's `DebugPutChar`, any thread's in a +`CONFIG_PRINTING` build; a console at a well-known handle. **Adopted in spirit:** Zircon's write-only +debuglog. Once a userspace UART driver exists the root hands out an endpoint speaking `WRITE`, no client +changes, and the Console object is **deleted** (increment 13). ## 12. `user/abi` — the kernel's own binding -**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment — -allocation, formatting, start-up — and issues no syscalls: relibc sits on `redox_syscall`, versioned with -the Redox kernel, and seL4 keeps libsel4 in its kernel's repository. The binding is the user half of §3's -table, with nothing to port. **Verdict sought: the microkernel-core row, write ourselves**; anything that -would read the same on another kernel moves out. Appending "syscall ABI binding (`user/abi`)" to that row -is constitution text and the maintainer's (Constitution §4, logged in `docs/CHANGELOG.md`); increments 2 -and 7 wait on it. `abi` holds the numbers, `Handle`, `Tag`, `Status`, labels, `BootCap`, the layouts with -`const` assertions, and safe wrappers — no allocator, no formatting. The kernel and `xtask` build it -without its `syscalls` feature, the one that compiles `src/arch/**`, so all three share one data half. - -**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is `user/abi/src/arch/**` -and holds only: the `svc`/`syscall` wrappers; the discovery-register read; and the entry stubs. -`root_start` copies the BootInfo page by volatile reads onto its stack and hands `main` a reference to the -copy — sound because the kernel mapped that page read-only before the process's first instruction and -nothing else has run; `child_start` passes `a0` and `a1`. User programs contain no `unsafe`, and no Rust -reference is formed into memory another thread may write. **Sought separately, not for the demo:** -volatile word accessors for the IPC buffer in `user/abi/src/buffer/**`, whose invariant — the buffer stays -mapped at its bound address while the thread runs — the process itself upholds; broken, it faults the -thread and touches no other process. +**Which ledger row.** The libc/runtime row ("port code — relibc pieces") gives a program an environment +and issues no syscalls: relibc sits on `redox_syscall`, versioned with the Redox kernel, and seL4 keeps +libsel4 in its kernel's repository. The binding is the user half of §3's table, with nothing to port. +**Verdict sought: the microkernel-core row, write ourselves**; anything that would read the same on +another kernel moves out. Appending "syscall ABI binding (`user/abi`)" to that row is constitution text, +the maintainer's (Constitution §4, logged in `docs/CHANGELOG.md`); increments 2 and 7 wait on it. `abi` +holds the numbers, `Handle`, `Tag`, `Status`, labels, `BootCap`, the layouts with `const` assertions and +safe wrappers — no allocator, no formatting. The kernel and `xtask` build it without its `syscalls` +feature, the one that compiles `src/arch/**`, so all three share one data half. + +**The third designated `unsafe` tree**, approved in principle on 2026-09-27, is `user/abi/src/arch/**`, +holding only the `svc`/`syscall` wrappers, the discovery-register read and the entry stubs. `root_start` +copies the BootInfo page by volatile reads onto its stack and hands `main` a reference to the copy — sound +because the kernel mapped the page read-only before the process's first instruction and nothing else has +run; `child_start` passes `a0` and `a1`. User programs contain no `unsafe`, and no Rust reference is formed +into memory another thread may write. **Sought separately, not for the demo:** volatile IPC-buffer word +accessors in `user/abi/src/buffer/**`, whose invariant — the buffer stays mapped at its bound address while +the thread runs — the process itself upholds; broken, it faults that thread and touches no other process. **The edits, all in increment 7:** `CLAUDE.md` § `unsafe` policy gains "`user/abi/src/arch/**` — trap instructions and process entry stubs" and § Layout gains `user/`; threat-model O-6 names three trees; the -workspace `Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep widen to the repository; +`Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep widen to the repository; and `xtask/Cargo.toml`'s "No dependencies" becomes "no external dependencies". ## 13. Lineage -| Source | What is taken | What is left | -|--------|---------------|--------------| -| **seL4** | the root task and boot info; kernel objects invoked as IPC, errors in the reply label; presented capabilities (`extraCaps`); four physical message registers; `ReplyRecv`; reusable reply objects; transfer at rendezvous; libsel4 shipped with its kernel | CNodes and depth-addressed pointers; `DebugPutChar`; 120-word messages; the IPC-buffer thread-local | -| **Zircon / Fuchsia** | the process as an object holding a table and an address space; debuglog as a write-only handle | a syscall per operation; one start handle as the only channel; a process ending with its last thread | -| **Linux** | `svc #0` + `x8`; `syscall` + `r10`; the `sysretq` guards; `nr_open` as a sizing reference | everything the registers carry | -| **x86_64 ELF TLS** | a block whose first word holds its own address | `FS` itself, left to TLS | -| **Hubris** | the build tool assembles one bootable image | a static task set with no spawn | -| **Redox** | `redox_syscall` versioned with the kernel, relibc above it | ambient path syscalls (research/0001) | -| **Genode** | boot modules handed to init as read-only memory | — | +Named inline. Mostly **seL4** (root task, invocation as IPC, `extraCaps`, reply objects, transfer at +rendezvous; not CNodes, `DebugPutChar` or 120-word messages) and **Zircon** (the process object, debuglog; +not a syscall per operation), with **Linux**, **Hubris**, **Genode** and **Redox** for smaller pieces. ## 14. Obligations @@ -434,7 +398,6 @@ workspace `Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep ## 15. Graves checked (§3) - **Policy in the kernel.** The kernel runs one image and lists what it granted; the rest is the root's. - The boot manifest and `spawn_from_module` were this grave's doors, and stay shut. - **The catch-all right.** No root identity survives `eret`; no right is added. A Process capability reaches its table only through `GRANT` before start, then means `KILL`; control of a running child lives in separate AddressSpace and Thread capabilities, each droppable. @@ -442,24 +405,23 @@ workspace `Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep - **Bolted-on multicore.** Entry is per-core from the first stub (`SP_EL1`, `TPIDR_EL1`; `swapgs`). RFC-0006 (proposed) §12 provides the big kernel lock taken on entry, answering RFC-0003 §14.2; single fetch stays necessary under it, because other threads write the buffer from EL0 without the lock. -- **Drivers pulled in; unused device paths.** The Console is the only device method reachable from EL0: - write-only, bounded, used by every boot-test, its deletion scheduled. +- **Drivers pulled in; unused device paths.** The Console alone is reachable from EL0, bounded, its + deletion scheduled. ## 16. Costs — what this makes harder -- **`call` is overloaded** — per-type method decoding and two error layers, the price of interposition. - **Interposition is partial:** a forwarder for a method with handle-word arguments must first be handed - those capabilities, costing the client a `TRANSFER` it may not hold — input to RFC-0003a. -- **Four physical registers and 64 words:** past 32 bytes a message touches memory, and a 65–120-word - message seL4 would copy needs a Region here; raising either later breaks binaries. +- **`call` is overloaded** — per-type method decoding and two error layers, the price of interposition — + and **interposition is partial:** a forwarder for a method with handle-word arguments must first be handed + those capabilities, costing the client a `TRANSFER` it may not hold (input to RFC-0003a). +- **Four physical registers and 64 words:** past 32 bytes a message touches memory, a 65–120-word message + seL4 would copy needs a Region here, and raising either later breaks binaries. - **Handles are 20/44 for good,** the accepted crate changes by amendment, and `TPIDRRO_EL0` and the user GS base belong to the ABI rather than a runtime. - **A spawner keeps control if it wants to:** the window stops injection, not a loader that kept derived copies of the child's AddressSpace or Threads, and the child cannot yet check. -- **The root task is the most sensitive process,** as in seL4, its capabilities not revocable until - RFC-0003a; it must stay small. **No ambient printing; late grants need a willing child.** -- **The Console moves at most 24 bytes per entry** and returns short counts, so every writer loops. -- **No PAN on the demo CPU;** `user/abi` is a third `unsafe` tree to audit. +- **The root is the most sensitive process,** as in seL4, and not revocable until RFC-0003a; it must stay + small. **No ambient printing; late grants need a willing child; every console writer loops** on short + 24-byte writes. **No PAN on the demo CPU;** `user/abi` is a third `unsafe` tree to audit. ## 17. Open questions @@ -477,42 +439,38 @@ workspace `Cargo.toml` comment ("the two trees") and `kernel/src/main.rs`'s grep Each is one reviewable PR; **[demo]** marks what the Phase-1 demo needs. Userspace boot-tests expect prefixed strings, since bare `Kaya!` matches the kernel's greeting; every demo line fits one `WRITE`. -1. **[demo] `capability`: the handle word** — `to_word`/`from_word`, `MAX_SLOT_GENERATION`, `free_slots`, - with RFC-0003's amendment. Host tests: round trips; word 0 and index ≥ 2^20 refused; a slot retires at - the bound while an object generation passes it; `free_slots` exact beside retired slots. -2. **[demo] `user/abi`, data half**, after the ledger verdict — numbers, `Tag`, `Status`, labels, `BootCap`, - the rights assertion, no `unsafe`. Host tests refuse every reserved-bit (9–15 included) and over-length - encoding; CI builds both targets. -3. **[demo] AArch64 trap path, MMU off** — save into one kernel-held frame (RFC-0006's increment 4 moves - frames into TCBs), dispatch, `eret`; `yield`, `thread_exit`, `InvalidSyscall`; an EL0 fault stops the - self-test, not the kernel: `--features syscall-selftest --expect "syscall round trip"`. **Deleted** when - RFC-0005's increment 9 turns the MMU on; 6 replaces it. +1. **[demo] `capability`: the handle word**, `free_slots`, with RFC-0003's amendment. Host tests: round + trips; word 0 and index ≥ 2^20 refused; retirement at the bound; `free_slots` exact beside retired slots. +2. **[demo] `user/abi`, data half**, after the ledger verdict, no `unsafe`. Host tests refuse every + reserved-bit (tag bits 9–15 included) and over-length encoding; CI builds both targets. +3. **[demo] AArch64 trap path, MMU off** — one kernel-held frame (RFC-0006's increment 4 moves frames into + TCBs), dispatch, `eret`; `yield`, `thread_exit`, `InvalidSyscall`; an EL0 fault stops the self-test, not + the kernel: `--features syscall-selftest --expect "syscall round trip"`. **Deleted** by RFC-0005's 9. 4. **[demo] `xtask`: repeated `--expect`, matched in order**, host-tested. -5. **[demo] The bundle** — the ELF reader, host-tested on W+X, overlapping, truncated, `PT_TLS`, - `PT_DYNAMIC` and executable-stack fixtures; `build.rs`, `.incbin`, `.payload`: `--expect "boot modules: 0"`. -6. **[demo] Objects, `cap_*`, the dispatcher, the Console and the Process's kernel half** (create, - `INFO`), on RFC-0005's store and root Pool with its increment 14. An EL0 self-test in a kernel-built - process prints through a Console handle: `--features syscall-selftest --expect "[el0] Kaya!"`. -7. **[demo] The root reaches EL0** — `user/abi/src/arch/**` and every §12 edit in one PR, once the - maintainer approves the tree; module 0 becomes the root: `--expect "boot modules: 2" --expect "[root] up"`. -8. **[demo] `GRANT`, the window, `INFO` and `KILL`** as pure logic in a host-tested `process/` crate, then - wired. Host tests: born empty; `GRANT` all-or-nothing on an exact free-slot count; `WindowClosed` after - the first resume; an AddressSpace bound once; `KILL` empties the table. +5. **[demo] The bundle** — the ELF reader, host-tested on W+X, overlapping, truncated, `PT_TLS`, `PT_DYNAMIC` + and executable-stack fixtures; `build.rs`, `.incbin`, `.payload`: `--expect "boot modules: 0"`. +6. **[demo] Objects, `cap_*`, the dispatcher, the Console and the Process's kernel half** (create, `INFO`) + on RFC-0005's store and root Pool, with its 14: an EL0 self-test in a kernel-built process prints + through a Console handle, `--features syscall-selftest --expect "[el0] Kaya!"`. +7. **[demo] The root reaches EL0** — `user/abi/src/arch/**` and every §12 edit in one PR, once the tree is + approved; module 0 becomes the root: `--expect "boot modules: 2" --expect "[root] up"`. +8. **[demo] `GRANT`, the window, `INFO`, `KILL`** in a host-tested `process/` crate, then wired: born empty; + `GRANT` all-or-nothing; `WindowClosed` after the first resume; an AddressSpace bound once. 9. **[demo] The root spawns the client,** in three PRs: - - **a.** The loader plan in a host-tested `no_std` `user/loader` crate: descriptor in, method calls - out, W^X and overlaps refused; a host test that the spawner ends holding only the Process capability. - - **b.** The root runs it, granting a `WRITE`-only Console and a send-only endpoint: + - **a.** A host-tested `no_std` `user/loader` crate: descriptor in, method calls out, W^X and overlaps + refused; a host test that the spawner ends holding only the Process capability. + - **b.** The root runs the plan, granting a `WRITE`-only Console and a send-only endpoint: `--expect "[root] up" --expect "[client] up"`. - - **c.** O-27 evidence: the client probes indices 0–63 at generation 1 with `cap_rights` — exhaustive - while no slot has been vacated — printing `[client] 2 handles`; the root prints `INFO`'s count, + - **c.** O-27 evidence: the client probes indices 0–63 at generation 1 with `cap_rights` (exhaustive + while no slot has been vacated), printing `[client] 2 handles`, and the root prints `INFO`'s count, `[root] client: 2 caps`. Evidence, not proof: the proof is increment 8's host test. 10. **[demo] IPC syscalls** over RFC-0004 endpoints, reusable Reply objects and RFC-0006's blocking states: - `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!` in order. Host test: calls from clients - that exit mid-call leave the server's Pool balance unchanged. -11. **Fault messages and death notification** (§17.1): a faulting child is reported to the root, which - prints `[root] child faulted`; `report-user-faults` leaves the boot-tests. -12. **x86_64 trap path and `abi` stubs** — `EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the `sysretq` - guards — after the x86_64 boot RFC's UEFI stub; the demo boot-tests on `q35`. + `[client] -> Kaya!`, `[server] <- Kaya!`, `[client] <- Kaya!` in order. Host test: clients that exit + mid-call leave the server's Pool balance unchanged. +11. **Fault messages and death notification** (§17.1): the root prints `[root] child faulted`, and + `report-user-faults` leaves the boot-tests. +12. **x86_64 trap path and `abi` stubs** (`EFER.SCE`, `LSTAR`/`STAR`/`FMASK`, `swapgs`, the `sysretq` + guards), after the x86_64 boot RFC's UEFI stub: the demo boot-tests on `q35`. 13. **Delete the Console object** when the Phase-2 UART driver lands. 14. **`Revoked`**: the crate's `ObjectDestroyed` split, with RFC-0005's destruction (its 17). 15. **The root spawns the server** and stops serving, removing that interim. @@ -520,18 +478,17 @@ prefixed strings, since bare `Kaya!` matches the kernel's greeting; every demo l **Demo order across the three RFCs,** for the maintainer to fix: RFC-0005 1–8, RFC-0006 1–2 and this RFC's 1, 2 and 4 in any order → 3 → RFC-0006 3–4 → RFC-0005 9–10, retiring the MMU-off self-tests → RFC-0005 11–14 → RFC-0006 5 → 5–6 → 7 → 8 → RFC-0006 6a–6b → 9 → RFC-0006 7a–7b → 10 with RFC-0006 7c. -RFC-0006 places its 3–4 before RFC-0005's MMU and RFC-0005 after; this takes RFC-0006's, with RFC-0005's -10 repointing the GIC, as RFC-0005 allows. +RFC-0006 puts its 3–4 before RFC-0005's MMU and RFC-0005 after; this takes RFC-0006's, with RFC-0005's 10 +repointing the GIC, as RFC-0005 allows. **The demo's interims, with their costs:** - **Faults print and stop** (§7): an ambient output channel and ~100 bytes of kernel UART time per fault. -- **The in-kernel Console** (§11), bounded to 24 stores per entry; **modules in the AArch64 image** (§10), - so a user change relinks the kernel. +- **The in-kernel Console** (§11); **modules in the AArch64 image** (§10), relinking the kernel per change. - **One root Pool, no destruction** (RFC-0005): charges unattributed, `KILL` releases nothing, no `Revoked`. - **Register-only messages** (RFC-0005's IPC-buffer module, its 16, is not demo): no thread has a buffer and no capability crosses an endpoint; the demo's grants use `GRANT`, whose words are registers by design. -- **An active server** (RFC-0006's A1), so RFC-0004's donation is not exercised; and **the root doubles as +- **An active server** (RFC-0006's A1), so RFC-0004's donation is not exercised; **the root doubles as server**, running client-facing code in the process holding every boot capability, until increment 15. - **The MMU-off self-test** (increment 3); **RFC-0006's provisional constants** for the root's scheduling. From 8d9feb030ef97a9107ad514f26ea20928adf0fc1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:54:38 +0000 Subject: [PATCH 19/20] docs(changelog): record RFC-0005, RFC-0006 and RFC-0007 as proposed Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/CHANGELOG.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 05dc535..eb2dbda 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -12,6 +12,51 @@ recorded is indistinguishable from law that was never agreed. ### Added +- `docs/rfcs/0005-address-spaces-and-physical-memory.md` — **proposed**, the memory half of Phase 1 and + the first of three interlocking kernel RFCs drafted together with RFC-0006 and RFC-0007. Physical + memory comes from firmware tables into one bounded `MemoryMap` (the image moves to 0x4020_0000 because + QEMU places no DTB for an ELF at the base of RAM), and a colour-capable bitmap hands out frames. O-7's + "who pays" is answered with typed fixed-capacity slot arrays reused by generation and a + capability-named `Pool` charged for every post-boot frame and slot, `map`'s tables included. The Pool + is declared the single accounting domain for counted units, with CPU time left to RFC-0006's `Core`: a + stated split from research/0002. `Region` and `AddressSpace` join RFC-0003's objects, and `EXECUTE` + joins its rights as bit 5 by a proposed dated amendment. W^X (O-13) holds in four layers, including per + Region across every address space, backed by `SCTLR_EL1.WXN`, whose full value the RFC states. O-5 + becomes a structural rule: the kernel never dereferences a user virtual address. The SMP shootdown and + ASID-rollover protocol is designed now rather than bolted on later. Donation (`reissue`) is + deliberately not proposed but handed to RFC-0003a as input, because gating it on `REVOKE` would + pre-empt that RFC's (a)/(b) choice. Twenty increments, fourteen marked for the Phase-1 demo, each with + the interims it carries named and costed. +- `docs/rfcs/0006-threads-and-scheduling.md` — **proposed**, revised the same day against a two-lens + review (constitution and implementability). Threads, scheduling contexts and interrupts for Phase 1. + The spine is unchanged. The kernel is event-driven, with one stack per core and preemption points, and + time is a capability: a `SchedContext` configured only through a per-core `SchedControl` (seL4 MCS). + Priorities are 256 fixed levels behind a priority ceiling that any capability to a thread confers. The + timer is tickless on both Tier-1 targets. FP/SIMD is enabled per thread and saved lazily. A device line + reaches its driver as a notification and stays masked until `ack`. The review turned several quiet + choices into verdicts. Endpoint queues are now priority-ordered, because the FIFO citation was wrong. + Budgets are declared caps, not minima, departing from research/0002 on purpose. Userspace timers are + named: deadline objects on the release queue. A caller's death can no longer take time away from a + server mid-request. The departures from accepted papers are now listed as amendments A1–A4: donation + only to passive receivers, a short caller waits instead of being refused, seL4's timeout faults + corrected, and the big lock rejecting RFC-0003 §14.2's premise. The review also fixed the state machine + (a tenth state, `Unfunded`), the FP ownership races, the SCTLR bits RFC-0005 had inverted, and the + parked-core start-up (checked against QEMU). The increments now interlock with RFC-0005 and RFC-0007. + The demo's interims are named with their numbers. +- `docs/rfcs/0007-system-calls-and-process-bootstrap.md` — **proposed**, the third of the Phase-1 kernel + RFCs drafted alongside RFC-0005 and RFC-0006, and the one that makes RFC-0003 §9 live. Every + kernel-object operation is a `call` on its capability (seL4's uniform invocation), with twelve syscalls + in all. A method's extra handle words ride in message registers, *presented* rather than moved, so + `map` never costs the mapper its Region. The IPC register budget RFC-0004 §9.4 left open is settled at + four physical and 64 virtual registers, with an optional 1 KiB buffer fetched once at rendezvous. The + handle word is fixed at a 20-bit index and a 44-bit slot generation. A Process object is born with an + empty table and filled only by explicit `GRANT` before its first thread runs, which discharges O-27 and + states honestly what that window does not close: a spawner that keeps the child's AddressSpace keeps + control. There is one root task with every boot capability listed in BootInfo, and no kernel branch on + "is root". The kernel never parses ELF. The console is a capability that never waits. The `user/abi` + binding gets a third designated `unsafe` tree, limited to trap instructions and entry stubs. It + proposes dated amendments to RFC-0003 (the handle word) and RFC-0004 (reusable reply objects), and + names every demo interim with its cost. - `docs/research/0002-virtualisation-and-containment-prior-art.md` — the second prior-art review, on the maintainer's initiative: what the virtualisation and containerisation fields learned under production fire, mined for Setonix. Six research sweeps (hypervisors/microVMs, container From 647ff5d2304ab8745a969944eea293f90e5b511d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 14:54:50 +0000 Subject: [PATCH 20/20] docs(changelog): RFC-0006 was reviewed through three lenses, not two Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D1gBBT3tN99wzeJjf3tLWW --- docs/CHANGELOG.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index eb2dbda..8989bba 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -27,22 +27,22 @@ recorded is indistinguishable from law that was never agreed. deliberately not proposed but handed to RFC-0003a as input, because gating it on `REVOKE` would pre-empt that RFC's (a)/(b) choice. Twenty increments, fourteen marked for the Phase-1 demo, each with the interims it carries named and costed. -- `docs/rfcs/0006-threads-and-scheduling.md` — **proposed**, revised the same day against a two-lens - review (constitution and implementability). Threads, scheduling contexts and interrupts for Phase 1. - The spine is unchanged. The kernel is event-driven, with one stack per core and preemption points, and - time is a capability: a `SchedContext` configured only through a per-core `SchedControl` (seL4 MCS). - Priorities are 256 fixed levels behind a priority ceiling that any capability to a thread confers. The - timer is tickless on both Tier-1 targets. FP/SIMD is enabled per thread and saved lazily. A device line - reaches its driver as a notification and stays masked until `ack`. The review turned several quiet - choices into verdicts. Endpoint queues are now priority-ordered, because the FIFO citation was wrong. - Budgets are declared caps, not minima, departing from research/0002 on purpose. Userspace timers are - named: deadline objects on the release queue. A caller's death can no longer take time away from a - server mid-request. The departures from accepted papers are now listed as amendments A1–A4: donation - only to passive receivers, a short caller waits instead of being refused, seL4's timeout faults - corrected, and the big lock rejecting RFC-0003 §14.2's premise. The review also fixed the state machine - (a tenth state, `Unfunded`), the FP ownership races, the SCTLR bits RFC-0005 had inverted, and the - parked-core start-up (checked against QEMU). The increments now interlock with RFC-0005 and RFC-0007. - The demo's interims are named with their numbers. +- `docs/rfcs/0006-threads-and-scheduling.md` — **proposed**, revised the same day against a three-lens + review (constitution, technical correctness and implementability). Threads, scheduling contexts and + interrupts for Phase 1. The spine is unchanged. The kernel is event-driven, with one stack per core and + preemption points, and time is a capability: a `SchedContext` configured only through a per-core + `SchedControl` (seL4 MCS). Priorities are 256 fixed levels behind a priority ceiling that any + capability to a thread confers. The timer is tickless on both Tier-1 targets. FP/SIMD is enabled per + thread and saved lazily. A device line reaches its driver as a notification and stays masked until + `ack`. The review turned several quiet choices into verdicts. Endpoint queues are now priority-ordered, + because the FIFO citation was wrong. Budgets are declared caps, not minima, departing from + research/0002 on purpose. Userspace timers are named: deadline objects on the release queue. A caller's + death can no longer take time away from a server mid-request. The departures from accepted papers are + now listed as amendments A1–A4: donation only to passive receivers, a short caller waits instead of + being refused, seL4's timeout faults corrected, and the big lock rejecting RFC-0003 §14.2's premise. + The review also fixed the state machine (a tenth state, `Unfunded`), the FP ownership races, the SCTLR + bits RFC-0005 had inverted, and the parked-core start-up (checked against QEMU). The increments now + interlock with RFC-0005 and RFC-0007. The demo's interims are named with their numbers. - `docs/rfcs/0007-system-calls-and-process-bootstrap.md` — **proposed**, the third of the Phase-1 kernel RFCs drafted alongside RFC-0005 and RFC-0006, and the one that makes RFC-0003 §9 live. Every kernel-object operation is a `call` on its capability (seL4's uniform invocation), with twelve syscalls