Optimize anonymous mmap and page-fault paths

[Max042004]

Anonymous memory is reworked on a lazy foundation: sys_mmap only records the region, while page tables, zeroing and host-memory commit are deferred to first touch. Two optimization layers sit on top.

mmap: the common shape (addr == 0, MAP_PRIVATE|MAP_ANONYMOUS, RW) is served entirely at EL1 with no vCPU exit and no lock. The shim stays stateless: it bump-allocates from a host-carved per-vCPU VA arena and reports each grant through a per-vCPU SPSC ring that the host drains whenever it takes mmap_lock, so no cross-vCPU synchronization exists. Arenas recycle freed address space through the gap allocator, refill automatically when a request does not fit (the new arena sized to cover the triggering request, with an ARENA_MAX guard so oversized requests fall back without churning the arena), and decay back to workload scale based on served-length history.

Page fault: a per-2MiB-block dirty map lets materialization skip zeroing blocks whose slab bytes are known zero, moving the host page commit off the fault and out of mmap_lock onto the guest's own first writes. The TLBI protocol emits a single range op per materialized block instead of per-page invalidations, and a repeat fault on an already-valid block returns without re-zeroing.

Host-side access to untouched lazy memory (I/O buffers, futex words, signal frames, shmat/shmdt copies) materializes on demand with lock-order-safe pre-faulting, and PROT_NONE stays a pure faulting reservation. tests/test-mmap-lazy.c and tests/test-mmap-fastpath.c pin the contract: zero-on-reuse, huge sparse reservations, host access to untouched memory, PROT_NONE round trips, fork, racing first touch, and arena refill/revocation.

Close #165

---

Summary by cubic

Implements lazy anonymous mmap (defers PTEs, zeroing, and host commit to first touch) and adds an EL1 fast path for common anonymous RW mmaps to cut latency and avoid contention. Also adds safe pre‑faulting for host use, drains EL1 rings on lock acquisition, and bumps the fork IPC protocol. Closes #165.

• New Features

  • Lazy anon memory with a per‑2MiB dirty map: skip zeroing clean blocks, batch TLBI per block, and ignore repeat faults on already‑valid blocks.
  • EL1 mmap fast path for mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, ...): per‑vCPU VA arenas + SPSC rings, no exits/locks, adaptive refill with max‑size guard and automatic fallback; host drains rings via mmap_lock_acquire/release/cond_wait; per‑vCPU counters and tests.
  • Host access pre‑faulting to materialize untouched lazy memory for futex words, signal frames, exec argv/env copying, and SysV SHM; PROT_NONE remains a pure reservation.
  • Docs and tools: usage docs for the fast path, new tests and a microbench, and stats scripts for dirty‑map and fast‑path behavior.

• Migration

  • The EL1 fast path is on by default and falls back when shape unsupported, arenas exhausted, or rings full. Set ELFUSE_MMAP_FASTPATH=0 to disable; verbose tracing, the syscall histogram, GDB, and Rosetta force the host path.
  • Fork IPC protocol bumped to ELFM. Update both parent and child when mixing versions.

^{Written for commit e75a363. Summary will update on new commits.}

[jserv]

Provide comprehensive benchmarks.

[jserv]

The EL1 fast path publishes the bump cursor (str x21, [x12, #40]) before the release stlr that publishes the ring tail. Because the host consumer runs concurrently on another thread and mmap_fastpath_skip_reserved (src/syscall/mem.c:417) only excludes the unconsumed tail [cursor, arena_limit), there is a window where a just-granted span leaks out as free VA:

vCPU A (in mmap_anon_fast) writes its ring slot for [old_cursor, new_cursor) and stores the new cursor, but has not yet executed the stlr tail. vCPU B takes a slow-path sys_mmap; mmap_lock_acquire drains A's ring but sees the old tail, so no region is recorded. find_free_gap_inner then observes A's new cursor and protects only [new_cursor, arena_limit), so [old_cursor, new_cursor) is neither in regions[] nor reserved, and gets handed to B. Two guest mappings alias the same VA -- data corruption and stale-byte exposure.

The window is real regardless of store order, because seeing a fresh cursor never implies the covering ring entry is visible. The robust fix is host-side: have mmap_fastpath_skip_reserved (and the recycle scan in mmap_fastpath_refill_thread_locked) exclude the entire enabled arena [arena_base, arena_limit) for every sibling slot, not just [cursor, limit). Reordering the shim so the cursor store follows the stlr narrows the window but does not close it under weak memory unless the cursor also carries release semantics paired with the consumer's acquire.

[jserv]

The drain fails closed only on ring corruption, arena-bounds violation, and table-full. guest_region_add_ex_owned_gpa (src/core/guest.c:2106) inserts a sorted region with no overlap check, so if a bad arena base or the cursor race in the fast path ever produces an overlapping grant, this path silently inserts a duplicate region over live VA instead of detecting the invariant break. Adding an overlap assertion here (the insertion neighbor is already known via region_lower_bound_start) turns a silent corruption into a fail-closed abort -- cheap defense-in-depth for the fast path's "EL1 only bump-allocates disjoint VA" contract.