Skip to content

feat(shim): containerd Runtime v2 shim for k8s RuntimeClass and docker - #10

Merged
reiase merged 6 commits into
developfrom
feat-pvisor-containerd-shim
Sep 30, 2026
Merged

reiase merged 6 commits into
developfrom
feat-pvisor-containerd-shim

Conversation

@lizhicui

Copy link
Copy Markdown
Collaborator

Summary

  • add persisting-shim, a containerd Runtime v2 shim (io.containerd.pvisor.v2, binary containerd-shim-pvisor-v2) that exposes pVisor as a Kubernetes RuntimeClass (handler: pvisor) and a Docker runtime (--runtime=pvisor), as a new crate without touching the existing product surface
  • implement the ttrpc task service (create/start/kill/wait/delete/state/pids/connect/shutdown): Create builds a plan from the OCI config plus the CreateTaskRequest mounts and re-executes the shim binary as an internal parent that enters namespaces, materializes the snapshotter rootfs, forks the init process, and releases it to pivot root and exec the workload on Start
  • keep task IO ownership in the shim: bundle FIFOs with a stdin keepalive so CloseIO yields EOF, PTY handling for both the console-socket convention (SCM_RIGHTS master handoff) and FIFO-based terminal IO (shim-side PTY relay), and ResizePty through the retained master; exec into running tasks joins the init process's namespaces and carries its own IO, exit tracking, and TaskCreate/Start/Exit/Delete plus exec events
  • add the pod-level Sandbox API for sandboxer = "shim": a dedicated bootstrap serves the task and sandbox services on one socket, a long-lived holder process replaces the pause container (joins the CRI pod network namespace, creates uts/ipc namespaces, optional pid sharing), and member containers join the holder's namespaces through the namespace paths in their specs
  • add a libkrun microVM executor behind the off-by-default vm cargo feature: annotation-routed tasks (io.pvisor.executor=vm) boot one VM per task with a virtio-fs rootfs, virtio-console IO, the implicit vsock disabled, and exec-in-VM through a vsock guest agent whose binary is injected into the rootfs at boot
  • fix real-machine compatibility issues found during bring-up: the fork-report pipe read-end plumbing, the BootstrapParams payload containerd >= 2.3 writes on stdin, runc-style PATH resolution for bare argv[0], shutdown/delete response ordering, TaskExit container-id semantics required by moby, CLOEXEC clearing for descriptors that cross the self-exec boundary, and namespace joins by path for pod containers

Validation

  • cargo fmt --all -- --check passes; cargo clippy -p persisting-shim --all-targets is warning-clean on the host and on the x86_64-unknown-linux-gnu target; the shim unit suite (39 tests) passes via cargo nextest run --locked -p persisting-shim
  • cross-builds pass for x86_64-unknown-linux-musl with cargo-zigbuild, both default and --features vm (the VM build yields a ~4 MB static binary)
  • on a remote Ubuntu 24.04 host with containerd 2.3.4 and Docker 29.7: ctr run covered the full lifecycle including exit-code propagation, signals, and deletion; docker run --runtime=pvisor covered attached runs, exit codes, docker exec, piped stdin with EOF, and stop semantics that match runc for PID-1 processes without SIGTERM handlers
  • CRI validation with sandboxer = "shim": crictl runp --runtime=pvisor reached Ready, a pod container shared the holder's network/uts/pid namespaces (verified by comparing namespace inodes), crictl exec worked, and the container completed a Running→Exited cycle
  • docker -t still reports "not a tty" inside the container: the PTY is allocated in the host namespace and the container's devpts instance does not contain the slave; the fix needs PTY allocation inside the container's mount namespace and is left out
  • the libkrun VM executor remains compile-level validated only: the available test host has no /dev/kvm, so VM workloads, the vsock agent, and exec-in-VM were not exercised on real hardware; pod-level VM sandboxes return a clear not-implemented error pointing at per-container VMs, and bind mounts and cgroup limits are not mapped into VMs yet

@lizhicui
lizhicui force-pushed the feat-pvisor-containerd-shim branch from 93c06e0 to b7cfc49 Compare September 30, 2026 11:35
First milestone of the pVisor containerd Runtime v2 shim: the host process
path. New crate crates/persisting-shim with the ttrpc task service, a
two-phase init pipeline (namespaces, rootfs mounts, pivot_root),
bundle FIFO stdio, task events, and a cgroup v2 subset.
@lizhicui
lizhicui force-pushed the feat-pvisor-containerd-shim branch from b7cfc49 to 2d99fd1 Compare September 30, 2026 11:36
@reiase
reiase merged commit 6d4d95b into develop Sep 30, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants