Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 93 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Run OpenCode inside a bubblewrap sandbox for security isolation.
- **Controlled Filesystem**: Most system filesystem mounted read-only
- **Custom Bind Mounts**: Add read-write or read-only access with `--with` and `--with-ro`
- **Mise Support**: Integrated with [mise](https://mise.jdx.dev) for tool management
- **Direct GPU Access (Opt-In)**: Expose NVIDIA/DRM GPU devices with `--gpu`
- **SSH Agent Forwarding**: Supports SSH commit signing through the host `ssh-agent`
- **Seccomp Sandbox Filter**: Mitigates kernel privilege escalation vulnerabilities (see [details](#seccomp-sandbox-filter))

Expand Down Expand Up @@ -50,6 +51,7 @@ opencodebox [OPTIONS] [OPENCODE_ARGS...]

- `--with /host[:/sandbox]` - Bind host path read-write to sandbox
- `--with-ro /host[:/sandbox]` - Bind host path read-only to sandbox
- `--gpu` - Expose host GPU devices for direct sandbox workloads; see [direct GPU access](#direct-gpu-access-opt-in)

These options allow you to specify additional directories to mount inside the sandbox for read-write or read-only access.

Expand All @@ -74,7 +76,7 @@ opencodebox --with /data --with-ro /config serve

## How It Works

1. Parse arguments (`--with`, `--with-ro`, `--version`, `--help`)
1. Parse arguments (`--with`, `--with-ro`, `--gpu`, `--version`, `--help`)
2. Check prerequisites (bwrap and opencode)
3. Load seccomp sandbox filter (see [details](#seccomp-sandbox-filter))
4. **Enforce security restrictions**:
Expand Down Expand Up @@ -150,6 +152,95 @@ Each tool is mounted only when `command -v <tool>` succeeds on the host. If the
- **State directory**: `~/.local/state/opencode` is mounted read-write so that locks and TUI state stay shared between the sandbox and the host. The launcher masks the host background-service registration inside the sandbox, but a sandboxed process can still create files in that directory, and the host client reads some of them (including the pty runtime registration). Treat that directory as shared state, not as a trust boundary.
- **SSH directory**: `~/.ssh` must have permissions `0700`. Fix with: `chmod 700 ~/.ssh`

## Direct GPU Access (Opt-In)

`--gpu` exposes GPU devices to programs running **directly inside opencodebox**:

```bash
opencodebox --gpu
```

When testing a checkout, use `./opencodebox` rather than an older installed copy.
Launch from your normal host environment: an existing sandbox without GPU
mounts cannot forward devices it cannot see. No mounts are added to an already
running session.

### Devices and driver dependencies

The switch discovers and mounts these existing character devices individually
using bubblewrap's `--dev-bind` (not a bind of the entire host `/dev`):

| Device family | Paths |
|---|---|
| NVIDIA GPUs | `/dev/nvidiaN` (numeric N) |
| NVIDIA auxiliary devices | `/dev/nvidiactl`, `/dev/nvidia-uvm`, `/dev/nvidia-uvm-tools`, `/dev/nvidia-modeset`, `/dev/nvidia-caps/nvidia-capN` |
| DRM GPUs (Intel/AMD/NVIDIA) | `/dev/dri/cardN`, `/dev/dri/renderDN` |
| AMD compute | `/dev/kfd` |

Note that `/dev/dri/cardN` also exposes the display/KMS interface, not just
rendering; `/dev/dri/renderDN` is the render-only node. There is no per-device
selection: both are mounted when present. NPU devices (`/dev/accel/*`) are out
of scope.

It requires at least one NVIDIA GPU or DRM card/render character device. It
skips non-character files and symlinked device nodes, rejects symlinked GPU
parent directories, and fails clearly when no GPU is found. These checks assume
trusted host `/dev`; they do not authenticate device drivers or eliminate
concurrent replacement races. Devices disappearing during launch cause a mount
failure rather than silent partial access.

GPU mode also mounts these **read-only, if present**:

- `/sys`, preserving the host sysfs topology and its symlink relationships.
- `/etc/ld.so.cache`, for host driver library discovery.
- `/etc/OpenCL/vendors`, `/etc/vulkan/icd.d`, `/etc/glvnd/egl_vendor.d`.
- `/proc/driver/nvidia`, when an NVIDIA GPU device was found.

Driver libraries and `/usr/share` vendor manifests under `/usr` are already
visible. The switch does not install CUDA, ROCm, PyTorch, or other SDKs. Libraries,
cache entries, or manifests pointing outside the exposed filesystem may need
explicit same-path `--with-ro` mounts. For example, a ROCm installation under
`/opt/rocm` may require both that path and its versioned symlink target. Direct
GPU access is not a guarantee of compatibility with every driver/SDK layout.

### Verify and troubleshoot

For NVIDIA, first ensure `nvidia-smi -L` works **on the host**. After launching
with `--gpu`, run inside the new sandbox:

```bash
nvidia-smi -L
# If a CUDA-enabled PyTorch environment is already installed:
python -c 'import torch; print(torch.cuda.is_available()); print(torch.cuda.device_count())'
```

`nvidia-smi` succeeding alone does not prove a CUDA workload works. When NVIDIA
GPU nodes exist but `/dev/nvidia-uvm` is absent, the launcher warns that CUDA may
fail. Initialize the NVIDIA/UVM driver using your distribution's normal host
setup procedure, then restart opencodebox. The launcher never loads modules,
runs `sudo`, changes device permissions, or initializes GPUs automatically.

Existing host permissions, ACLs, supplementary group access, SELinux rules, and
device-cgroup restrictions still apply. Fix access through the host's normal
GPU configuration rather than granting blanket privileges. Hotplugged or
replaced device nodes may require a sandbox restart. Explicit `--with` mounts
still take precedence and can hide or replace GPU-related mounts.

### Security Notes

> **Warning:** `--gpu` exposes all matching GPU nodes present at launch, subject
> to host access controls. It adds access to GPU driver ioctls, shared GPU
> resources, and potentially display/control interfaces—not just compute. Driver
> vulnerabilities, resource exhaustion, and host display interference are risks.
> Read-only `/sys` also exposes non-GPU host hardware/kernel information. There
> is no GPU memory quota, per-device selection, or cross-session isolation.

The local namespace, capability, and seccomp settings are unchanged; the seccomp
filter does not broadly block GPU ioctls. GPU visibility environment variables
are not security boundaries. This switch does not forward X11/Wayland sockets
or otherwise provide desktop-session access. Without `--gpu`, no GPU-specific
mounts or probes are added.

## SSH Agent and Git Signing

If `SSH_AUTH_SOCK` points to a valid socket, `opencodebox` forwards that socket into the sandbox. This allows SSH commit signing with keys already loaded by `ssh-add` on the host. This feature does not mount private SSH keys into the sandbox.
Expand Down Expand Up @@ -204,7 +295,7 @@ bash tests/seccomp.sh # seccomp integration (real bwrap + prebuilt .bpf
```

Launcher tests are black-box: `bwrap` is mocked, the argv it would receive is
asserted. GPU tests (future) plug into `tests/conftest.py` the same way.
asserted. GPU tests (`tests/test_gpu.py`) use the same `tests/conftest.py` harness.

### Seccomp BPF filter

Expand Down
88 changes: 88 additions & 0 deletions opencodebox
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Usage:
Options:
--with /host[:/sandbox] Bind host path read-write (sandbox path defaults to host path)
--with-ro /host[:/sandbox] Bind host path read-only (sandbox path defaults to host path)
--gpu Expose host GPU devices and read-only driver metadata
--version, -v Show version information
--help, -h Show this help message

Expand All @@ -39,6 +40,7 @@ Isolation:
the printed URL in the host browser).

Examples:
opencodebox --gpu
opencodebox --with ../app-backend
opencodebox --with /mnt/data:/workspace/data
opencodebox --with-ro /etc/hosts
Expand Down Expand Up @@ -163,10 +165,87 @@ sanitize_ssh_public_key_file() {
esac
}

# BEGIN GPU MOUNTS
# Internal root arguments allow fixture testing without real GPU hardware.
# The launcher always calls this with the host's standard paths. /dev must be
# trusted: type/name checks are not driver authentication or race protection.
configure_gpu_mounts() {
local dev_root="${1:-/dev}" sys_root="${2:-/sys}"
local etc_root="${3:-/etc}" proc_root="${4:-/proc}"
local device relative primary=0 nvidia=0 uvm=0 dri_dir=0 caps_dir=0
local -a mounts=()

if [[ -L "$dev_root/dri" || -L "$dev_root/nvidia-caps" ]]; then
echo "Error: --gpu refuses symlinked GPU device directories." >&2
return 1
fi

for device in "$dev_root"/nvidia[0-9]* \
"$dev_root"/nvidiactl "$dev_root"/nvidia-uvm \
"$dev_root"/nvidia-uvm-tools "$dev_root"/nvidia-modeset \
"$dev_root"/nvidia-caps/nvidia-cap[0-9]* \
"$dev_root"/dri/card[0-9]* "$dev_root"/dri/renderD[0-9]* \
"$dev_root"/kfd; do
relative="${device#"$dev_root/"}"
# Globs discover candidates; anchored expressions enforce numeric names.
if [[ ! "$relative" =~ ^nvidia[0-9]+$ &&
! "$relative" =~ ^dri/(card[0-9]+|renderD[0-9]+)$ &&
! "$relative" =~ ^nvidia-caps/nvidia-cap[0-9]+$ ]]; then
case "$relative" in
nvidiactl|nvidia-uvm|nvidia-uvm-tools|nvidia-modeset|kfd) ;;
*) continue ;;
esac
fi
# stat without -L does not follow symlinks. Fix locale for its type text.
[[ ! -L "$device" ]] || continue
[[ "$(LC_ALL=C stat -c '%F' "$device" 2>/dev/null)" == "character special file" ]] || continue

if [[ "$relative" =~ ^nvidia[0-9]+$ ]]; then
primary=$((primary + 1))
nvidia=1
elif [[ "$relative" =~ ^dri/(card[0-9]+|renderD[0-9]+)$ ]]; then
primary=$((primary + 1))
fi
[[ "$relative" == nvidia-uvm ]] && uvm=1
if [[ "$relative" == dri/* && "$dri_dir" == 0 ]]; then
mounts+=(--dir /dev/dri)
dri_dir=1
fi
if [[ "$relative" == nvidia-caps/* && "$caps_dir" == 0 ]]; then
mounts+=(--dir /dev/nvidia-caps)
caps_dir=1
fi
mounts+=(--dev-bind "$device" "/dev/$relative")
done

if [[ "$primary" == 0 ]]; then
echo "Error: --gpu found no NVIDIA or DRM GPU character devices. Initialize the GPU driver on the host and relaunch opencodebox." >&2
return 1
fi

# Preserve sysfs symlink topology and the host's driver library discovery.
# /usr (including /usr/share vendor manifests) is already read-only mounted.
mounts+=(--ro-bind-try "$sys_root" /sys)
mounts+=(--ro-bind-try "$etc_root/ld.so.cache" /etc/ld.so.cache)
mounts+=(--ro-bind-try "$etc_root/OpenCL/vendors" /etc/OpenCL/vendors)
mounts+=(--ro-bind-try "$etc_root/vulkan/icd.d" /etc/vulkan/icd.d)
mounts+=(--ro-bind-try "$etc_root/glvnd/egl_vendor.d" /etc/glvnd/egl_vendor.d)
if [[ "$nvidia" == 1 ]]; then
mounts+=(--ro-bind-try "$proc_root/driver/nvidia" /proc/driver/nvidia)
if [[ "$uvm" == 0 ]]; then
echo "Warning: NVIDIA UVM device is absent; CUDA workloads may fail. Initialize the NVIDIA/UVM driver on the host and restart the sandbox." >&2
fi
fi
BWRAP_ARGS+=("${mounts[@]}")
echo "Warning: --gpu exposes host GPU devices and read-only sysfs; GPU resources and driver access are shared with the host." >&2
}
# END GPU MOUNTS

# Parse --with and --with-ro arguments
# These allow users to bind additional paths into the sandbox
declare -a EXTRA_BINDS=()
declare -a REMAINING_ARGS=()
GPU_ENABLED=false

add_readonly_regular_file() {
local file="$1"
Expand Down Expand Up @@ -253,6 +332,10 @@ while [[ $# -gt 0 ]]; do
parse_bind_arg "--ro-bind" "$1"
shift
;;
--gpu)
GPU_ENABLED=true
shift
;;
# Pass through all other args to opencode
*)
REMAINING_ARGS+=("$1")
Expand Down Expand Up @@ -513,6 +596,11 @@ declare -a BWRAP_ARGS=(
"$OPCODE_BIND_FLAG" "$OPCODE_ROOT" /opt/opencode
)

# Device mounts must follow the private /dev; explicit user binds still come last.
if [[ "$GPU_ENABLED" == true ]]; then
configure_gpu_mounts || exit 1
fi

# Private keyring (unmount if present)
[[ -d "$HOME/.local/share/keyrings" ]] && BWRAP_ARGS+=(--tmpfs "$HOME/.local/share/keyrings")

Expand Down
6 changes: 4 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ def command(self, name: str, body: str) -> None:
)
path.chmod(0o755)

def run_launcher(self, *args: str, cwd: Path | None = None) -> subprocess.CompletedProcess:
def run_launcher(self, *args: str, cwd: Path | None = None,
script: Path | None = None) -> subprocess.CompletedProcess:
"""Run the launcher (or an adapted copy of it) as a black box."""
self.capture.unlink(missing_ok=True)
self.calls.unlink(missing_ok=True)
result = subprocess.run(
[str(self.bin / "bash"), str(LAUNCHER), *args],
[str(self.bin / "bash"), str(script if script is not None else LAUNCHER), *args],
cwd=str(cwd or self.project),
env=self.env,
capture_output=True,
Expand Down
Loading
Loading