Run a Qwen model on a DGX Spark and connect to it from macOS with the native
(no-Docker) OpenCode client (opencode_native/, a
terminal coding agent with a web UI and HTTP API). It drives the vLLM endpoint
over an SSH tunnel, so nothing on the DGX side changes.
Push notifications of agent activity to your phone via
ntfy are included (ntfy/ + the stack's
README → Notifications).
The DGX side currently offers one serving option, on port 8000:
qwen38-flash-next-fp4— NVIDIA's Qwen3.8-Flash-Next-FP4-Local quantization of the 125B MoEQwen3.8-Flash-Next.
See dgx_spark_host/ → The stack.
Host a Qwen model via vLLM on the DGX Spark machine — one self-contained stack per model, in its own folder (own image, compose, entrypoint and defaults; stacks share no code or parameters). All bind port 8000; run only ONE at a time.
| Stack | Serves | How it's run | Served alias |
|---|---|---|---|
qwen38-flash-next-fp4/ |
nvidia/Qwen3.8-Flash-Next-FP4-Local (QAD NVFP4 of the 125B MoE, ~96 GiB) |
cd dgx_spark_host/qwen38-flash-next-fp4 && docker compose up --build |
qwen-local |
Run it:
cd dgx_spark_host/qwen38-flash-next-fp4
docker compose -f compose.yml up --buildThe API is available at http://localhost:8000/v1. For shared networks, bind
to 127.0.0.1 in the stack's compose.yml.
Each stack's README.md documents its own environment variables (defaults live
in its Dockerfile; override via .env or compose.yml) and tuning
notes (DGX Spark, GB10, 128 GB unified memory, LLM-only box). The checkpoint is
a native vision-language model and ships an MTP head, so MTP speculative
decoding is on by default. The stack follows NVIDIA's own serve command, which
leaves the --limit-mm-per-prompt image cap out.
- What it is:
nvidia/Qwen3.8-Flash-Next-FP4-Local— NVIDIA Model Optimizer's QAD (quantization-aware distillation) NVFP4/MXFP8 build ofQwen/Qwen3.8-Flash-Next, a 125B multimodal MoE with 6B active parameters plus a 51B n-gram (PLE) embedding table and a 4B MTP module. ~96 GiB on disk; NVIDIA validated it on the DGX Spark. - Why fully isolated: it needs a different vLLM image (the community B12X
container NVIDIA prescribes, not
vllm/vllm-openai:nightly), a different loader/quantization path (--load-format b12x --quantization modelopt_mixed) and its ownVLLM_*/B12X_*environment — so it is a self-contained folder with its own defaults rather than a shared stack with presets. It is served on port 8000 under the aliasqwen-local— clients need no changes. - Requirements: accept the NVIDIA Open Model Agreement on the model page
and put
HF_TOKENin this stack's.env; ~96 GiB of free disk for the weights. The first start also spends a long while on weight load + compile. - Run:
cd dgx_spark_host/qwen38-flash-next-fp4 && docker compose -f compose.yml up --build.
Full details, provenance, and the overridable parameters are in
dgx_spark_host/qwen38-flash-next-fp4/README.md.
Run OpenCode natively as the macOS-side client: a terminal coding agent that ships as a single self-contained binary (no Docker, and no Node.js needed), with a TUI, a bundled web UI, and a headless HTTP API. Agents run as your user on the local filesystem — there is no container sandbox, so the edit/shell approval prompt stays on by default.
./opencode_native/install.sh # one-time (upgrades on re-run)
./opencode_native/run.sh # TUI in the current directory
./opencode_native/run.sh /path/to/repo # TUI on a project
./opencode_native/run.sh server # headless API + web UI
./opencode_native/run.sh web # + published on the tailnet
./opencode_native/run.sh attach [dir] # TUI attached to that server- Model: OpenCode has no UI for adding a provider, so
install.shgeneratesopencode-state/config/opencode.jsonfromopencode.example.jsonc— providerdgx-vllmagainsthttp://localhost:8000/v1, modeldgx-vllm/qwen-local. The endpoint and key are substituted fromVLLM_BASE_URL/VLLM_API_KEYat launch (no secret in the file), which is why you always start throughrun.sh. - Server mode listens on
http://localhost:4096(loopback only; basic authopencode/OPENCODE_SERVER_PASSWORD, generated byinstall.sh—run.shrefuses to bind a shell-capable port without it) and is whatattach, the web UI, and the notifier use. Reach it remotely with an SSH tunnel (ssh -L 4096:localhost:4096 host) rather than a wider bind. - All state — config copy, sessions (
opencode.db), snapshots, logs — stays insideopencode_native/opencode-state/viaXDG_*isolation, so this stack never touches~/.config/opencodeor~/.local/share/opencodeand can't collide with a personal OpenCode install. - Optional ntfy push notifications: with
NTFY_ENABLED=trueinopencode_native/.env,run.sh serveralso startsntfy_notifier.py(and the per-PC ntfy server fromntfy/) that pings you when a session finishes, waits for your approval, or errors — with tap-to-open deep links into the web UI. Seeopencode_native/README.md→ Notifications (ntfy).
| Variable | Default | Description |
|---|---|---|
VLLM_BASE_URL |
http://localhost:8000/v1 |
Endpoint run.sh preflights and the config substitutes |
VLLM_API_KEY |
local-dgx-key |
API key for the preflight check and the provider |
VLLM_MODEL_ID |
qwen-local |
Model id vLLM serves (the alias the DGX stack serves) |
OPENCODE_SERVER_PORT |
4096 |
Server-mode port (API + web UI) |
OPENCODE_SERVER_PASSWORD |
(generated) | Basic-auth password for server/attach |
OPENCODE_AUTO |
no |
yes = auto-approve edit/shell (equivalent of --auto) |
OPENCODE_STATE_DIR |
./opencode-state |
Isolated config/XDG state, sessions, logs |
Needs curl + git, the opencode binary (install.sh installs it), and —
only for notifications — any python3. Node/npm are needed only if you add
OpenCode plugins. Ports, permissions, deep links, and troubleshooting are
documented in opencode_native/README.md.
Per-PC ntfy push server (single Docker container, port
2020). Pairs with the client's notifier
(opencode_native/ntfy_notifier.py) to send push notifications to your
phone when an agent finishes a turn, needs input, or hits an error. Designed
for multiple PCs on Tailscale, each running its own server + notifier:
the notifier publishes to 127.0.0.1:2020, the phone subscribes over
Tailscale at http://<pc>.tail:2020/<topic>.
cd ntfy
cp example.env .env # set NTFY_BASE_URL=http://<this-pc>.tail:2020
docker compose up -d # healthcheck via docker compose psSecurity model (unguessable topic = credential on Tailscale), optional
account auth, phone setup (Android instant delivery / iOS relay) and the
Firebase/custom-APK caveat are documented in
ntfy/README.md. Enable end-to-end notifications via
NTFY_ENABLED=true in opencode_native/.env (see
opencode_native/README.md → Notifications (ntfy)).