Nimbus

Nimbus is a lightweight Zig control plane and agent for operating heterogeneous edge AI, intermediary, server, and cloud nodes. One binary provides:

• a long-running agent with stable on-disk identity;
• interval, jitter, exponential retry, and graceful POSIX shutdown;
• labels and roles for targeting glasses, drones, vehicles, desktops, and servers;
• bounded NVIDIA and Jetson accelerator discovery with opaque stable IDs;
• declarative accelerator requirements with exclusive logical reservations;
• generation-fenced accelerator execution with exact CDI/runtime handles;
• deterministic platform/accelerator artifact variants and a pinned local cache;
• deterministic edge-aware placement using bounded live telemetry and cache locality;
• a versioned desired-state and reconciliation loop;
• opt-in process, systemd, Docker, and containerd (nerdctl) runtime adapters;
• batched rollout, health gates, status history, and automatic rollback;
• SHA-256 artifact verification with optional Ed25519 signatures;
• per-node bearer credentials with reload-on-request rotation, plus a separate administrative credential;
• persistent node reports and heartbeat history in embedded SQLite;
• online/stale fleet views through the CLI and HTTP API;
• static Linux and native Windows/macOS cross-builds.

Agents and the CLI can connect to HTTPS endpoints. The embedded server listener is HTTP-only, so terminate TLS at a reverse proxy for non-local deployments. Set NIMBUS_CA_FILE to a PEM bundle when that proxy uses a private or local CA. An unauthenticated server may bind only to loopback unless the explicitly unsafe --allow-insecure-no-auth option is supplied.

Requirements

• Just 1.43 or newer
• Zig 0.16.0
• Git, ShellCheck, curl, Python, and OpenSSL for the complete validation workflow
• Docker when using the container recipes

All project operations are defined in justfile. List them with:

just

The official redistributed Zig toolchain can be installed through Python:

just bootstrap
ZIG="python -m ziglang" just doctor

Set ZIG="python -m ziglang" when invoking other recipes if zig is not on PATH. just doctor checks the local toolchain and reports optional Docker availability.

Quick start

Start the control plane:

just server

Start an agent in another terminal:

just agent

The default .nimbus-node-id file is created once and reused on later starts. Use --identity-file to place it elsewhere, or --id for an explicit node ID.

Inspect the fleet:

just nodes
just node NODE_ID

Print a report without sending it:

just inspect

Run the discovery demonstration with just demo. On Linux, run the complete process-runtime deployment, reconciliation, health, and deletion flow with:

just orchestration-demo

scripts/build-all.sh and scripts/demo.sh are thin wrappers around the corresponding recipes.

Project tasks

justfile is the canonical entry point for local development and CI:

Area	Recipes
Setup	just bootstrap, just doctor, just help
Development	just fmt, just build, just test, just check, just version
Running Nimbus	just server, just agent, just orchestrator, just inspect, just nodes, just node NODE_ID
Desired state	just deploy FILE, just deployments, just deployment NAME, just rollback NAME, just undeploy NAME
Integration	just demo, just api-check, just orchestration-demo, just integration, just run ARGS
Release	just release, just verify-static, just artifacts, just checksums
Docker	just docker-build, just docker-run, just docker-check
Source control	just git-status, just git-diff, just git-log, just pre-commit
Cleanup	just clean

Recipe parameters can override defaults. For example:

just server 0.0.0.0 9090 /var/lib/nimbus/nimbus.db node-token operator-token
just agent http://127.0.0.1:9090 server node-token
just deploy examples/deployments/process-demo.json http://127.0.0.1:9090 operator-token
just demo 19090 demo-token
IMAGE=registry.example/nimbus:dev just docker-build

NIMBUS_SERVER, NIMBUS_TOKEN, ZIG, and IMAGE are also honored where applicable. Run just --show RECIPE to inspect the exact command before use.

Configuration

YAML is the default format for human-authored node configuration. Every operational command accepts it with --config; use examples/config/agent.yaml as the starting point for deployed agents.

server: http://127.0.0.1:8080
role: smart-class
labels:
  - site=school-a
  - device=desktop
  - accelerator=jetson
node_id_file: /var/lib/nimbus/node-id
interval_seconds: 30
jitter_seconds: 5
retry_initial_seconds: 1
retry_max_seconds: 30
orchestration: true
state_dir: /var/lib/nimbus/state
runtimes: systemd,docker,containerd
artifact_public_key: HEX_ENCODED_ED25519_PUBLIC_KEY
require_artifact_signatures: true
max_artifact_bytes: 8589934592
artifact_cache_bytes: 17179869184
connectivity_quality_percent: 100
power_source: mains
power_budget_milliwatts: 30000
cost_microunits_per_hour: 0
token_file: /run/secrets/nimbus-node-token
admin_token_file: /run/secrets/nimbus-admin-token
node_token_dir: /run/secrets/nimbus-node-tokens
bind: 127.0.0.1
port: 8080
database: nimbus.db
stale_after_seconds: 90
allow_insecure_no_auth: false

The YAML reader intentionally supports this flat schema: scalar values and the labels list. It does not interpret YAML tags, anchors, or aliases, and rejects nested mappings and multiline values, avoiding implicit YAML type conversions. Quote a scalar when it must remain a string. JSON configuration files remain fully supported for automation and existing installations; both formats are converted to the same strict schema, so unknown fields and type mismatches fail.

Precedence is command-line option, environment variable, configuration file, then built-in default. Supported environment variables include:

• NIMBUS_CONFIG, NIMBUS_SERVER, NIMBUS_TOKEN, NIMBUS_TOKEN_FILE, NIMBUS_ADMIN_TOKEN, NIMBUS_ADMIN_TOKEN_FILE, NIMBUS_NODE_TOKEN_DIR, and NIMBUS_CA_FILE;
• NIMBUS_NODE_ID, NIMBUS_NODE_ID_FILE, NIMBUS_ROLE, and NIMBUS_LABELS;
• NIMBUS_INTERVAL_SECONDS, NIMBUS_JITTER_SECONDS, NIMBUS_RETRY_INITIAL_SECONDS, and NIMBUS_RETRY_MAX_SECONDS;
• NIMBUS_BIND, NIMBUS_PORT, NIMBUS_DATABASE, and NIMBUS_STALE_AFTER_SECONDS, and NIMBUS_ALLOW_INSECURE_NO_AUTH;
• NIMBUS_ORCHESTRATION, NIMBUS_RUNTIMES, NIMBUS_STATE_DIR, NIMBUS_ARTIFACT_PUBLIC_KEY, NIMBUS_REQUIRE_ARTIFACT_SIGNATURES, and NIMBUS_MAX_ARTIFACT_BYTES, and NIMBUS_ARTIFACT_CACHE_BYTES;
• NIMBUS_CONNECTIVITY_QUALITY_PERCENT, NIMBUS_POWER_SOURCE, NIMBUS_POWER_BUDGET_MILLIWATTS, and NIMBUS_COST_MICROUNITS_PER_HOUR.

Use separate configuration files for the server/operator and agent in real deployments so the administrative token is never copied to managed nodes. The packaged systemd unit reads /etc/nimbus/agent.yaml; environment variables in /etc/nimbus/nimbus.env remain available as higher-precedence overrides.

HTTP API

GET /healthz and GET /readyz are public. Other endpoints require Authorization: Bearer TOKEN when authentication is configured. --token protects agent routes; --admin-token protects operator routes and falls back to --token only in shared-token compatibility mode. For production, pass --node-token-dir: each file is named exactly after a node ID and contains only that node's bearer token. Files are read for each request, so an atomic replacement rotates a node credential without restarting the server. When this directory is enabled, the shared node token cannot authorize node routes.

POST /v1/heartbeat
GET  /v1/nodes
GET  /v1/nodes/{node_id}
GET  /v1/nodes/{node_id}/desired-state
POST /v1/nodes/{node_id}/workload-status
GET  /v1/deployments
PUT  /v1/deployments/{name}
GET  /v1/deployments/{name}
DELETE /v1/deployments/{name}
POST /v1/deployments/{name}/rollback

GET /v1/nodes accepts limit=1..500 and an optional after=NODE_ID cursor, and returns { "items": [...], "next_after": "..." | null }.

Heartbeats are schema-versioned and validated before they are written. The server accepts legacy v1 reports, v2 reports with a required accelerator inventory, v3 reports with bounded feature negotiation, v4 reports with bounded placement telemetry, and v5 reports that distinguish the binary target architecture from the runtime host architecture. Current agents emit v5. Upgrade the server before agents during a rolling deployment. CPU-only discovery is distinct from a failed or unavailable probe. SQLite always updates current node state, samples heartbeat history at most every five minutes per node, retains it for seven days, and retains audit events for 30 days. Enrollment is audited once instead of auditing every accepted heartbeat. The list and inspect endpoints calculate online or stale from the server's receipt time and --stale-after threshold. Desired state, assignments, rollout progress, and workload status history are persisted in the same database.

Workload orchestration

Apply and inspect a deployment:

nimbus deployments apply examples/deployments/process-demo.json \
  --server http://127.0.0.1:8080 --token "$NIMBUS_ADMIN_TOKEN"
nimbus deployments list --server http://127.0.0.1:8080
nimbus deployments inspect process-demo --server http://127.0.0.1:8080

Enable only the runtimes a node is trusted to execute:

nimbus agent run --orchestrate --runtimes systemd,docker,containerd \
  --label site=school-a --label device=edge-server \
  --state-dir /var/lib/nimbus/state

Runtime adapters are deliberately allowlisted per agent:

Runtime	Desired-state reference	Notes
process	Absolute argv or verified {artifact}	Linux bootstrap workloads; no shell expansion
systemd	Existing unit name	Uses systemctl; preferred for host processes
docker	Image pinned by @sha256:	Creates nimbus-NAME with Docker restart policy
containerd	Image pinned by @sha256:	Uses nerdctl in the nimbus namespace

Targets may use node IDs, roles, all, or an AND set of labels. Rollouts have a deterministic node order, bounded batch size, health-gated waves, an optional pause, and an unavailable threshold. A failed wave automatically restores the previous revision when auto_rollback is enabled. Deleting a deployment causes agents to stop it on their next reconciliation.

Deployments may also declare accelerator count, kind, vendor, memory, and capability requirements. Compatible Linux agents negotiate the fenced A3 lifecycle, receive exclusive generation-scoped claims, and inject only exact CDI devices or vendor-verified host allowlists. Claims remain held through stop, rollback, crash recovery, and the final release acknowledgement. NVIDIA container execution requires the exact device to be present in the local CDI catalog; Nimbus never falls back to all or broad host-device access.

An optional placement policy selects a replica count from the explicitly targeted pool using liveness, connectivity, power, cost, accelerator headroom, temperature, and verified artifact-cache locality. Decisions and reason codes are persisted and remain sticky across polling and server restarts. Nimbus does not automatically move an already selected singleton merely because its node goes offline; safe cross-node failover requires the future signed-lease policy.

See Workload orchestration for the schema, runtime behavior, security controls, and production limitations.

Cross-compile

just release
just verify-static
just artifacts
just checksums

Artifacts are written to:

zig-out/releases/linux-x86_64/nimbus
zig-out/releases/linux-aarch64/nimbus
zig-out/releases/windows-x86_64/nimbus.exe
zig-out/releases/macos-x86_64/nimbus
zig-out/releases/macos-aarch64/nimbus

SQLite is compiled from the vendored public-domain amalgamation. Linux release binaries use musl and remain statically linked.

Deployment

The Docker image runs nimbus server and expects /data to be writable:

just docker-build
just docker-run

just docker-check builds the image, starts a disposable container, checks /healthz, and verifies graceful shutdown.

The long-running systemd unit is at deploy/systemd/nimbus-agent.service. Put secrets such as NIMBUS_TOKEN in /etc/nimbus/nimbus.env rather than directly in the unit.

Documentation

The documentation index links to the architecture, workload orchestration, and development guides. The root README focuses on setup and operation; the documents describe internal design and contributor workflows.