Skip to content

fix(guest): bind GPU attestation to measured app policy#789

Open
kvinwang wants to merge 32 commits into
masterfrom
hapi-gpu-verify
Open

fix(guest): bind GPU attestation to measured app policy#789
kvinwang wants to merge 32 commits into
masterfrom
hapi-gpu-verify

Conversation

@kvinwang

@kvinwang kvinwang commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

What this changes

When a VM has an NVIDIA GPU attached, dstack now verifies the GPU once during guest boot, before app keys are provisioned and before the workload starts.

  • If verification succeeds, the GPU is marked ready and boot continues.
  • If verification or the configured policy fails, boot stops and the app does not receive its keys.
  • A VM without a GPU continues to boot normally.

How to use it

GPU attestation is enabled by default, so no app configuration is required for the default production policy. An app can customize it in app-compose.json:

{
  "manifest_version": "3",
  "requirements": {
    "gpu_policy": {
      "attest_gpu": true,
      "allow_devtools": false,
      "allow_debug": false,
      "allow_insecure_boot": false
    }
  }
}

An optional Rego v0 policy can be supplied in gpu_policy.rego; it receives NVIDIA's complete claims array as input. A minimal app-compose.json containing one looks like this:

{
  "manifest_version": "3",
  "name": "gpu-app",
  "runner": "docker-compose",
  "requirements": {
    "gpu_policy": {
      "rego": "<rego-v0-policy>"
    }
  }
}

Replace the placeholder with the policy source. For example, this policy requires exactly one H100:

package policy
default nv_match = false

nv_match {
  count(input) == 1
  input[0].hwmodel == "GH100 A01 GSP BROM"
}

The required entrypoint is data.policy.nv_match. Setting attest_gpu to false explicitly skips GPU attestation and is not recommended for production.

After boot, the app can read the attestation result through the guest-agent API. This returns the result produced during boot; it does not run a new attestation:

curl --unix-socket /var/run/dstack.sock http://dstack/GpuInfo
{
  "attestation": "{\"result_code\":0,\"claims\":[...]}"
}

SDK methods are GpuInfo() in Go, gpuInfo() in JavaScript, and gpu_info() in Python and Rust. The attestation field is empty when no boot-time GPU attestation result is available.

When and how attestation runs

The check runs after the app compose hash is measured, but before the GPU ready state, app key provisioning, and workload startup:

measure app compose -> inventory GPUs -> run nvattest -> apply GPU policy
-> mark GPU ready -> record gpu-attestation event -> provision app keys

dstack runs NVIDIA nvattest with a fresh random nonce and local verification:

nvattest attest --device gpu --verifier local --nonce <fresh-nonce> --format json

It requires every attached display GPU to be NVIDIA, and requires the GPU counts reported by PCI inventory, nvattest, and NVML to match. Each GPU must pass NVIDIA's appraisal, nonce validation, confidential-compute checks, and the app's optional policy. DevTools, debug mode, and insecure GPU boot are rejected by default.

How to verify the result

The configured policy digest is SHA-256(JCS(requirements.gpu_policy)), using {} when the policy is omitted. It can be verified on every supported platform:

  • TDX: gpu-policy-hash is measured into RTMR3. A successful GPU launch produces compose-hash -> gpu-policy-hash -> gpu-attestation -> instance-id -> boot-mr-done.
  • AWS NitroTPM: the same events are measured into non-resettable SHA384 PCR14 and verified by replaying PCR14 against the signed NitroTPM Attestation Document.
  • AMD SEV-SNP: MrConfigV3.gpu_policy_hash, when present, is bound by the signed report's HOST_DATA. AMD SEV-SNP can therefore verify the policy hash, but its current evidence cannot bind the later gpu-attestation event because there is no quote-bound runtime register.

The gpu-attestation event contains the verified device count, CC/DevTools state, and evidence_sha256 = SHA-256(complete nvattest JSON). To authenticate GpuInfo.attestation on TDX or AWS:

  1. Verify the quote/Attestation Document and replay RTMR3/PCR14.
  2. Require exactly one pre-system-ready gpu-attestation event.
  3. Hash the exact UTF-8 bytes of GpuInfo.attestation and compare them with the event's evidence_sha256.
  4. Only then inspect the returned NVIDIA claims.

See docs/security/security-model.md for the full event schema and platform-specific limitations.

Related to #778.

Copilot AI review requested due to automatic review settings July 16, 2026 12:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens the guest-side NVIDIA GPU attestation gate by binding GPU appraisal and an optional application-provided GPU policy into the measured launch event chain before key provisioning, and by enforcing CC mode while rejecting DevTools by default.

Changes:

  • Introduces requirements.gpu_policy (strict/deny-unknown-fields) with optional Rego v0 (data.policy.nv_match) and allow_devtools (default false), and measures its JCS-canonical SHA-256 digest as a gpu-policy event.
  • Tightens GPU attestation semantics by reconciling sysfs PCI inventory, vm_config.num_gpus, and nvattest claim counts; saves full nvattest JSON output and measures a versioned gpu-attestation event (including digest).
  • Enforces CC feature state ON and DevTools mode OFF (unless explicitly allowed by measured policy) before setting the GPU “ready” state.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
os/yocto/layers/meta-nvidia/recipes-graphics/nvidia/files/nvidia-persistenced.service Updates unit comments to reflect the new attestation/policy gating and attest_gpu flag.
os/yocto/layers/meta-nvidia/recipes-graphics/nvattest/nvattest_2026.06.09.bb Updates nvattest recipe description and installs ordering drop-in.
os/yocto/layers/meta-nvidia/recipes-graphics/nvattest/files/10-nvidia-gpu-ordering.conf Updates boot ordering documentation to reference requirements.attest_gpu.
dstack/dstack-util/src/system_setup.rs Implements GPU inventory binding, nvattest JSON capture/digesting, CC/DevTools enforcement, and GPU policy measurement/evaluation in the launch chain.
dstack/dstack-util/Cargo.toml Adds serde_jcs and regorus dependencies for policy digesting and Rego evaluation.
dstack/dstack-types/src/lib.rs Adds GpuPolicy and wires it into Requirements with strict schema/defaults.
dstack/Cargo.toml Adds the regorus workspace dependency.
dstack/Cargo.lock Locks new transitive dependencies for regorus/serde_jcs.
docs/security/security-model.md Documents the new GPU policy and measured-event binding model, including verifier expectations and SNP limitations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dstack/dstack-util/src/system_setup.rs Outdated
Comment thread dstack/dstack-util/src/system_setup.rs
kvinwang added 4 commits July 18, 2026 05:12
Upgrade regorus to 0.10.1 and set an execution timer so a runaway
application policy cannot hang boot indefinitely. nvattest already runs
under a timeout; the Rego evaluation was the remaining unbounded step.
The engine needs the arc feature when built without default features.
GpuInfo no longer fails when no boot-time attestation output exists
(no GPU attached or attestation disabled); it returns an empty
attestation string instead, so callers do not need to treat the
GPU-less case as an error. Unexpected read failures are logged and
also reported as empty.
Clippy's -D clippy::expect_used denies expect() on Option, which
was breaking rust-checks CI. Use unwrap_or(NonZeroU32::MIN) instead
since the literal 1024 is always non-zero.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants