Confidential GPU Inference: Encryption, Attestation and Air-Gapped Deployment

Confidential GPU inference protects sensitive execution from specified actors, such as a host operator, by placing the workload inside a supported CPU/GPU trust boundary. Prompts and weights still become plaintext there. Attestation supplies evidence about the environment; a release policy decides whether it may receive protected keys. An air gap adds a separate network constraint and requires a plan for dependencies that normally refresh online.

These distinctions matter in architecture reviews and interviews. “The data is encrypted” leaves several unanswered questions: encrypted where, decrypted by whom, and authorized using which evidence? This guide gives you the starting model. The linked Premium concepts, questions and course module develop the implementation and troubleshooting exercises.

Start where the prompt decrypts

A customer sends a prompt over HTTPS to a gateway. If the gateway terminates TLS on a host the operator controls, it can read the prompt before any GPU executes. Placing a confidential accelerator behind it does not undo that exposure.

For the intended design, the customer accepts an endpoint bound to the approved workload. The outer gateway can route an encrypted payload, while decryption happens inside the protected guest. Tokenization, retrieval, cached attention state and response handling must follow the same data-flow contract.

Three questions before choosing a confidential platform

1. Where does the request become plaintext?

   Place the protected endpoint inside the intended boundary.

2. What authorizes that workload to receive keys?

   Verify fresh evidence, workload policy and recipient binding.

3. What can carry plaintext back out?

   Review responses, tools, logs, caches and failure handling.

The Confidential Inference concept follows these paths in detail. It also explains a limitation that hardware cannot remove: approved application code sees plaintext and can disclose it through an authorized output or a bug.

Know what each security term promises

These terms describe different mechanisms and can be combined. A private network is not evidence that its servers execute confidentially. A TEE does not automatically give the outputs a differential-privacy guarantee. See NIST's privacy-enhancing cryptography overview and differential-privacy guidance for those separate guarantees. The Confidential Computing Consortium's comparison with homomorphic encryption explains protected plaintext execution versus computation on ciphertext.

Do confidential GPUs encrypt HBM and every link?

Avoid that blanket claim. NVIDIA's documented threat model describes plaintext in GPU memory protected by hardware access restrictions, alongside protected CPU/GPU transfers. The details depend on the supported configuration.

Multi-GPU support needs particular care. Hopper Protected PCIe and supported Blackwell multi-GPU configurations have different peer-fabric protection. An encrypted host transfer does not establish that a separate GPU-to-GPU path is encrypted. Cloud products can expose a narrower configuration than the underlying hardware supports.

For a two-GPU sketch, draw three logical edges: CPU to GPU A, CPU to GPU B, and GPU A to GPU B. Checking two edges leaves a potentially sensitive third path. The GPU trust-boundary question develops that reasoning and its physical assumptions.

Attestation must connect to a release decision

A signed device report is evidence, not an instruction to release every secret. A verifier checks authenticity and acceptable state. The resource owner then applies a policy for the workload, tenant and requested key. Freshness prevents old evidence from being reused outside its accepted window; recipient binding prevents valid evidence from being relayed to obtain a secret on an unapproved connection.

GPU firmware evidence also does not identify the entire serving application. The approved guest needs to enforce what code and configuration can run, including mutable mounts and downloaded extensions. Remote Attestation and Key Release separates these decisions and explains why decoding a token is not signature verification.

The Kubernetes configuration question maps the process to CPU TEE setup, device passthrough, runtime selection and guest policy. A node label or a running pod selects and reports operational behavior; neither is a substitute for accepted evidence.

An air gap creates a freshness problem

A local verifier may still call an online service for reference measurements or certificate status. NVIDIA states those dependencies in its local-verifier integration. Disconnected operation therefore needs supported provisioning and refresh procedures, not simply a copy of the verifier executable.

Take an illustrative bundle whose earliest required admission input remains usable for 72 hours. With 24 hours reserved for recovery, refresh must start by hour 48. A site that imports material every seven days cannot promise replacement admission throughout that interval using this bundle. Seven days plus the reserve requires 192 hours of usable horizon. That requirement cannot extend a certificate's validity.

The air-gapped deployment concept covers artifact provenance, local identity, time, storage and recovery. Its key acceptance test is a fresh replacement with external routes unavailable. A warm process can conceal missing downloads and resident keys.

Learn it through an acceptance runbook

The new Premium module in Fleet and Reliability produces three artifacts: a plaintext-boundary worksheet, a policy decision table, and a disconnected deployment runbook. The CPU-only policy lab tests one accepted fixture and 15 denied variants. It does not claim to perform hardware attestation.

Continue with failed-attestation diagnosis and confidential-serving performance measurement. A useful design can explain both why a launch is denied and how its accepted configuration meets the service objective. Keep source-reviewed configuration, simulated results and hardware measurements visibly distinct.