hardNewCoreWeaveNebiusLambda

Design the control plane for a GPU cloud: tenancy, provisioning, health, billing, and the API a customer sees.

A GPU cloud sells whole nodes and fabric slices to tenants who expect them in minutes and expect them to work. The node state machine, the reconciler that keeps thousands converging, the isolation lines on shared fabric, the health loop that pulls a bad node first, and billing that survives a control-plane outage.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

📐 AI Systems Design

Control Plane and API Design for GPU PlatformsEvery GPU platform has a control plane, and its API is what the rest of the organization experiences as the platform. Three semantics decide whether it survives contact with a network: idempotent creation so a retried request does not launch a second job on sixty-four GPUs, cancellation modelled as intent because only the node agent can stop a running process, and cursor pagination that does not skip rows when work is created during a listing.

Foundational

🧮 Open Weights & Serving Engines

Capacity Planning for Open-Weights FleetsPlanning a fleet for a sparse open-weights model works differently from planning one for a dense model, because memory follows total parameters and throughput follows active parameters, and those now differ by more than twenty times. The sizing goes in one direction only: from a traffic forecast to tokens per second, to replicas at a measured operating point, to GPUs, to racks and kilowatts. Doing it in the other direction, from an available GPU count, produces a fleet that fits the hardware rather than the demand.

Foundational

🖧 Hardware & Cluster Build-Out

Cluster Bring-Up: Firmware, Drivers and the StackBring-up is an ordered dependency chain and skipping a step produces a symptom that points somewhere else. Firmware first, then the operating system and kernel, then the GPU driver, then the fabric manager, then the network stack, then GPUDirect, then CUDA and NCCL, then the container and scheduling layer. On Blackwell HGX systems the fabric manager reaches the NVSwitches through a bridge device and therefore depends on the InfiniBand stack being present, which is a dependency that surprises almost everyone the first time.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

Incident Response for GPU FleetsAn incident on a GPU fleet is a training run that stopped, a serving endpoint burning its error budget, or a fleet-wide symptom nobody has explained yet. The response has a shape: detect, stabilize, diagnose, repair, return through the gate, write it up. The stabilizing move (drain the node, restart from checkpoint, or shift traffic) comes before the diagnosis, because a frontier run loses more per minute than any investigation is worth. This page gives the triage order, the 3am decision tree, the spare-capacity arithmetic behind drain-and-replace, and what a fleet postmortem has to contain.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign an LLM service for a 200 ms time-to-first-token SLO at p99. Decompose the budget and say what you would give up.Next in this trackDesign the runtime for long-running agents: sessions, tool sandboxes, KV that lives for hours, and checkpoints.Next in this trackDesign serving for a video generation model: diffusion steps, batching, memory, and a latency profile unlike an LLM's.

FEDITOR'S NOTE

Scored on a node state machine with a reconciler, on tenant isolation across compute, fabric and storage, on health as a continuous loop that returns nodes to a pool only after checks, and on billing built from an append-only ledger independent of the control plane's availability.

DISCUSSION · 0

No comments yet — be the first to share your approach.