Design the complete network for a 1,024-GPU cluster: compute fabric, storage, management. What are the counts and where does the money go?

Three networks, not one, and the mistake that hurts most is putting two of them on the same wires. The port and switch counts for each, the cable plan, the split of the bill between switches and optics, and the two decisions that are hard to change once the cluster is racked.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🖧 Hardware & Cluster Build-Out

The Bill of Materials for a Training ClusterA GPU cluster is not a pile of GPUs. A 512-GPU scalable unit built to NVIDIA's DGX SuperPOD B300 reference architecture needs 64 nodes, four separate networks, thousands of transceivers, storage that can absorb a checkpoint burst, a management plane, racks, power distribution and cooling equipment. Writing the list out in order is how a design becomes a purchase order, and the items people forget are the ones that hold up a deployment for weeks.

Advanced

📐 AI Systems Design🔒 Premium

Training Cluster Design at 10k GPUsDesign a cluster for training frontier models is the prompt that tests whether a candidate can hold hardware, network, storage, scheduling and reliability in one head at once. The answer is a bill of materials with a reason for every line: how many GPUs and why, how they are grouped into pods, how the fabric connects the pods and what it costs a collective to cross one, how much storage bandwidth the checkpoints and the data loader need, how power and cooling bound the whole thing, and how the failure statistics set the spare pool and the checkpoint cadence. This page derives each line for a 10,240-GPU cluster.

Advanced

🔌 Networking & Storage🔒 Premium

Rail-Optimized and Fat-Tree FabricsA GPU cluster's network is built from two ideas: a fat tree (Clos) that gives every node a path to every other node with a chosen amount of oversubscription, and rail optimization, which wires GPU i of every node to the same leaf switch so the collectives that dominate training stay one hop away. Sizing one is arithmetic on port counts, and the interview question is usually that arithmetic: how many switches, what oversubscription, and where the NVLink domain ends and the fabric begins.

Foundational

🖧 Hardware & Cluster Build-Out

Cables, Transceivers and the Optics Power BudgetCable choice is set by distance and it is the most common ordering mistake in a GPU cluster build. Passive copper reached 3 m at 400G and tops out near 2 m at 800G, so a bill of materials copied from the previous generation produces links that will not come up. Beyond copper come active copper, then active optical cables, then transceivers and fiber. Each step adds reach and adds power, and at cluster scale the transceivers alone draw tens of kilowatts that nobody budgeted.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDoes multi-node inference need the same fabric as training, or can it run on ordinary networking?Next in this trackOne fabric link goes down and comes back once an hour. What does that do to a 1,024-GPU training job, and how would you find it?Next in this trackWhy does object storage behave badly for random reads, and what does a training pipeline do about it?

FEDITOR'S NOTE

Scored on separating the three networks and saying why, on the port and optics counts, and on identifying which decisions are irreversible after installation.

DISCUSSION · 0

No comments yet — be the first to share your approach.