← Hardware, Cabling & Cluster Build-Out / 27

mediumNewCrusoeCoreWeaveNVIDIA

Walk me through cabling a rail-optimized cluster. What goes where, and what goes wrong?

The rule is one sentence and the execution is thousands of connections where a single transposition creates a fault that is invisible until a collective runs. The mapping, the labelling scheme that makes errors findable, and the verification that has to happen before anyone accepts the cluster.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

The rule is one sentence and the execution is thousands of connections where a single transposition creates a fault that is invisible until a collective runs. The mapping, the labelling scheme that makes errors findable, and the verification that has to happen before anyone accepts the cluster.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

🖧 Hardware & Cluster Build-Out

Scale-Out Fabric Choice: InfiniBand XDR vs Spectrum-XOutside the NVLink domain every GPU talks over a scale-out fabric, and as of September 2026 NVIDIA sells two at the same 800 Gb/s per port: Quantum-X800 InfiniBand and Spectrum-X Ethernet. They differ in congestion handling, operational familiarity and what happens when something misbehaves rather than in headline speed. The switch radix decides how many endpoints a two-tier fabric reaches, and that single number drives the switch count, the cable count and a large part of the budget.

🔌 Networking & Storage🔒 Premium

Rail-Optimized and Fat-Tree FabricsA GPU cluster's network is built from two ideas: a fat tree (Clos) that gives every node a path to every other node with a chosen amount of oversubscription, and rail optimization, which wires GPU i of every node to the same leaf switch so the collectives that dominate training stay one hop away. Sizing one is arithmetic on port counts, and the interview question is usually that arithmetic: how many switches, what oversubscription, and where the NVLink domain ends and the fabric begins.

🖧 Hardware & Cluster Build-Out

Cables, Transceivers and the Optics Power BudgetCable choice is set by distance and it is the most common ordering mistake in a GPU cluster build. Passive copper reached 3 m at 400G and tops out near 2 m at 800G, so a bill of materials copied from the previous generation produces links that will not come up. Beyond copper come active copper, then active optical cables, then transceivers and fiber. Each step adds reach and adds power, and at cluster scale the transceivers alone draw tens of kilowatts that nobody budgeted.

🧮 Open Weights & Serving Engines

Model Onboarding: From Hugging Face to ProductionA new open-weights model lands and someone asks how long until it is serving traffic. The answer depends on a sequence that is the same every time: read the card and the config, check engine support for the exact attention and quantization combination, size it, pull the weights, bring up one replica, validate correctness against the authors' own outputs, benchmark, then roll out behind a flag. The steps that surprise people are the download, which is hours for a trillion-parameter model, and the correctness check, which almost nobody does and which catches the wrong template.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackThe cluster passed every acceptance test and production throughput is 20 percent below plan. Where do you look?Next in this trackRoll out a driver and firmware upgrade across a live 2,048-GPU fleet without losing a training run.Next in this trackA new GPU generation is arriving. Plan the migration of a live 2,048-GPU cluster.

FEDITOR'S NOTE

Scored on stating the rail mapping precisely, on a labelling and verification scheme, and on the specific failure a transposition produces.

DISCUSSION · 0

No comments yet — be the first to share your approach.