Design the fabric for 10,000 GPUs. Two tiers or three, and what does the choice cost?

Count endpoints, allocate every switch port once, and compare a common fabric with independent rails before adding a third tier.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Count endpoints, allocate every switch port once, and compare a common fabric with independent rails before adding a third tier.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🔌 Networking & Storage🔒 Premium

Rail-Optimized and Fat-Tree FabricsRail-optimized and Clos fabrics organize traffic between GPU nodes. Port counts, placement and oversubscription determine which paths are available to collectives.

Foundational

🖧 Hardware & Cluster Build-Out

Scale-Out Fabric Choice: InfiniBand XDR vs Spectrum-XInfiniBand and Spectrum-X connect GPUs across nodes. Compare congestion behavior, topology and operational requirements as well as port speed.

Foundational

🧮 Open Weights & Serving Engines

Multi-Node Serving TopologiesChoose what to distribute: complete replicas, tensor operations, layers, experts or request phases. Compare rank-local fit, communication dependencies, workload concurrency and failure domains. A node boundary is not a universal performance boundary, and multiple layouts can be combined.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackFor 64 to 72 GPUs, is one GB300 NVL72 rack better than eight HGX B300 nodes?Next in this trackBuy hardware, colocate, or rent from a GPU cloud? Work the decision for a 512-GPU need.Next in this trackYou have exactly 1 MW of facility capacity. Maximize training throughput.

FEDITOR'S NOTE

Look for an explicit port budget, defensible topology assumptions and a workload-based choice of oversubscription.

DISCUSSION · 0

No comments yet — be the first to share your approach.