Advanced

Rail-Optimized and Fat-Tree Fabrics

Rail-optimized and Clos fabrics organize traffic between GPU nodes. Port counts, placement and oversubscription determine which paths are available to collectives.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

RDMA, InfiniBand and RoCEv2 NVLink, NVSwitch and PCIe Topology-Aware Communication NCCL and Collective Algorithms Training Cluster Design at 10k GPUs Congestion Control for AI Fabrics

LESSONS THAT TEACH THIS

Fleet and ReliabilitySeparate network roles, then test the path behind the symptom14 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureDesign a non-blocking network fabric for 512 H100s. How many switches, how are they wired, and where does the NVLink domain end?→Networking, Interconnects & StorageDesign the fabric for 4,096 GPUs with 64-port switches. What does 1:1 cost, and what does 2:1 give up?→Networking, Interconnects & StorageExplain a rail-optimized fabric. What does it buy over a plain fat tree, and how many switches does one need?→Hardware, Cabling & Cluster Build-OutDesign the fabric for 10,000 GPUs. Two tiers or three, and what does the choice cost?→Hardware, Cabling & Cluster Build-OutDuring acceptance, seven rails hit expected bandwidth and one is 15 percent low. Find it.→Hardware, Cabling & Cluster Build-OutWalk me through cabling a rail-optimized cluster. What goes where, and what goes wrong?→