Core

RDMA, InfiniBand and RoCEv2

RDMA lets network adapters transfer data directly between registered memory regions. InfiniBand and RoCE provide different network and operating environments for those transfers.

This concept is included with a free account. Sign in to continue reading.

free account · no card needed

RELATED CONCEPTS

NCCL and Collective Algorithms Rail-Optimized and Fat-Tree Fabrics Congestion Control for AI Fabrics GPUDirect RDMA and GPUDirect Storage NVLink, NVSwitch and PCIe Debugging a Slow All-Reduce

LESSONS THAT TEACH THIS

Fleet and ReliabilitySeparate network roles, then test the path behind the symptom14 min→

PRACTICE THIS IN REAL QUESTIONS

Networking, Interconnects & StorageInfiniBand or RoCE version 2 for a new GPU training cluster. Make the call and say what would reverse it.→GPU & Accelerator ArchitectureExplain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?→Hardware, Cabling & Cluster Build-OutA vendor offers 15 percent off if you take their Ethernet fabric instead of InfiniBand. Decide.→Networking, Interconnects & StorageWhat is incast, why does it break an RDMA fabric, and what do ECN and priority flow control each do about it?→Networking, Interconnects & StorageWhat is RDMA, and when does it help a training cluster more than TCP?→Hardware, Cabling & Cluster Build-OutAfter a maintenance window, two-node bandwidth halved. Single-node is unchanged. Diagnose it.→