Advanced

GPUDirect RDMA and GPUDirect Storage

By default a byte leaving a GPU for the network or the disk makes a detour through host memory, crossing PCIe twice and costing a CPU copy. GPUDirect RDMA lets the NIC read and write GPU memory directly, and GPUDirect Storage does the same for NVMe. The win is not raw bandwidth (PCIe is the ceiling either way) but the halving of PCIe traffic and the removal of the host as a bottleneck, which is what makes collectives run at NIC rate and checkpoints run at drive rate. When it is silently off, everything still works, at half speed.

Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew

RELATED CONCEPTS

RDMA, InfiniBand and RoCEv2 NVLink, NVSwitch and PCIe Checkpoint I/O Data Loading Pipelines for Training NCCL and Collective Algorithms Debugging a Slow All-Reduce

LESSONS THAT TEACH THIS

Fleet and ReliabilityStorage is a throughput problem until the files get small14 min→

PRACTICE THIS IN REAL QUESTIONS

Networking, Interconnects & StorageWhen does GPUDirect Storage actually help, and when is it just a more complicated read?→GPU & Accelerator ArchitectureExplain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?→Networking, Interconnects & StorageWhich NCCL environment variables are worth knowing, what does each change, and which should you almost never set?→Networking, Interconnects & StorageHow would you know GPUDirect RDMA is not being used, given that nothing reports an error when it is disabled?→Networking, Interconnects & StorageDesign the checkpoint write path for a 405B model on 16,384 GPUs. What has to be true for the pause to stay under a minute?→Hardware, Cabling & Cluster Build-OutAfter a maintenance window, two-node bandwidth halved. Single-node is unchanged. Diagnose it.→