Advanced

GPUDirect RDMA and GPUDirect Storage

GPUDirect enables supported devices to transfer data to or from GPU memory without a host staging buffer. Its benefit depends on topology, software support and the transfer path.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

RDMA, InfiniBand and RoCEv2 NVLink, NVSwitch and PCIe Checkpoint I/O Data Loading Pipelines for Training NCCL and Collective Algorithms Debugging a Slow All-Reduce

LESSONS THAT TEACH THIS

Fleet and ReliabilitySize storage for bytes, operations, concurrency and durability14 min→

PRACTICE THIS IN REAL QUESTIONS

Networking, Interconnects & StorageWhen does GPUDirect Storage actually help, and when is it just a more complicated read?→GPU & Accelerator ArchitectureExplain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?→Networking, Interconnects & StorageWhich NCCL environment variables are worth knowing, what does each change, and which should you almost never set?→Networking, Interconnects & StorageHow would you know GPUDirect RDMA is not being used, given that nothing reports an error when it is disabled?→Networking, Interconnects & StorageDesign the checkpoint write path for a 405B model on 16,384 GPUs. What has to be true for the pause to stay under a minute?→Hardware, Cabling & Cluster Build-OutAfter a maintenance window, two-node bandwidth halved. Single-node is unchanged. Diagnose it.→