mediumNewNVIDIACrusoe

Explain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?

Without it, every byte a GPU sends over the network crosses PCIe twice and passes through host memory. With it, the NIC reads and writes GPU memory directly. The PCIe arithmetic that shows why a 400G NIC cannot be fed the slow way, the NCCL line that tells you which path you are on, and the checkpoint case.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🔌 Networking & Storage🔒 Premium

GPUDirect RDMA and GPUDirect StorageBy default a byte leaving a GPU for the network or the disk makes a detour through host memory, crossing PCIe twice and costing a CPU copy. GPUDirect RDMA lets the NIC read and write GPU memory directly, and GPUDirect Storage does the same for NVMe. The win is not raw bandwidth (PCIe is the ceiling either way) but the halving of PCIe traffic and the removal of the host as a bottleneck, which is what makes collectives run at NIC rate and checkpoints run at drive rate. When it is silently off, everything still works, at half speed.

Core

🔌 Networking & StorageSign in

RDMA, InfiniBand and RoCEv2Training across nodes moves hundreds of gigabytes per step, and a CPU-driven TCP stack cannot feed a 400 Gb/s link. RDMA lets a NIC write straight into a remote GPU's memory with no kernel and no copies, and it runs over two fabrics: InfiniBand, which is lossless by design, and RoCEv2, which is Ethernet made lossless by configuration. The choice is operational as much as technical, and the numbers that decide it are per-GPU bandwidth, the collective's volume, and who will debug a pause storm at 3 a.m.

Advanced

🔌 Networking & Storage🔒 Premium

Topology-Aware CommunicationThe same collective can run at 900 GB/s or at 50 GB/s depending on which links it is laid across, so the mapping of parallel groups onto hardware is a performance decision, not a deployment detail. The rule: tensor-parallel groups inside the NVLink domain, data-parallel rings along rails, pipeline stages across the fabric, and every rank placed so its partner is one hop away. NCCL discovers the topology and does most of this when the job lets it; the failures come from placements that do not.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceA training job reports its all-reduce at a third of what the fabric should deliver, every node passed its health check, and nothing is logged. This page is the isolation order that finds the cause in an hour instead of a day: measure the collective in isolation, split the job until the slow pair or rank appears, then check the specific things that make a link, a node or a placement slow. Most cases end at one NIC, one topology mismatch, or GPUDirect silently off.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackAn Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?Next in this trackWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.Next in this trackWhat can you rely on from the L2 cache on a GPU? When does it save a kernel, and when does it mislead you?

FEDITOR'S NOTE

Scored on a correct picture of the two data paths, the PCIe bandwidth arithmetic that makes the bounce path a bottleneck at 400G, and the concrete checks that confirm GDR is active on a node.

DISCUSSION · 0

No comments yet — be the first to share your approach.