mediumNVIDIACrusoe

Explain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?

Compare host-staged and direct GPU-to-NIC transfers, count traffic at each boundary, and check transport selection before blaming a slow collective on missing GPUDirect.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Compare host-staged and direct GPU-to-NIC transfers, count traffic at each boundary, and check transport selection before blaming a slow collective on missing GPUDirect.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🔌 Networking & Storage🔒 Premium

GPUDirect RDMA and GPUDirect StorageGPUDirect enables supported devices to transfer data to or from GPU memory without a host staging buffer. Its benefit depends on topology, software support and the transfer path.

Core

🔌 Networking & StorageSign in

RDMA, InfiniBand and RoCEv2RDMA lets network adapters transfer data directly between registered memory regions. InfiniBand and RoCE provide different network and operating environments for those transfers.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.

Advanced

🔌 Networking & Storage🔒 Premium

Topology-Aware CommunicationMap communication groups onto real GPU and network paths. A four-rank example shows how two tensor-parallel streams share one adapter, why per-rank bandwidth can mislead, and when locality is worth waiting for.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackAn Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?Next in this trackWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.Next in this trackWhat can you rely on from the L2 cache on a GPU? When does it save a kernel, and when does it mislead you?

FEDITOR'S NOTE

Draw the actual GPU, host-memory and NIC paths. Count traffic on each link separately, then verify which transport the application used.

DISCUSSION · 0

No comments yet — be the first to share your approach.