hardCrusoeLambdaNebius

Explain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?

Follow a host buffer across a NUMA boundary, calculate the cost with declared bandwidths, and align CPU workers and memory with the GPU that consumes the data.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Follow a host buffer across a NUMA boundary, calculate the cost with declared bandwidths, and align CPU workers and memory with the GPU that consumes the data.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

NVLink, NVSwitch and PCIeNVLink, NVSwitch and PCIe provide different GPU communication paths. Learn how their topology and bandwidth affect the placement of parallel work.

Advanced

🔌 Networking & Storage🔒 Premium

Data Loading Pipelines for TrainingTrace batches from storage to GPU readiness, budget workers and prefetch memory, and resume from the input position saved with training state. An executable example shows how a read-ahead cursor can silently skip samples.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.

Advanced

🔌 Networking & Storage🔒 Premium

GPUDirect RDMA and GPUDirect StorageGPUDirect enables supported devices to transfer data to or from GPU memory without a host staging buffer. Its benefit depends on topology, software support and the transfer path.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackCompare A100, H100 and B200 for a 70B serving fleet. Which gives the most tokens per dollar, and where do fp8 and fp4 change the ranking?Next in this trackIf you could change one thing about GPU architecture for LLMs, what would it be, and what argues against it?Next in this trackOne node in your cluster shows half the expected NVLink bandwidth in nccl-tests. Walk me through isolating it.

FEDITOR'S NOTE

Check the actual NUMA topology and buffer placement. CPU affinity alone does not move existing pages, and a remote path does not imply a universal twofold slowdown.

DISCUSSION · 0

No comments yet — be the first to share your approach.