Derive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.
This answer is included with a free account. Sign in to read it and save your progress.
free account · no card needed
← Networking, Interconnects & Storage / 17
mediumNVIDIAMetaxAI
Why do large training clusters provision roughly 400 gigabits per second per GPU rather than more or less?
Derive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.
Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.
The concepts behind this question
Use these concept explanations to review the ideas behind this question.
Foundational
Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.🔌 Networking & Storage
Foundational
NCCL and Collective AlgorithmsNCCL implements GPU collectives using algorithms, protocols and channels over available hardware paths. Learn what each choice changes, how to read bandwidth results, and which evidence separates a communication problem from late application work.🔌 Networking & Storage
Advanced
Tensor ParallelismTensor parallelism splits operations within a layer across GPUs. Derive the column/row matrix split, count training and inference collectives, and choose a group size from memory, latency and actual interconnect performance.🕸️ Distributed Training🔒 Premium
Foundational
Collective Communication PrimitivesFollow small vectors through all-reduce, reduce-scatter, all-gather and all-to-all. Separate the result each worker needs from the algorithm-specific byte count and measured completion time.🕸️ Distributed Training
UP NEXT ON YOUR JOURNEY
Next in this trackWhat is Ultra Ethernet trying to fix, and does it change how you would choose a fabric today?Next in this trackDesign the complete network for a 1,024-GPU cluster: compute fabric, storage, management. What are the counts and where does the money go?Next in this trackDoes multi-node inference need the same fabric as training, or can it run on ordinary networking?
FEDITOR'S NOTE
Derive sent bytes from parameter ownership and collective choice. Check both average capacity and the completion time of the last ready bucket.
DISCUSSION · 0
No comments yet — be the first to share your approach.