← 🧩 GPU & Accelerator Architecture
Advanced
NVLink, NVSwitch and PCIe
Inside a node, GPUs talk over NVLink at 900 GB/s per H100 through an NVSwitch fabric that gives all eight cards full bandwidth to each other; to the host and to anything outside the node they talk over PCIe at 64 GB/s or a 400 Gb/s NIC at 50 GB/s. That fifteen-fold gap is why tensor parallelism stays inside the eight-GPU domain, why NVL72 changes the serving math for MoE, and why the question "how many GPUs share an NVLink domain?" is the first thing to ask about any cluster.
Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew
RELATED CONCEPTS
LESSONS THAT TEACH THIS
PRACTICE THIS IN REAL QUESTIONS
GPU & Accelerator ArchitectureNVLink versus PCIe: when does the interconnect actually matter for an LLM workload, and when can you get away with PCIe?→Kubernetes, Slurm & GPU SchedulingTwo identical 8-GPU jobs get 8 GPUs each. One runs at half the speed of the other. What did the scheduler do, and how do you stop it?→Networking, Interconnects & StorageMap a 1,024-GPU job's parallelism onto the hardware. Which dimension goes on NVLink, which on the fabric, and what does a wrong order cost?→GPU & Accelerator ArchitectureWhat actually changes with Blackwell and the NVL72 rack, and what does it do to how you would serve a large MoE model?→GPU & Accelerator ArchitectureExplain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?→Networking, Interconnects & StorageA mixture-of-experts model does an all-to-all twice per layer. What does that demand of the fabric, and what changes on a rack-scale system?→