← GPU & Accelerator Architecture
Advanced
NVLink, NVSwitch and PCIe
NVLink, NVSwitch and PCIe provide different GPU communication paths. Learn how their topology and bandwidth affect the placement of parallel work.
This concept is part of Premium, which includes all concepts, course lessons and answers.
Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew
RELATED CONCEPTS
LESSONS THAT TEACH THIS
PRACTICE THIS IN REAL QUESTIONS
GPU & Accelerator ArchitectureNVLink versus PCIe: when does the interconnect actually matter for an LLM workload, and when can you get away with PCIe?→Kubernetes, Slurm & GPU SchedulingTwo identical 8-GPU jobs get 8 GPUs each. One runs at half the speed of the other. What did the scheduler do, and how do you stop it?→Networking, Interconnects & StorageMap a 1,024-GPU job's parallelism onto the hardware. Which dimension goes on NVLink, which on the fabric, and what does a wrong order cost?→GPU & Accelerator ArchitectureWhat actually changes with Blackwell and the NVL72 rack, and what does it do to how you would serve a large MoE model?→GPU & Accelerator ArchitectureExplain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?→Networking, Interconnects & StorageA mixture-of-experts model does an all-to-all twice per layer. What does that demand of the fabric, and what changes on a rack-scale system?→