Advanced

Topology-Aware Scheduling

Two placements of the same 64-GPU job can differ by 2x in step time: one keeps every tensor-parallel group on a single NVSwitch node and every data-parallel ring on a single rail, the other scatters ranks across racks and pushes per-layer traffic through the spine. The scheduler is the only thing that can prevent the second placement, because the framework maps ranks to whatever GPUs it is handed. Topology-aware scheduling means the scheduler knows the hierarchy (NVLink domain, rail, rack, spine block) and places gangs to keep traffic low in it.

Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew

RELATED CONCEPTS

Topology-Aware Communication Gang Scheduling with Kueue and Volcano Rail-Optimized and Fat-Tree Fabrics NVLink, NVSwitch and PCIe Kubernetes GPU Scheduling Training Cluster Design at 10k GPUs

LESSONS THAT TEACH THIS

Fleet and ReliabilityNinety percent allocated and thirty percent used is the normal failure14 min→Fleet and ReliabilityWhere the devices sit matters more than which scheduler assigned them14 min→

PRACTICE THIS IN REAL QUESTIONS

Kubernetes, Slurm & GPU SchedulingTwo identical 8-GPU jobs get 8 GPUs each. One runs at half the speed of the other. What did the scheduler do, and how do you stop it?→Coding for InfraPlace GPU jobs onto nodes. Compare first fit, best fit and worst fit, and say which one a training cluster wants.→Kubernetes, Slurm & GPU SchedulingWhat is gang scheduling, and what goes wrong on a Kubernetes cluster that does not have it?→Kubernetes, Slurm & GPU SchedulingKubernetes device plugins versus Dynamic Resource Allocation: what changes for GPU scheduling, and what would you adopt in 2026?→Kubernetes, Slurm & GPU SchedulingEight research teams share 1,024 GPUs. Design the quota and fairness policy, and tell me how they will game it.→Hardware, Cabling & Cluster Build-OutFor 64 to 72 GPUs, is one GB300 NVL72 rack better than eight HGX B300 nodes?→