Advanced

Gang Scheduling with Kueue and Volcano

Gang scheduling treats a distributed job’s required workers as one admission unit. It prevents partial allocations from holding resources while waiting for the rest of the job.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Kubernetes GPU Scheduling Topology-Aware Scheduling Multi-Tenancy, Quotas and Fair Share Spot, Preemption and Capacity Strategies GPU Job Scheduler Design Slurm vs Kubernetes

LESSONS THAT TEACH THIS

Fleet and ReliabilityAllocated, active and placeable are different capacity measurements14 min→Fleet and ReliabilityCoordinate admission so partial jobs do not hold the fleet indefinitely13 min→Fleet and ReliabilityPlace communication groups against measured topology14 min→

PRACTICE THIS IN REAL QUESTIONS

Kubernetes, Slurm & GPU SchedulingWhat is gang scheduling, and what goes wrong on a Kubernetes cluster that does not have it?→Kubernetes, Slurm & GPU SchedulingEight research teams share 1,024 GPUs. Design the quota and fairness policy, and tell me how they will game it.→Kubernetes, Slurm & GPU SchedulingA 64-node gang-scheduled job has been pending for six hours while the cluster shows free GPUs. Debug it live.→Kubernetes, Slurm & GPU SchedulingDesign a job queue for 100k GPU jobs with preemption: what state, what ordering, and what happens when a quota owner returns?→Kubernetes, Slurm & GPU SchedulingSlurm or Kubernetes for a 2,000-GPU training cluster? Make the case, and tell me what you lose either way.→AI Infrastructure System DesignDesign a job scheduler for 100,000 jobs on a shared GPU cluster, with preemption and checkpointing. Show me the state machine.→