← 📐 AI Systems Design
Core
GPU Job Scheduler Design
Design a scheduler for a shared GPU cluster is the most common design prompt in AI infrastructure interviews, because it touches everything: queues and priorities, gang placement, topology, fairness across teams, preemption and the checkpoints that make it survivable, and the failure handling that keeps a 512-GPU job alive. This page builds the design in layers, states the data model and the scheduling loop, derives the numbers (how long a job waits, how much preemption costs, how much fragmentation wastes), and lists the trade-offs the interviewer will push on.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
Kubernetes, Slurm & GPU SchedulingDesign a job queue for 100k GPU jobs with preemption: what state, what ordering, and what happens when a quota owner returns?→Kubernetes, Slurm & GPU SchedulingDesign a GPU-aware scheduler that supports fractional GPUs: what isolation does each fraction get, and where does it break?→Coding for InfraPlace GPU jobs onto nodes. Compare first fit, best fit and worst fit, and say which one a training cluster wants.→AI Infrastructure System DesignDesign a job scheduler for 100,000 jobs on a shared GPU cluster, with preemption and checkpointing. Show me the state machine.→Coding for InfraImplement a scheduler that admits jobs by priority and preempts lower-priority work when it must. What are the rules?→Kubernetes, Slurm & GPU SchedulingOur cluster is 85% allocated but 64-GPU jobs wait for hours. Explain the fragmentation and what a scheduler should do about it.→