Design a job queue for 100k GPU jobs with preemption: what state, what ordering, and what happens when a quota owner returns?

Build a durable, bounded scheduling loop with reservations and checkpoint-aware preemption, and avoid universal queue-wait formulas.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Build a durable, bounded scheduling loop with reservations and checkpoint-aware preemption, and avoid universal queue-wait formulas.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Core

📐 AI Systems DesignSign in

GPU Job Scheduler DesignA GPU scheduler decides which jobs run, where they run and when resources can be reclaimed. Its placement rules must account for whole jobs, topology and tenant policy.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Gang Scheduling with Kueue and VolcanoGang scheduling treats a distributed job’s required workers as one admission unit. It prevents partial allocations from holding resources while waiting for the rest of the job.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Multi-Tenancy, Quotas and Fair ShareFollow three teams through GPU borrowing and reclaim. Distinguish quotas, reservations and fair sharing; see why a returning team may leave another above its nominal share, and why failures or indivisible jobs change what can be promised.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Spot, Preemption and Capacity StrategiesPreemptible capacity trades a lower price for interruptions and uncertain availability. Compare useful completed work, billed recovery and capacity wait separately, and restore only a complete checkpoint on a compatible replacement.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhat is Ray on Kubernetes good for, and where do its scheduler and the Kubernetes scheduler fight each other?Next in this trackOur GPU pods take four minutes to start on a fresh node and sometimes fail with 'no CUDA-capable device'. Walk me through both.Next in this trackWe rent GPUs. When should we buy committed capacity instead of paying on demand, and what do we do with the rest of the demand?

FEDITOR'S NOTE

Explain durable ownership and bounded candidate selection before queue ordering. Check the scan-cost and victim-cost examples without inferring wait time from utilization alone.

DISCUSSION · 0

No comments yet — be the first to share your approach.