Advanced

Multi-Tenancy, Quotas and Fair Share

Follow three teams through GPU borrowing and reclaim. Distinguish quotas, reservations and fair sharing; see why a returning team may leave another above its nominal share, and why failures or indivisible jobs change what can be promised.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Gang Scheduling with Kueue and Volcano MIG, MPS and Time-Slicing Capacity Planning and Utilization Spot, Preemption and Capacity Strategies GPU Job Scheduler Design Multi-Tenant Fine-Tuning Service

LESSONS THAT TEACH THIS

Fleet and ReliabilityCoordinate admission so partial jobs do not hold the fleet indefinitely13 min→Fleet and ReliabilitySpecify entitlement, borrowing and reclaim under contention20 min→Fleet and ReliabilitySpecify isolation by resource and validate the end-to-end promise30 min→

PRACTICE THIS IN REAL QUESTIONS

Kubernetes, Slurm & GPU SchedulingEight research teams share 1,024 GPUs. Design the quota and fairness policy, and tell me how they will game it.→Kubernetes, Slurm & GPU SchedulingOne fleet: training that wants every idle GPU, and inference with a p99 SLO. Separate pools, or one pool with preemption? Show the numbers.→AI Infrastructure System DesignDesign a GPU credit system: accounts, spending rates, priorities and fairness. Start with the data model.→GPU & Accelerator ArchitectureMIG versus MPS: what isolation does each give you when sharing a GPU, and which would you pick for a multi-tenant inference node?→Open-Weights Models & Serving EnginesOne tenant's traffic is pushing everyone else past their latency objective. What do you do?→AI Infrastructure System DesignDesign quota and fairness for a shared research cluster: hierarchical quotas, preemption, and the incentives that keep it honest.→