mediumNewOpenAIAnthropicMeta

Design quota and fairness for a shared research cluster: hierarchical quotas, preemption, and the incentives that keep it honest.

A research cluster is shared by teams whose managers bought it and by researchers who want it now. A quota tree with guarantees and borrowing, a fair-share formula that decays so last week's usage does not cost you today, preemption that is cheap because checkpoints are, and the incentives that make hoarding lose.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Multi-Tenancy, Quotas and Fair ShareA shared GPU pool is cheaper than ten private ones because ten teams' demand is smoother than one team's, and it only works if the sharing is enforced. Quotas say what each team is guaranteed, borrowing lets idle guarantees be used by others, fair share decides who waits when everyone wants more, and preemption reclaims borrowed capacity. This page works the arithmetic that makes pooling worth it, the layers of isolation a tenant needs, and the incentive problems (hoarding, gaming, the research-versus-product tension) that any policy has to survive.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Gang Scheduling with Kueue and VolcanoA distributed training job is 64 pods that start together or not at all: if 40 are running and 24 are Pending, the 40 hold their GPUs idle at a collective barrier waiting for ranks that may never come, and two such jobs can deadlock a whole cluster. Gang scheduling makes the job the unit of admission. Kueue and Volcano add queues, quotas, priorities and preemption on top, which is what turns a pile of GPUs into a platform several teams can share without starving each other.

Core

📐 AI Systems DesignSign in

GPU Job Scheduler DesignDesign a scheduler for a shared GPU cluster is the most common design prompt in AI infrastructure interviews, because it touches everything: queues and priorities, gang placement, topology, fairness across teams, preemption and the checkpoints that make it survivable, and the failure handling that keeps a 512-GPU job alive. This page builds the design in layers, states the data model and the scheduling loop, derives the numbers (how long a job waits, how much preemption costs, how much fragmentation wastes), and lists the trade-offs the interviewer will push on.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Spot, Preemption and Capacity StrategiesSpot and preemptible GPUs cost a fraction of on-demand and can be taken back with a couple of minutes' notice, so using them well is an expected-value calculation: the discount against the work lost per preemption, which is set by checkpoint cadence and restart time. The same arithmetic governs internal preemption in a shared cluster. This page works the break-even, the checkpoint interval that makes spot pay, and the fleet mix (reserved baseline, on-demand headroom, spot for tolerant work) that a capacity strategy is built from.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign disaster recovery for a three-month training run: checkpoint replication, cluster failover, and the RTO you can promise.Next in this trackDesign token-level billing and metering for an LLM API: count at the engine, aggregate, reconcile, and survive the failure modes.Next in this trackDesign prefill and decode disaggregation for a 70B serving fleet: pool sizes, KV transfer, routing, and when to turn it on.

FEDITOR'S NOTE

Scored on a hierarchical quota with guaranteed and borrowable shares, on decayed fair share as the tie-breaker, on preemption rules bounded by checkpoint cadence, and on incentives (charge for allocation, publish utilization) as part of the design.

DISCUSSION · 0

No comments yet — be the first to share your approach.