Design storage for a multi-tenant GPU cloud. How do you keep one customer's checkpoint burst from slowing another's training run?

Every tenant bursts independently, so the shared tier sees the sum of uncorrelated peaks rather than the sum of averages. The provisioning arithmetic that follows, the three isolation boundaries, and why per-tenant local storage solves most of it before any policy is written.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Multi-Tenancy, Quotas and Fair ShareA shared GPU pool is cheaper than ten private ones because ten teams' demand is smoother than one team's, and it only works if the sharing is enforced. Quotas say what each team is guaranteed, borrowing lets idle guarantees be used by others, fair share decides who waits when everyone wants more, and preemption reclaims borrowed capacity. This page works the arithmetic that makes pooling worth it, the layers of isolation a tenant needs, and the incentive problems (hoarding, gaming, the research-versus-product tension) that any policy has to survive.

Core

🗂️ Scheduling & OrchestrationSign in

MIG, MPS and Time-SlicingA whole H100 is far more than a notebook, a small inference service or a CI job needs, and giving each of them a card leaves most of the fleet idle. Three mechanisms share a GPU, and they differ in what they isolate: MIG partitions the hardware into up to seven slices with their own memory and compute, MPS lets several processes share one GPU's SMs concurrently with no memory isolation, and time-slicing context-switches between processes with no isolation at all. The choice is the isolation the workload needs against the utilization the platform wants.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Gang Scheduling with Kueue and VolcanoA distributed training job is 64 pods that start together or not at all: if 40 are running and 24 are Pending, the 40 hold their GPUs idle at a collective barrier waiting for ranks that may never come, and two such jobs can deadlock a whole cluster. Gang scheduling makes the job the unit of admission. Kueue and Volcano add queues, quotas, priorities and preemption on top, which is what turns a pile of GPUs into a platform several teams can share without starving each other.

Advanced

📐 AI Systems Design🔒 Premium

Multi-Tenant Fine-Tuning ServiceA fine-tuning service takes a customer's dataset and a base model and returns a model, and the design problem is that many customers want this at once, cheaply, without seeing each other's data, on GPUs that must not sit idle between jobs. LoRA changes the shape: an adapter is a few hundred megabytes rather than a copy of the base, so many jobs can share a base in memory and many adapters can be served from one replica. This page designs the service end to end: the pipeline, the LoRA arithmetic that sets memory and cost, the isolation, the scheduler that packs jobs, and the serving path.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackAfter a switch firmware update, all-reduce bandwidth across the cluster fell 20 percent. Every link is up and no errors are logged. Find it.Closest to this topicOne tenant's traffic is pushing everyone else past their latency objective. What do you do?Closest to this topicEight research teams share 1,024 GPUs. Design the quota and fairness policy, and tell me how they will game it.

FEDITOR'S NOTE

Scored on provisioning for uncorrelated peaks rather than averages, on the three isolation boundaries (capacity, throughput, namespace), and on preferring node-local capacity over shared-tier arbitration.

DISCUSSION · 0

No comments yet — be the first to share your approach.