Run untrusted user code at 50,000 concurrent sessions, some on GPUs. Pick the isolation boundary and defend the density you lose.

Choose a sandbox from its threat model and supported workload, then calculate density from total session memory and CPU demand. GPU access needs its own isolation and compatibility review.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Choose a sandbox from its threat model and supported workload, then calculate density from total session memory and CPU demand. GPU access needs its own isolation and compatibility review.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

📐 AI Systems Design🔒 Premium

Serverless GPU PlatformsA serverless GPU platform hides allocation and startup from its users. The design balances cold-start latency, tenant isolation and the cost of idle capacity.

Core

🗂️ Scheduling & OrchestrationSign in

MIG, MPS and Time-SlicingMIG, MPS and time-slicing share a GPU in different ways. Compare memory capacity, execution behavior and isolation before choosing a mechanism.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Multi-Tenancy, Quotas and Fair ShareFollow three teams through GPU borrowing and reclaim. Distinguish quotas, reservations and fair sharing; see why a returning team may leave another above its nominal share, and why failures or indivisible jobs change what can be promised.

Advanced

📐 AI Systems Design🔒 Premium

Multi-Tenant Fine-Tuning ServiceA fine-tuning service turns a tenant’s dataset into an evaluated model or adapter. Its pipeline must protect data while scheduling training and serving the result efficiently.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYour fine-tunes run on spot GPUs preempted about once every four hours. How often should they checkpoint, and when does spot stop paying?Next in this trackOne fleet: training that wants every idle GPU, and inference with a p99 SLO. Separate pools, or one pool with preemption? Show the numbers.Next in this trackDesign a job queue for 100k GPU jobs with preemption: what state, what ordering, and what happens when a quota owner returns?

FEDITOR'S NOTE

Separate application memory from sandbox overhead, distinguish gVisor from a guest-kernel VM, and avoid treating a GPU partition as a complete tenant-security boundary.

DISCUSSION · 0

No comments yet — be the first to share your approach.