Our cluster is 85% allocated but 64-GPU jobs wait for hours. Explain the fragmentation and what a scheduler should do about it.

Count feasible placements for the waiting job, including its per-node and topology requirements. Then compare waiting, reservations and checkpoint/restart consolidation using their actual disruption costs.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Core

📐 AI Systems DesignSign in

GPU Job Scheduler DesignA GPU scheduler decides which jobs run, where they run and when resources can be reclaimed. Its placement rules must account for whole jobs, topology and tenant policy.

Advanced

🚀 Inference & Serving🔒 Premium

PagedAttentionFollow a token position through a block table, watch a cache grow, and trace a shared tail through copy-on-write. Separate memory saved from throughput gained.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Gang Scheduling with Kueue and VolcanoGang scheduling treats a distributed job’s required workers as one admission unit. It prevents partial allocations from holding resources while waiting for the rest of the job.

Advanced

📐 AI Systems Design🔒 Premium

Serverless GPU PlatformsA serverless GPU platform hides allocation and startup from its users. The design balances cold-start latency, tenant isolation and the cost of idle capacity.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackKubernetes device plugins versus Dynamic Resource Allocation: what changes for GPU scheduling, and what would you adopt in 2026?Next in this trackWhat do you run on a GPU node before you let a job land on it, how long does it take, and what happens on failure?Next in this trackA GPU in a running 512-GPU training job is throwing errors. How do you get it out of the job without losing the run?

FEDITOR'S NOTE

Explain free versus placeable capacity with a consistent node inventory. Correctly distinguish spreading from first-fit packing, and account for reservations and migration without inventing idle bills.

DISCUSSION · 0

No comments yet — be the first to share your approach.