Our GPU pods take four minutes to start on a fresh node and sometimes fail with 'no CUDA-capable device'. Walk me through both.

Measure scheduling, image transfer, unpacking and model initialization separately. A missing-device error needs its own checks for allocation, container access and CUDA compatibility.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Measure scheduling, image transfer, unpacking and model initialization separately. A missing-device error needs its own checks for allocation, container access and CUDA compatibility.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Containers, Images and GPU Cold StartsGPU cold starts include scheduling, image loading, model loading and engine initialization. Measure each stage before choosing caches, warm pools or supported snapshot mechanisms.

Foundational

🩺 Fleet Reliability & Observability

GPU Failure Modes and XID ErrorsXID messages report GPU errors that need interpretation alongside logs and workload behavior. Use the code, recovery guidance and recurrence pattern to choose the next check.

Advanced

🚀 Inference & Serving🔒 Premium

Inference Autoscaling and Cold StartsInference autoscaling adds or removes model replicas as demand changes. Queue pressure, latency and startup time help determine how much ready capacity to keep.

Advanced

📐 AI Systems Design🔒 Premium

Serverless GPU PlatformsA serverless GPU platform hides allocation and startup from its users. The design balances cold-start latency, tenant isolation and the cost of idle capacity.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWe rent GPUs. When should we buy committed capacity instead of paying on demand, and what do we do with the rest of the demand?Next in this trackOur cluster is 85% allocated but 64-GPU jobs wait for hours. Explain the fragmentation and what a scheduler should do about it.Next in this trackKubernetes device plugins versus Dynamic Resource Allocation: what changes for GPU scheduling, and what would you adopt in 2026?

FEDITOR'S NOTE

Use a timed startup trace and an explicit compatibility matrix. Distinguish compressed image bytes from unpacked bytes, and verify device visibility before assigning a cause to a CUDA error.

DISCUSSION · 0

No comments yet — be the first to share your approach.