Define which work survived, count the resources it occupied, and explain the remaining time without counting the same delay twice.
This answer is included with a free account. Sign in to read it and save your progress.
free account · no card needed
← GPU Fleet Reliability & Observability / 19
mediumMetaAnthropic
How do you measure effective training time, and where does the missing ten percent of a well-run cluster actually go?
Define which work survived, count the resources it occupied, and explain the remaining time without counting the same delay twice.
Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.
The concepts behind this question
Use these concept explanations to review the ideas behind this question.
Advanced
Disaggregated Prefill and DecodeSeparate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.🚀 Inference & Serving🔒 Premium
Foundational
Serving Benchmarks That Do Not LieDesign reproducible serving experiments with realistic arrivals, cache state and failure accounting. Interpret latency-qualified throughput and conditional per-rank bounds without turning optimistic kernel estimates into production capacity.🧮 Open Weights & Serving Engines
Core
SLOs for AI SystemsSLOs define the service outcomes users or training teams need. Choose measurable latency, availability and progress targets, then use their error budgets to guide decisions.🩺 Fleet Reliability & ObservabilitySign in
Foundational
Training Uptime and Interruption StatisticsTraining interruption statistics connect failure frequency to lost progress. Use a stated rate, checkpoint cost and recovery model to estimate how much time a run can lose.🩺 Fleet Reliability & Observability
UP NEXT ON YOUR JOURNEY
Next in this trackWalk me through reading a collective flight-recorder dump. What is in it, and how do you find the rank that caused a hang?Next in this trackThe facility asks you to cap GPU power by 30 percent for the summer. What does that cost a training run, and how would you do it?Next in this trackA GPU computes the wrong answer and reports no error. How would you detect that, and what does it look like in a training run?
FEDITOR'S NOTE
Look for consistent accounting across restarts, changing allocations, and time spent waiting inside a training step.
DISCUSSION · 0
No comments yet — be the first to share your approach.