Your product forecasts 12,800 output tokens per second at peak. Size the fleet.

The output forecast is a starting point. Match the prompt and state workload, derive replica counts from an SLO-tested rate, check simultaneous loss scenarios and keep the cost denominator consistent with the forecast.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Foundational

🧮 Open Weights & Serving Engines

Capacity Planning for Open-Weights FleetsMatch forecast traffic to a measured replica operating envelope, then check failure domains, rollout capacity, readiness time and cost. Output tokens alone omit prompt work and live state; total parameter count alone does not determine a replica's throughput.

Foundational

🧭 Ownership & Judgment

Talking About Cost and Capacity with LeadershipA capacity discussion should connect technical measurements to cost, delivery dates and risk. Present the assumptions, feasible options and a recommendation, and distinguish committed capacity from actual consumption under the relevant contract.

Foundational

🧮 Open Weights & Serving Engines

Reading config.json to Size a Model You Have Never RunRead a model configuration as an architecture description, then combine it with tensor metadata and engine allocation rules. Learn which fields determine attention state, why mixed precision needs tensor-level accounting, and how to turn a first estimate into a testable per-GPU capacity plan.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Capacity Planning and UtilizationCapacity planning connects demand, useful throughput and acceptable waiting time. Serving and training need different models of bursts, job shape and spare capacity.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackIs speculative decoding worth enabling on a trillion-parameter mixture-of-experts model?Next in this trackYour deployment is limited by KV cache rather than by compute. What are the options?Next in this trackYour p99 time to first token is four times p50. Find out why.

FEDITOR'S NOTE

Scored on conditional replica arithmetic, workload-matched measurements, failure-domain and rollout capacity, sensitivity to throughput error and a consistent average-cost calculation.

DISCUSSION · 0

No comments yet — be the first to share your approach.