← Napkin Math, Cost & Capacity / 22

hardOpenAIAnthropic

Traffic peaks at three times the daily average. Capacity-plan the serving fleet.

Size for a tested peak and a stated failure, then price an explicit daily scaling schedule. Keep request-slot occupancy separate from GPU utilization.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Size for a tested peak and a stated failure, then price an explicit daily scaling schedule. Keep request-slot occupancy separate from GPU utilization.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧮 Napkin Math & Capacity🔒 Premium

Capacity Planning and UtilizationCapacity planning connects demand, useful throughput and acceptable waiting time. Serving and training need different models of bursts, job shape and spare capacity.

🧮 Napkin Math & Capacity🔒 Premium

Cost per Million TokensCost per million tokens connects fleet spending to useful output. Keep the time window, token definition and utilization assumptions consistent when comparing deployments.

🧮 Napkin Math & Capacity🔒 Premium

TCO: Buy vs RentBuying and renting GPU capacity have different cost and availability risks. Compare them over a stated lifetime and demand pattern, including idle time and operating costs.

🧮 Napkin Math & Capacity🔒 Premium

Bandwidth-Bound Decode ThroughputEstimate decode throughput from bytes per step, then separate aggregate output from per-user pace. A finite-memory example shows how batching, cache reads and quantization change the bound.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhat does one training token cost?Next in this trackEstimate the latency of one decode step for a 70B model under tensor parallelism across eight H100s Next in this trackHow big can the decode batch get before the step becomes compute-bound?

FEDITOR'S NOTE

Check units, failure scope, interval-by-interval capacity and startup delay before claiming an idle-capacity saving.

DISCUSSION · 0

No comments yet — be the first to share your approach.