← Napkin Math, Cost & Capacity / 23

hardOpenAIAnthropicMeta

What does one training token cost?

Derive training cost from work per token, achieved model FLOPs utilization and a declared hourly price. Check the total through GPU-hours and keep project costs separate.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Derive training cost from work per token, achieved model FLOPs utilization and a declared hourly price. Check the total through GPU-hours and keep project costs separate.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧮 Napkin Math & CapacitySign in

GPU-Hours and Time to TrainConvert a stated work budget into elapsed time and GPU-hours. Vary utilization, invert for a deadline, and separate productive time from calendar delays without counting them twice.

🧮 Napkin Math & Capacity🔒 Premium

Cost per Million TokensCost per million tokens connects fleet spending to useful output. Keep the time window, token definition and utilization assumptions consistent when comparing deployments.

🧮 Napkin Math & Capacity

Training FLOPs: 6NDDerive the six from one forward and two backward matrix products. Count training tokens, attention and recomputation consistently before turning model work into a schedule.

🚀 Inference & Serving🔒 Premium

The Serving Optimization WaterfallTune an LLM deployment one change at a time against one fixed workload, in an order that follows the binding constraint. A measured run went from 789 to 6,568 output tokens per second, and the two biggest wins were not the knobs people usually reach for first.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackEstimate the latency of one decode step for a 70B model under tensor parallelism across eight H100s Next in this trackHow big can the decode batch get before the step becomes compute-bound?Next in this trackA startup wants to train a 7B model on 1 trillion tokens in three weeks. What cluster do they rent?

FEDITOR'S NOTE

Require a consistent FLOP convention and paid-time denominator; do not substitute hardware activity counters for MFU.

DISCUSSION · 0

No comments yet — be the first to share your approach.