← Napkin Math, Cost & Capacity / 26

mediumLambdaRunPodNebius

A startup wants to train a 7B model on 1 trillion tokens in three weeks. What cluster do they rent?

Calculate the fleet, duration and rental budget for a 7B training run. Check the deadline, actual sharding scheme and paid reservation period before placing the order.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Calculate the fleet, duration and rental budget for a 7B training run. Check the deadline, actual sharding scheme and paid reservation period before placing the order.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧮 Napkin Math & CapacitySign in

GPU-Hours and Time to TrainConvert a stated work budget into elapsed time and GPU-hours. Vary utilization, invert for a deadline, and separate productive time from calendar delays without counting them twice.

🧮 Napkin Math & Capacity

KV Cache SizingDerive cache payload from the stored attention state, then account for mixed sequence lengths, block allocation and per-device limits. Worked examples distinguish exact bytes from deployable capacity.

🧮 Napkin Math & Capacity🔒 Premium

Bandwidth-Bound Decode ThroughputEstimate decode throughput from bytes per step, then separate aggregate output from per-user pace. A finite-memory example shows how batching, cache reads and quantization change the bound.

🕸️ Distributed Training

Data Parallelism and DDPReplicas combine gradients from different data. Follow a two-worker update, fix unequal-token weighting, and see why queued gradient buckets can leave more than one reduction exposed.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackEstimate the activation memory for training a 70B model at an 8k sequence length Next in this trackModel our serving request queue with Little's law. What happens as we approach saturation?Next in this trackRank the H100, MI300X and Trainium2 by cost per token for decode

FEDITOR'S NOTE

Require coherent calendar, memory and billing assumptions; distinguish a rounded fleet from a spare node.

DISCUSSION · 0

No comments yet — be the first to share your approach.