TL;DR: GPUs = FLOPs ÷ (seconds × peak × MFU) = 6.35e24 ÷ (2.59e6 × 989e12 × 0.4) ≈ 6,200 H100s at 40% MFU. At 30% MFU it is 8,300, at 50% it is 5,000. Add headroom for failures, restarts and evaluation, and the practical order is 8,192.
How to approach it
Ask what MFU the interviewer wants assumed, or state one and defend it (40% for a dense bf16 run with good overlap). Compute the FLOPs budget in one line, convert 30 days to seconds, and then compute what one GPU delivers over the whole window; the GPU count is the budget divided by that per-GPU delivery, which keeps the arithmetic to two divisions. Give the number with its MFU attached, the range across MFUs, and then the fleet you would actually ask for, which is larger and a power of two.
A strong answer
A typical situation: a deadline arrives first and the GPU count has to follow it, which is the same equation solved for a different unknown. The judgment is in one term, so the answer should be a range with that term named.
The fleet equation rearranged for the count: GPUs = C ÷ (T × peak × MFU). The only judgment call is MFU, so the answer is a function of it and should be presented that way.
inputs: C = 6 × 70.6e9 × 15e12 = 6.35e24 FLOPs
T = 30 days = 30 × 86,400 = 2.59e6 s
peak = 989e12 FLOP/s (H100, dense bf16)
MFU = 0.40
per-GPU delivery over the window = T × peak × MFU
= 2.59e6 × 989e12 × 0.40
= 1.025e21 FLOPs per GPU
GPUs = C ÷ per-GPU delivery
= 6.35e24 ÷ 1.025e21
≈ 6,197 → about 6,200 H100s
sanity: 6,200 GPUs × 30 days × 24 h = 4.5M GPU-hours, the same total as 16,384 GPUs for 11.3 days,
which it must be, since the compute budget did not change.
The GPU-hours are conserved; the deadline just sets how many cards run in parallel. That equivalence is the useful check in the room: any (GPUs, days) pair for this run multiplies to about 4.5 million GPU-hours at 40% MFU.
| MFU | GPUs | why it would be this |
|---|---|---|
| 30% | 8,262 | communication not overlapped, small micro-batches, stragglers |
| 40% | 6,197 | a well-tuned dense bf16 run |
| 50% | 4,957 | rare on H100 for a 70B; requires fp8 or excellent overlap |
Now the part that separates a calculation from a plan. The 6,200 figure is compute-only over 30 calendar days with no interruptions. A real schedule loses time to:
- Failures and restarts. A large fleet sees hardware faults daily; each costs the time since the last checkpoint plus the restart. Budget 5 to 10% of the calendar.
- Evaluation and checkpointing. Periodic eval passes and synchronous checkpoint writes are not model FLOPs.
- Ramp. The first days are rarely at full MFU while parallelism and batch size are tuned.
Adding 15% to the count covers those, giving about 7,100, which nobody provisions; fleets are built in powers of two because data and tensor parallel groups divide evenly, so the request is 8,192 H100s (1,024 nodes). That fleet finishes the compute in about 22.6 days at 40% MFU and leaves a week of slack, which is the right shape for a deadline that matters. If 8,192 is not available, the alternative is to renegotiate the deadline (6,144 cards, three quarters of a power of two, is also a common size, and finishes the compute in about 29 days at 42% MFU, with no slack) or to move to fp8, which raises the per-GPU delivery.
The reversal condition: if the model is a MoE with 70B total and 15B active, N in 6ND drops to 15e9, the budget drops to 1.35e24, and the same deadline needs about 1,300 GPUs at 40% MFU. Whether the fleet holds the weights is then the constraint, not compute. Training FLOPs: 6ND supplies the budget, and DCGM_FI_PROF_PIPE_TENSOR_ACTIVE on day one tells you whether the MFU you assumed is the MFU you have. GPU-Hours and Time to Train is the same equation solved for the other unknown.
What interviewers probe next
- "Why a power of two?" Parallelism groups (TP × PP × DP) must divide the world size; 8,192 = 8 × 4 × 256 and its variants all work, while 7,100 does not factor cleanly.
- "What does the fleet cost?" 8,192 × $2.50 × 24 × 30 ≈ $14.7M for the month at on-demand rates; the compute-only 4.5M GPU-hours is $11.3M.
- "Would fp8 change the answer?" Peak doubles to 1,979 TFLOPS; at a realistic 30 to 35% fp8 MFU the delivery per GPU is about 1.5x, so roughly 4,100 cards for the same deadline.
- "How would you know you were on schedule?" Measured tokens per second × 6N ÷ (GPUs × peak) is the live MFU; compare it to the assumed 40% on day two, not day twenty.
Common mistakes
- Reporting the exact quotient, 6,197, as the fleet, with no allowance for failures, evaluation or divisibility.
- Not stating MFU, which turns a range of 5,000 to 8,300 into a single unverifiable number.
- Mixing 30 days with hours in the divisor and getting a count off by 24.
- Using total parameters for a MoE, overstating the compute by the total-to-active ratio.
Key takeaways
- GPUs = C ÷ (T × peak × MFU); 6.35e24 in 30 days on H100s at 40% MFU is about 6,200.
- GPU-hours are conserved: about 4.5M for this run at 40% MFU, however you split them.
- Add 15% for failures and evaluation, then round to a power of two: ask for 8,192.
- The MFU band 30 to 50% moves the answer from 8,300 to 5,000; always say which you used.