TL;DR: Under the stated 6.354e24-FLOP budget, 989 TFLOP/s dense BF16 peak and assumed 40% MFU, 30 productive days require about 6,196.64 GPUs. Round to a supported layout, then verify memory and scaling. For an illustrative TP8 × PP4 layout, 6,208 GPUs gives 194 model replicas, a data-parallel degree of 194. Powers of two are not a universal requirement, and a fixed 15% allowance does not prove a deadline is safe.
How to approach it
First define the work budget and whether 30 days means productive training or calendar time. State the peak and utilization convention, solve for GPU count, then round for node and parallel-layout constraints. Check whether the assumed throughput is achievable on that layout before treating the result as a provisioning plan.
A strong answer
A calculation returns 6,197 GPUs. The cluster uses eight-GPU nodes, and the proposed model replica spans four nodes. In the TP8 × PP4 layout, tensor parallelism (TP) splits each layer's matrices across 8 GPUs and pipeline parallelism (PP) splits the stack of layers into 4 consecutive stages, so one replica uses 8 × 4 = 32 GPUs. Buying the rounded integer does not produce a complete set of replicas. The arithmetic and the placement must agree.
Training FLOPs: 6ND gives the illustrative useful-work budget C = 6 × 70.6e9 × 15e12 = 6.354e24 FLOPs. Use a rounded H100 SXM dense BF16 peak of 989e12 FLOP/s. The 40% MFU is an assumption with the same useful-work convention; it is not a measured result for this proposed fleet.
T = 30 × 86,400 = 2,592,000 productive seconds
per-GPU useful work = T × 989e12 × 0.40
= 1.0253952e21 FLOPs
G = 6.354e24 / 1.0253952e21 = 6,196.64
sanity: G × 30 × 24 ≈ 4.462 million GPU-hours,
matching the fixed-utilization work/rate calculation.
GPU-Hours and Time to Train explains the assumptions behind that invariant. Device-hours are independent of fleet count only when useful utilization and workload remain fixed. More ranks can change communication, microbatch shapes and recomputation policy.
| Assumed MFU | Unrounded GPU requirement |
|---|---|
| 30% | 8,262.18 |
| 40% | 6,196.64 |
| 50% | 4,957.31 |
These are alternative assumptions, not labels for bad, good and exceptional systems. A pilot should report raw token throughput and a comparable MFU before choosing a row.
Now make the placement concrete. Suppose tensor parallelism uses eight GPUs per stage and pipeline parallelism uses four stages. One model replica then spans 8 × 4 = 32 GPUs. The next multiple of 32 above 6,196.64 is 6,208, forming 194 replicas. That is 776 eight-GPU nodes. The dimensions divide even though the fleet is not a power of two.
A fractional replica cannot take part in data-parallel training. Gradients are then reduced across 32 data-parallel groups, one for each tensor/pipeline position, and each group holds the matching rank from all 194 replicas. Round after identifying the supported replica dimensions, then verify that the recipe and measured throughput work at that replica count.
The example illustrates divisibility only. It does not certify that TP8 × PP4 is the fastest or even the intended memory layout. Check the model's tensor dimensions, layer placement, per-rank memory, microbatch and global batch before accepting it. The data-parallel degree, here the replica count, also has to work with the training recipe, and the engine may have additional constraints.
If the 30 days are calendar time, account for the scope of MFU. In a hypothetical case where only 90% of calendar time is productive and 40% MFU describes those productive intervals, the available training window is 27 days:
G = 6.354e24 / (27 × 86,400 × 989e12 × 0.40)
= 6,885.15
round to 32-GPU replicas: 6,912 GPUs = 216 replicas
Before applying the availability factor, check which time interval the MFU describes. If that interval already includes the outages, multiplying by availability would count them twice. The same rule applies to learning-rate warm-up: its work is already covered if those tokens are in D. Evaluation, compilation and recovery may need separate schedule entries, but each delay should appear once.
I would propose candidate layouts around the requirement and measure their useful throughput. If no measured configuration can finish the remaining recipe within the remaining window, the options include more capacity, a changed recipe or a changed date. Quantization and other precision changes require training-quality evaluation; they are not automatic schedule rescue factors.
Track the forecast through the run. Divide each remaining phase's tokens by its expected throughput, then include excluded delays. A throughput average from short-context steps can mislead when the remaining phase has longer sequences. MFU and HFU explains how to keep that comparison consistent.
What interviewers probe next
- “Why not automatically request 8,192 GPUs?” It is one possible fleet, not a mathematical requirement. Availability, topology, memory, throughput and the recipe should justify it.
- “What if only 6,144 are available?” At the example's fixed 40% MFU, productive duration is about 30.257 days. A 30-day calendar promise is unsupported without changing an input or demonstrating higher throughput.
- “What does it cost?” Multiply the actual billed GPU-hours by the applicable rate and add other billed resources. Useful compute-hours and reserved hours may differ.
- “Can I use an active parameter count for MoE?” Use a workload-specific compute model for selected experts and shared work, then separately check resident weights and expert communication. Reusing the dense 40% assumption needs evidence.
Common mistakes
- Treating an unrounded quotient as a complete deployable layout.
- Rounding to a power of two without checking the actual replica dimensions.
- Adding a fixed overhead allowance with no denominator or measurement scope.
- Assuming utilization remains constant after changing fleet size.
Key takeaways
- The stated productive-time estimate needs 6,196.64 GPUs at 40% MFU.
- A 32-GPU replica constraint rounds that example to 6,208, not necessarily 8,192.
- A separate 90%-availability assumption raises the rounded requirement to 6,912.
- Memory, recipe constraints and measured scaling decide whether the candidate can meet the deadline.