mfu

Reproduce the baseline, compare equivalent work, and trace the missing time. Acceptance, production and the original plan may differ in more than one way.

Compute MFU with a stated work model, then follow the training step’s critical path across input, host dispatch, computation and communication. Rank experiments by exposed time instead of assigning universal gains to fixes.

Compute MFU from a declared global token count and time interval. Check the accounting before judging the percentage or choosing an optimization.

Follow saved and recomputed values through backward. Compare complete update time and peak memory instead of promising a fixed memory saving or slowdown.

Use per-rank arrival times and controlled substitutions to explain a slow step without mistaking collective wait for network transfer.

Work from training FLOPs to fleet throughput, days and GPU-hours. Then see how utilization and downtime change the estimate.

Solve for useful throughput, round to a supported parallel layout, and account for calendar time once. The right GPU count need not be a power of two.

Derive GPU-hours from an explicit FLOP convention, compare Meta’s 30.84 million reported hours, and avoid mistaking the residual for a measured MFU.

Derive training cost from work per token, achieved model FLOPs utilization and a declared hourly price. Check the total through GPU-hours and keep project costs separate.