MFU is the only throughput number that cannot be gamed

Model FLOPs utilisation divides the arithmetic your model actually required by the arithmetic the machine can perform, which makes it the one number that ties hardware to delivered tokens. Device utilisation counters do not measure this and routinely read high on a run that is wasting most of the fleet.

15 MIN · PREMIUM