scaling

Every format is a trade between how big a number can be and how finely it is spaced. The bit layouts, the largest and smallest values derived from them, why fp16 needed loss scaling and bf16 did not, and why fp8 comes in two flavors with a scale factor attached.

The matmul runs on tiny formats and the accumulator does not, so every low-precision GEMM is really a scaled integer or float product with a correction applied on the way out. Where the scales come from, the three granularities and what each costs, and why one of them needs a periodic promotion into fp32.

The interesting content is the order things broke in, because that order is a property of the system and it repeats. What breaks first at each scale step, why the control plane usually goes before the data plane, and the thing you would skip if you did it again.