20You are moving a model to fp4 inference on Blackwell. What breaks first, and how would you measure whether the result is acceptable?▼hardNewNVIDIAFireworks4 replies○ sign inFour bits give eight magnitudes per sign, and the largest step between them is 50%. Block scaling rescues most of the tensor; a single outlier in a block of sixteen flushes its neighbors to zero. Where the precision goes, which layers fail first, and the measurements that separate acceptable from broken.Open full answer →
29Why is fp8 training hard, and how did DeepSeek-V3 make it work?▼hardNewDeepSeekNVIDIA4 replies◆ premiumThree mantissa bits and a range of 448 make fp8 unusable with one scale per tensor. The arithmetic of what an outlier channel destroys, the 128-element block scaling and fp32 promotion that the DeepSeek-V3 report used, the ops that stayed in bf16, and what the 2× peak bought in practice.Open full answer →
