online softmax

Standard attention writes an N by N score matrix to HBM and reads it back twice. At 8k context that is 134 MB per head. The traffic arithmetic before and after tiling, the online-softmax identity that makes one pass possible, where the kernel lands on the roofline, and why the memory saving matters more than the speed.

A bug that scales with the number of tiles is invisible in a unit test that fits in one tile. The three candidates that produce exactly this signature, the measurement that separates them, and a test design that would have caught all three before the kernel shipped.