IO-aware attention is a memory-traffic algorithm, not a faster matmul

IO-aware attention performs the same arithmetic as the naive version and changes only where bytes go: it never writes the quadratic score matrix and re-reads keys and values per query tile in a way the cache can serve. It is the reuse law's hardest case, with the online rescale as the device that makes streaming legal.

18 MIN · PREMIUM