Roofline calculator
Draw the roofline for H100, H200, B200, MI300X, TPU and Trainium, find the ridge point, and place a kernel by its arithmetic intensity to see whether it is memory-bound or compute-bound and how far below the roof it sits.
Hardware
Kernel
2 FLOPs per weight read, 2 bytes per weight. Deeply memory-bound on every part ever made.
Use the result to make a decision
Every performance and kernel loop asks some form of "is this memory-bound or compute-bound, and how do you know?" The roofline is the one-picture answer. Interviewers listen for the ridge point (peak divided by bandwidth), for decode at batch one sitting at intensity around one, and for the realization that fusion and batching are both ways of moving a point to the right.
Questions people ask
Peak compute divided by memory bandwidth, in FLOPs per byte. On an H100 SXM that is 989 TFLOPS over 3.35 TB/s, about 295. A kernel with lower arithmetic intensity is memory-bound and cannot reach the peak no matter how well it is written; one above it is compute-bound.
Roughly the batch size, for bf16 weights: each weight is read once (2 bytes) and used for two FLOPs per sequence in the batch. At batch one that is about 1 FLOP per byte, hundreds of times below the H100 ridge, which is why single-stream decode reaches under 1% of peak.
It raises intensity by removing trips to HBM: the intermediate tensor stays in registers or shared memory instead of being written and read back. A fused softmax does the same FLOPs as the three separate kernels it replaces with a third of the memory traffic, so its point moves right along the roofline.