memory hierarchy

Registers, shared memory, L1, L2, HBM, host memory: sizes, bandwidths and latencies for an H100, derived rather than recited, and the habit of asking 'which level am I hitting' before touching a kernel.

An H100 has 50 MB of L2, split into two partitions, with a bandwidth several times HBM's. That is enough to change the roofline for a kernel whose working set fits and to do nothing at all for the ones that stream. The arithmetic of what fits, the residency controls, and the two ways the L2 lies to a profile.