arithmetic intensity

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Derive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.

Derive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.

Follow an accumulator across K tiles, count reuse at each memory level, and decide whether a tile improves measured GEMM performance.

Follow the weight bytes through a small decode batch and a long prefill. A scoped roofline calculation explains the difference without pretending to predict actual latency.

Turn FLOPs and HBM bytes into resource bounds, then compare them with elapsed time. A tighter memory bound suggests an investigation rather than proving the measured bottleneck.

Derive the batch-295 shortcut, count the tensors it omits, and test whether the resulting kernel estimate is even inside the replica’s memory budget.