batch size

Derive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.

Derive the weighted gradient, test a short micro-batch, and separate activation savings from total memory and communication. Matching the optimizer step is only part of matching a larger batch.

Trace an accepted prefix, a rejection and the correction token; derive emitted progress and compare it with a measured round cost.

Separate device throughput bounds from measured service curves, then choose a feasible operating point using latency, quality and goodput.

Check the bound’s assumptions, locate the critical path, and quantify which optimization can actually improve one user’s decode latency.