awq

Separate weight storage, matrix arithmetic and attention-cache precision. Choose a supported candidate using task quality, memory and service measurements.

Compare actual encoded tensors, per-rank runtime fit, supported kernels and held-out task quality. Four-bit weight-only kernels can accelerate wider arithmetic, and smaller weights need not reduce the GPU count.

Define whether half the footprint means checkpoint bytes or runtime memory. Choose a supported recipe, measure which tensors it changes, separate calibration from held-out evaluation, and validate quality and serving cost.

Map a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.