A quantized format needs a compatible and measured execution path

A smaller checkpoint runs faster only when its execution path helps the workload. Compare narrow arithmetic, conversion inside a kernel and conversion into a separate tensor, then account for scales and measure the kernel that actually ran.

20 MIN · PREMIUM