evaluation

Four bits give eight magnitudes per sign, and the largest step between them is 50%. Block scaling rescues most of the tensor; a single outlier in a block of sixteen flushes its neighbors to zero. Where the precision goes, which layers fail first, and the measurements that separate acceptable from broken.

Peak numbers are almost never the reason a part wins or loses, and the software stack decides more deployments than the silicon does. The four things to measure, the question that eliminates most challengers in one conversation, and the pilot that gives a defensible answer.

Halving the weights halves the GPU count and moves the quality evaluation onto you. What the format actually buys in memory and in speed, why those are different questions, and the evaluation that has to run before it ships.

A model update is a deploy whose failures are invisible to every deployment metric. What has to be re-validated even for a point release, the canary that watches output shape rather than error rate, and the fingerprint that makes a customer report investigable.

The conversion is the easy part and the evaluation is the project. What calibration data does and why yours should come from production, the per-category evaluation that catches what an average hides, and the parts of the model that should not be quantized at all.

Model evaluations answer whether the weights are good and deployment evaluations answer whether your configuration serves them correctly, which is a different and more common failure. What to sample, why category averages hide the regressions that matter, and the reference that makes a result mean something.

Benchmark rankings answer a question your product did not ask, and candidates usually differ more in what they cost to serve than in what they can do. The four axes in elimination order, and the deployment floor that rules candidates out before quality is discussed.