Weight Formats: FP8 Blocks, MXFP4 and AWQ

TL;DR: Bytes per parameter is the number that matters, and it comes from the format's element width plus its scale overhead. Block-scaled FP8 in the e4m3 encoding stores one byte per weight and one FP32 scale per tile, commonly 128 by 128, so the overhead is 32 bits across 16,384 values and the effective cost is one byte. MXFP4, the microscaling four-bit format, stores a four-bit value per weight plus one eight-bit shared exponent per block of 32, giving 4.25 bits or about 0.53 bytes. Integer schemes such as AWQ store four-bit weights with a group-wise scale and zero point, landing in the same size range with a different error behaviour. As of September 2026 GLM-5.3 and DeepSeek V4 ship FP8 e4m3 and Kimi K3 ships its expert weights in MXFP4 with MXFP8 activations. The practical rule is to serve what was released, because that is the configuration the authors evaluated, and to check that your engine and GPU generation execute the format natively rather than emulating it.

Bytes per parameter, derived

bf16
  16 bits per weight, no scales
  2.00 bytes per parameter

FP8 e4m3 with 128x128 block scaling
  8 bits per weight
  one FP32 scale per 128 x 128 = 16,384 weights: 32 / 16,384 = 0.00195 bits per weight
  8.002 bits = 1.00 bytes per parameter

MXFP4 (microscaling, four-bit elements with a shared exponent per block of 32)
  4 bits per weight
  one 8-bit shared exponent per 32 weights: 8 / 32 = 0.25 bits per weight
  4.25 bits = 0.53125 bytes per parameter

AWQ or GPTQ style int4 with group size 128
  4 bits per weight
  a scale and a zero point per group of 128, in fp16: (16 + 16) / 128 = 0.25 bits per weight
  4.25 bits = 0.53125 bytes per parameter
sanity: the two four-bit schemes cost the same bytes and differ in how the error is
        distributed, so the choice between them is a quality question rather than a
        capacity one

What that does to the deployments in question

Kimi K3, 2.8T total, experts in MXFP4 and the rest in higher precision
  assume, as its published description implies, that the large majority of parameters are
    expert weights; take 90% experts and 10% other for an estimate
  experts:  2.8e12 x 0.90 x 0.53125 = 1,339 GB
  other:    2.8e12 x 0.10 x 1.0     = 280 GB
  estimated total                    = 1,619 GB
  if it were bf16 throughout:  2.8e12 x 2 = 5,600 GB
  ratio: 3.5x smaller

  GPUs needed for weights alone
    on B300 at 288 GB: 1,619 / 288 = 5.6 -> 8 GPUs, one node
    in bf16 it would be 5,600 / 288 = 19.4 -> 24 GPUs, three nodes
sanity: this reproduces the vLLM project's own statement that Kimi K3 needs at least one
        eight-GPU B300 node, which is a useful check that the estimate is in the right place
        and that the 90 percent split is not far off

GLM-5.3, 753B in FP8 e4m3
  753e9 x 1.0 = 753 GB, plus about 1% for scales = about 760 GB
  in bf16: 1,506 GB, which is 6 B300s of weights before any KV cache

Where the formats differ, beyond size

That bottom-right box is the trap. A four-bit format always saves memory, because the bytes on the GPU really are fewer. It only saves time when the hardware multiplies in that format. On a part without native support the engine dequantizes into a supported type before the matmul, so the deployment gets the capacity benefit and none of the throughput benefit, and a benchmark run on the wrong generation will mislead you about both.

The quality question, handled honestly

Numerics: FP32, BF16, FP8 and FP4 covers the representational side. For a deployment the practical position is:

• Serving the released format needs no quality argument from you. The authors evaluated it and published the results.
• Requantizing to something smaller needs your own evaluation, on your own traffic, because published benchmarks will not tell you whether your specific prompts degrade.
• KV cache quantization is a separate decision from weight quantization. Setting an FP8 KV cache halves cache bytes per token and has its own quality effect, and it can be enabled or not independently.
• The failure mode to look for is not a uniform accuracy drop but a task-specific one: long-context reasoning and code generation tend to degrade before short-answer tasks do.

What interviewers are listening for

Bytes per parameter, derived rather than remembered, including the scale overhead. The second signal is the native-versus-emulated distinction, because it is the difference between a format that speeds things up and one that only makes them fit. The third is the position on requantizing: an engineer who says "we serve what was released unless we have a measured reason, and if we requantize we evaluate on our own traffic" is describing how this is actually done, while one who reaches for the smallest format available is describing how it goes wrong.

Key takeaways

• Bytes per parameter: bf16 2.00, block-scaled FP8 1.00, MXFP4 0.53125, int4 with group 128 also 0.53125.
• MXFP4's 4.25 bits is four bits of value plus one eight-bit shared exponent per 32 weights.
• Kimi K3 at 2.8T works out to roughly 1,619 GB with MXFP4 experts against 5,600 GB in bf16, which is why it fits in one eight-GPU B300 node.
• A four-bit format always saves memory and only saves time where the hardware multiplies in it natively.
• Serve the released format by default; requantizing means owning the quality evaluation on your own traffic.