In an int8 or fp8 GEMM, what exactly gets scaled, where does the dequantization happen, and what breaks if you get it wrong?

Derive when operand scales can be applied after a dot product and when they must follow its K blocks. Separate integer zero points, floating-point range, scale granularity and accumulator behavior.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Derive when operand scales can be applied after a dot product and when they must follow its K blocks. Separate integer zero points, floating-point range, scale granularity and accumulator behavior.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

Roofline calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Core

🧩 GPU & Accelerator ArchitectureSign in

Numerics: FP32, BF16, FP8 and FP4Floating-point formats trade range and precision for storage and compute cost. Understand exponent bits, significand bits, scaling and accumulation before choosing a format.

Foundational

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQDerive the storage cost of low-precision tensors, including scales, zero points and higher-precision exceptions. Distinguish formats from quantization methods, understand fused dequantization, and test whether a smaller checkpoint actually improves serving capacity without unacceptable quality loss.

Advanced

🚀 Inference & Serving🔒 Premium

Quantization for InferenceQuantization reduces weight, activation or cache precision. Calculate payload plus scales, estimate whole-step speedup, and test task quality before choosing a smaller representation.

Foundational

🧮 Napkin Math & Capacity

Training FLOPs: 6NDDerive the six from one forward and two backward matrix products. Count training tokens, attention and recomputation consistently before turning model work into a schedule.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackA mixture-of-experts layer is 256 small matmuls. Why is a loop over cuBLAS calls slow, and what does a grouped GEMM do instead?Next in this trackSelect the top k logits from a 128,000-token vocabulary on the GPU. What shape does the kernel take and why not just sort?Next in this trackYou added three local variables to a working kernel and it got 30 percent slower. Explain what happened and how you would confirm it.

FEDITOR'S NOTE

Write the dequantization convention explicitly, keep scales inside the reduction when they vary along K, and distinguish integer quantization from FP8 rounding. Verify the complete numerical path, including output scaling.

DISCUSSION · 0

No comments yet — be the first to share your approach.