← Open-Weights Models & Serving Engines / 44

mediumNewNVIDIABasetenTogether AI

Which GPUs can run an FP8 or AWQ checkpoint, and what do you actually get on an older card?

Map a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Map a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQDerive the storage cost of low-precision tensors, including scales, zero points and higher-precision exceptions. Distinguish formats from quantization methods, understand fused dequantization, and test whether a smaller checkpoint actually improves serving capacity without unacceptable quality loss.

🚀 Inference & Serving🔒 Premium

Quantization for InferenceQuantization reduces weight, activation or cache precision. Calculate payload plus scales, estimate whole-step speedup, and test task quality before choosing a smaller representation.

🧩 GPU & Accelerator ArchitectureSign in

Numerics: FP32, BF16, FP8 and FP4Floating-point formats trade range and precision for storage and compute cost. Understand exponent bits, significand bits, scaling and accumulation before choosing a format.

🧩 GPU & Accelerator Architecture🔒 Premium

GPU Generations: A100 to BlackwellGPU generations change memory, arithmetic support and interconnect capacity at different rates. Compare the specification that limits your workload rather than the generation name.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYour model now serves 6,568 tokens per second. What does a million tokens cost, and what do you hand the gateway and eval teams?Next in this trackYou need product copy for a 150-item catalog by tomorrow. Batch job or the online endpoint, and how do you deliver it?Next in this trackDeploy one open-weights model on Kubernetes with vLLM. What goes in the manifest, and what breaks first?

DISCUSSION · 0

No comments yet — be the first to share your approach.