hardNewNVIDIAFireworks

You are moving a model to fp4 inference on Blackwell. What breaks first, and how would you measure whether the result is acceptable?

Four bits give eight magnitudes per sign, and the largest step between them is 50%. Block scaling rescues most of the tensor; a single outlier in a block of sixteen flushes its neighbors to zero. Where the precision goes, which layers fail first, and the measurements that separate acceptable from broken.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Core

🧩 GPU & Accelerator ArchitectureSign in

Numerics: FP32, BF16, FP8 and FP4Every number format is a trade between range (exponent bits), precision (mantissa bits) and throughput (fewer bits, more values per cycle through the tensor cores). bf16 won training because it keeps fp32's range; fp8 splits into E4M3 for precision and E5M2 for range and needs scaling factors; fp4 needs block scaling and careful outlier handling. Knowing which format goes where, and why accumulation stays fp32, is what the numerics question is really asking.

Advanced

🚀 Inference & Serving🔒 Premium

Quantization for InferenceQuantization stores weights, and sometimes activations and the KV cache, in fewer bits, which cuts the bytes a decode step has to stream and the memory a model occupies. Weight-only int4 (GPTQ, AWQ) is a capacity and single-stream latency play; fp8 for weights and activations (W8A8) doubles tensor-core throughput and helps prefill and large batch; fp8 KV cache doubles context per GPU. Each has an accuracy cost you measure rather than assume, and knowing which one to reach for from the bottleneck is the interview question.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

GPU Generations: A100 to BlackwellFour NVIDIA generations are in fleets at once, and interviewers ask what each one changed, not what it is called. A100 to H100 added fp8 and tripled compute; H200 kept the die and grew memory; B200 doubled everything and added fp4; B300 stacked more HBM and cut fp64. This page carries the dense numbers for each, what they did to training and serving, and the marketing traps (sparse peaks, 192 versus 180 GB, die counting) that trip candidates. Dated September 2026.

Foundational

🧮 Open Weights & Serving Engines

Serving Benchmarks That Do Not LieMost published serving numbers are not comparable to each other and not predictive of production, because they differ in the input distribution, the concurrency, whether the cache was warm, and which of several very different metrics is being reported. A benchmark that supports a decision has to fix all four, report a distribution rather than a mean, and be run against the traffic shape you actually serve. The single most useful discipline is to compute the bandwidth bound first, so you know what fraction of the possible you achieved.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign a non-blocking network fabric for 512 H100s. How many switches, how are they wired, and where does the NVLink domain end?Next in this trackExplain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?Next in this trackAn Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?

FEDITOR'S NOTE

Scored on the candidate knowing what an fp4 number can represent, showing with a worked block how outliers destroy neighbors, naming the layers that fail first, and proposing a measurement that is more sensitive than a benchmark score.

DISCUSSION · 0

No comments yet — be the first to share your approach.