A take-home gives you a working layernorm kernel at a tenth of memory bandwidth. Make it fast and justify every change.

Build a numerical reference, count the traffic and investigate a row-wise fused kernel. Separate bandwidth lower bounds from measured performance.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Build a numerical reference, count the traffic and investigate a row-wise fused kernel. Separate bandwidth lower bounds from measured performance.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

Roofline calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Core

🧮 Napkin Math & CapacitySign in

Arithmetic Intensity by OperationCount the FLOPs and memory transfers of a projection, attention and a residual add. Learn where the batch-size shortcut works, where it breaks, and which hardware roof applies.

Advanced

⚡ Kernels & Compilers🔒 Premium

Kernel Correctness ContractsA fast GPU kernel is useful only for inputs it handles correctly. Define shapes, strides, dtypes, numerical tolerance, storage ownership and synchronization before timing a replacement.

Foundational

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQDerive the storage cost of low-precision tensors, including scales, zero points and higher-precision exceptions. Distinguish formats from quantization methods, understand fused dequantization, and test whether a smaller checkpoint actually improves serving capacity without unacceptable quality loss.

Core

🧩 GPU & Accelerator ArchitectureSign in

Numerics: FP32, BF16, FP8 and FP4Floating-point formats trade range and precision for storage and compute cost. Understand exponent bits, significand bits, scaling and accumulation before choosing a format.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement the verification step of speculative decoding, including the rollback of the KV cache after a rejection.Next in this trackHow do you overlap host-to-device transfers with compute, and what has to be true before the overlap actually happens?Next in this trackWhy do tensor cores accumulate in fp32 when the inputs are bf16, and why do optimizers keep fp32 master weights?

FEDITOR'S NOTE

Establish correctness before optimizing. Explain row ownership, reduction precision and register pressure without inventing benchmark results.

DISCUSSION · 0

No comments yet — be the first to share your approach.