A take-home gives you a working layernorm kernel at a tenth of memory bandwidth. Make it fast and justify every change.

Normalization reads a row and writes a row, so a copy sets the ceiling and everything else is overhead you can remove. The six changes in the order a reviewer wants them, what each is worth, and the one that is a correctness fix rather than a speed fix, with the measured error that proves it.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Core

🧮 Napkin Math & CapacitySign in

Arithmetic Intensity by OperationThe roofline says a kernel's ceiling is set by its FLOPs per byte against the hardware's ridge point. This page does the FLOPs-per-byte arithmetic for the operations an LLM actually runs (decode at several batch sizes, prefill, the attention score matmul with and without FlashAttention, LayerNorm, an embedding lookup) so the reader can place any of them on the roofline from first principles and say which lever moves it. The numbers explain why a serving fleet's GPUs report 30% utilization while fully loaded.

Foundational

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQOpen-weights models now ship pre-quantized, and the format is part of the release rather than something you choose afterwards. Block-scaled FP8 gives one byte per parameter with a scale per tile. MXFP4 gives about 0.53 bytes by pairing four-bit values with a shared exponent every 32 elements. Integer schemes like AWQ reach similar sizes with a different error profile. What decides a deployment is not which is most accurate in the abstract but which one the model was released and evaluated in, and which one your engine and hardware can execute natively.

Core

🧩 GPU & Accelerator ArchitectureSign in

Numerics: FP32, BF16, FP8 and FP4Every number format is a trade between range (exponent bits), precision (mantissa bits) and throughput (fewer bits, more values per cycle through the tensor cores). bf16 won training because it keeps fp32's range; fp8 splits into E4M3 for precision and E5M2 for range and needs scaling factors; fp4 needs block scaling and careful outlier handling. Knowing which format goes where, and why accumulation stays fp32, is what the numerics question is really asking.

Foundational

⚡ Kernels & Compilers

CUDA Programming ModelCUDA splits a program into a host that allocates, copies and enqueues work, and a device that runs thousands of identical threads organized as a grid of blocks. Getting the split right, and knowing that a launch returns before the kernel runs, decides whether your first live-coding kernel produces a correct number or a silent zero.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement the verification step of speculative decoding, including the rollback of the KV cache after a rejection.Next in this trackHow do you overlap host-to-device transfers with compute, and what has to be true before the overlap actually happens?Next in this trackWhy do tensor cores accumulate in fp32 when the inputs are bf16, and why do optimizers keep fp32 master weights?

FEDITOR'S NOTE

Scored on computing the bandwidth ceiling before optimizing, on the ordering of the fixes, on knowing that fp32 accumulation is correctness rather than speed, and on reporting against the ceiling rather than against the starting point.

DISCUSSION · 0

No comments yet — be the first to share your approach.