Sum 100 million floats on the GPU as fast as the hardware allows. Write the kernel and justify each step.

A sum reads every byte once and does one add per element, so the only question is whether you reach the bandwidth ceiling. The ceiling in milliseconds, four versions from an atomic per element to warp shuffles with sixteen loads in flight, and the bytes-in-flight math behind the last jump.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

💻 Coding for Infra🔒 Premium

Concurrency in Python, Go and C++Infrastructure code is concurrent by nature: a loader feeding a GPU, a gateway holding ten thousand streams, a controller reconciling a fleet. The coding screen tests whether you know which primitive fits which problem in the language you claim, and the three languages the field uses answer differently: Python has one interpreter lock and an event loop, Go has cheap goroutines and channels, C++ has threads, mutexes and atomics with no safety net. This page gives the model of each, works the favourite problems (a thread-safe LRU, a worker pool, a bounded fan-out) in each, and derives when threads, processes or async buy throughput.

Foundational

⚡ Kernels & Compilers

Kernel FusionAn elementwise or reduction kernel does a few FLOPs per byte and runs at HBM speed, so a chain of five of them costs five trips through HBM for work that needs one. Fusion collapses the chain into a single kernel that keeps intermediates in registers. It is the first lever for anything memory-bound, and knowing what it cannot fix (weight reads in decode, the GEMMs themselves) is what the interview is really testing.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Bandwidth-Bound Decode ThroughputBecause decode reads every weight once per step, its speed is a division: memory bandwidth over bytes per step. That one formula gives single-stream tokens per second for any model on any card, the batch curve that flattens at the ridge point, the effect of quantization, and the point where the KV cache rather than the weights becomes the thing being read. This page derives it, works it for a 70B model on four accelerators, and shows how to read a vendor throughput claim against it.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

NVLink, NVSwitch and PCIeInside a node, GPUs talk over NVLink at 900 GB/s per H100 through an NVSwitch fabric that gives all eight cards full bandwidth to each other; to the host and to anything outside the node they talk over PCIe at 64 GB/s or a 400 Gb/s NIC at 50 GB/s. That fifteen-fold gap is why tensor parallelism stays inside the eight-GPU domain, why NVL72 changes the serving math for MoE, and why the question "how many GPUs share an NVLink domain?" is the first thing to ask about any cluster.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackTranspose a large matrix at close to copy bandwidth. Why is the naive version slow, and what does each fix buy?Next in this trackWhat does torch.compile actually do to your model, and when does it fail to help?Next in this trackWhy do serving engines capture decode steps into CUDA graphs, and what does capture require of the rest of the code?

FEDITOR'S NOTE

Scored on computing the bandwidth ceiling first, on the three-level reduction structure (warp, block, grid) with atomics used once per block, and on knowing that work per thread rather than occupancy is what closes the last gap.

DISCUSSION · 0

No comments yet — be the first to share your approach.