Practice tests · 30 questions

CUDA, Triton & Kernel Engineering: the practice test

Coalescing, shared memory and bank conflicts, occupancy, fusion, tiled GEMM, FlashAttention internals, Triton, CUTLASS, Nsight profiling and torch.compile: the live-coding and take-home round at NVIDIA, Fireworks, Together and the labs' performance teams. This test drills exactly that: 10 easy, 10 medium and 10 hard questions, every one explained, every explanation linking into the worked material.

Set up your test

Topic

How confident are you feeling?

Questions

10 in this pool · about 7 min

Reveal answers

Sample questions, answered

easy · sample

What does memory coalescing mean for a CUDA kernel, and what happens without it?

The kernel compresses its inputs before loading them, so fewer bytes cross the bus

✓Adjacent threads in a warp read adjacent addresses, so one transaction serves the warp

Each thread reads a whole cache line for itself, so the L2 stays warm across the warp

Loads are issued from shared memory only, because global memory cannot be read directly

A warp's 32 threads issue a load together. When they touch 32 consecutive 4-byte words, the memory system serves the whole warp with a few 128-byte transactions. When they stride through memory (thread i reads row i of a row-major matrix, say), every thread touches a different line and the warp needs 32 transactions for the same useful data, so effective bandwidth collapses by an order of magnitude. Choosing the thread-to-data mapping so that the fast index of the data matches the thread index is the first rule of every kernel.

easy · sample

Shared memory is divided into 32 banks. What is a bank conflict?

Two thread blocks on the same SM requesting the same shared memory allocation in the same cycle

A thread reading shared memory that another thread wrote without a barrier in between

✓Threads in a warp hitting different words in the same bank, which serializes them

The L1 cache and shared memory competing for the same physical SRAM on the SM

Shared memory serves one 4-byte word per bank per cycle, with consecutive words in consecutive banks. If several threads in a warp hit different words that map to the same bank, the hardware replays the access once per conflicting word, so a 32-way conflict takes 32 cycles instead of one. Column-wise access of a row-major tile with a width that is a multiple of 32 is the classic case; padding the tile by one element, or swizzling the layout, spreads the accesses across banks. Reading the same word is a broadcast and costs nothing.

Go deeper than the quiz

A practice test measures recall. The material it draws from teaches the reasoning: