← Kernels & Compilers

Advanced

Shared Memory and Bank Conflicts

Shared memory lets threads in a block exchange and reuse data. Bank conflicts occur when an access needs multiple distinct words from the same bank.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

GPU Memory Hierarchy Memory Coalescing Tiled Matrix Multiplication Occupancy and Register Pressure CUTLASS and Tensor Core Kernels FlashAttention Internals

LESSONS THAT TEACH THIS

Kernels and the MachineCoalescing is the whole of memory performance, stated once15 min→Kernels and the MachineShared memory is a manual cache with a bank conflict waiting in it17 min→Kernels and the MachineA tiled matrix multiply connects ownership, reuse and synchronization18 min→

PRACTICE THIS IN REAL QUESTIONS

CUDA, Triton & Kernel EngineeringExplain shared memory bank conflicts with the bank arithmetic, show a kernel that has them, and fix it with padding.→CUDA, Triton & Kernel EngineeringTranspose a large matrix at close to copy bandwidth. Why is the naive version slow, and what does each fix buy?→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→CUDA, Triton & Kernel EngineeringWhat changed between FlashAttention 1, 2 and 3, and why did each change buy what it did?→GPU & Accelerator ArchitectureA kernel runs at 25% occupancy. Is that a problem? Walk me through what occupancy buys and when you would leave it low on purpose.→CUDA, Triton & Kernel EngineeringHow do you overlap host-to-device transfers with compute, and what has to be true before the overlap actually happens?→