Advanced

GEMM Work Partitioning and Wave Quantization

Output tiles determine how much independent matrix work is available. Split-K and work-centric schedules can reduce an underfilled final wave, while adding partial-result reduction and workspace costs.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Tiled Matrix Multiplication Occupancy and Register Pressure Tensor Cores and Matrix Units Roofline Model Expert Parallelism for MoE

LESSONS THAT TEACH THIS

Kernels and the MachineBalance the grid before tuning the tile18 min→

PRACTICE THIS IN REAL QUESTIONS

CUDA, Triton & Kernel EngineeringA GEMM launches 96 tiles into 80 execution slots. Should you split K?→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→GPU & Accelerator ArchitectureWhat is a tensor core, and what does a kernel have to do to actually use one?→GPU & Accelerator ArchitectureWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.→CUDA, Triton & Kernel EngineeringSketch a paged attention kernel. What changes from FlashAttention once the KV cache is not contiguous?→CUDA, Triton & Kernel EngineeringWhat changed between FlashAttention 1, 2 and 3, and why did each change buy what it did?→