Advanced

CUTLASS and Tensor Core Kernels

CUTLASS provides components for building GPU matrix kernels. It is useful when the required precision, batching or fused operations need more control than a library call offers.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Tensor Cores and Matrix Units Tiled Matrix Multiplication Shared Memory and Bank Conflicts FlashAttention Internals Numerics: FP32, BF16, FP8 and FP4 Quantization for Inference

LESSONS THAT TEACH THIS

Kernels and the MachineA tiled matrix multiply connects ownership, reuse and synchronization18 min→Kernels and the MachineMap coordinates before moving bytes18 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureWhat is a tensor core, and what does a kernel have to do to actually use one?→CUDA, Triton & Kernel EngineeringcuBLAS already gives you a fast GEMM. What does CUTLASS give you that it does not, and when is that worth it?→CUDA, Triton & Kernel EngineeringWhat changed between FlashAttention 1, 2 and 3, and why did each change buy what it did?→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→GPU & Accelerator ArchitectureWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.→CUDA, Triton & Kernel EngineeringA GEMM launches 96 tiles into 80 execution slots. Should you split K?→