← ⚡ Kernels & Compilers
Advanced
CUTLASS and Tensor Core Kernels
CUTLASS is NVIDIA's template library for building GEMM-shaped kernels that run tensor cores at near cuBLAS speed while letting you change the data types, the tile shapes and the epilogue. Its hierarchy (device, kernel, collective mainloop, tile, instruction) is the vocabulary of every tensor-core discussion, and knowing when it beats calling cuBLAS or writing Triton is the judgment question kernel interviews end on.
Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
GPU & Accelerator ArchitectureWhat is a tensor core, and what does a kernel have to do to actually use one?→CUDA, Triton & Kernel EngineeringcuBLAS already gives you a fast GEMM. What does CUTLASS give you that it does not, and when is that worth it?→CUDA, Triton & Kernel EngineeringWhat changed between FlashAttention 1, 2 and 3, and why did each change buy what it did?→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→GPU & Accelerator ArchitectureWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.→GPU & Accelerator ArchitectureWhy are GPUs so much faster than CPUs for deep learning? Be specific about what the silicon is doing differently.→