Core

Tensor Cores and Matrix Units

Tensor cores accelerate supported matrix multiply-accumulate operations. Their performance depends on precision, matrix shape and how efficiently the kernel supplies data.

This concept is included with a free account. Sign in to continue reading.

free account · no card needed

RELATED CONCEPTS

GPU Execution Model Roofline Model Numerics: FP32, BF16, FP8 and FP4 Tiled Matrix Multiplication CUTLASS and Tensor Core Kernels TPU Architecture and Systolic Arrays

LESSONS THAT TEACH THIS

Foundations of AI InfrastructureFollow operands from memory to execution units13 min→Training at ScaleMixed precision is a numerics contract, not a speed setting14 min→Inference EngineeringA quantized format needs a compatible and measured execution path20 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureWhat is a tensor core, and what does a kernel have to do to actually use one?→GPU & Accelerator ArchitectureWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.→GPU & Accelerator ArchitectureWhy are GPUs so much faster than CPUs for deep learning? Be specific about what the silicon is doing differently.→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→GPU & Accelerator ArchitectureExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?→CUDA, Triton & Kernel EngineeringA GEMM launches 96 tiles into 80 execution slots. Should you split K?→