← 🧩 GPU & Accelerator Architecture
Core
Tensor Cores and Matrix Units
Tensor cores are fixed-function units that compute a small matrix multiply-accumulate per instruction, and they are where almost all of a modern GPU's FLOPS live: 989 dense bf16 TFLOPS on an H100 against about 67 from the general-purpose lanes. Only dense, well-shaped matrix multiplication at a supported precision can use them, which is why GEMMs reach peak and nothing else does, and why precision choices are throughput choices.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
LESSONS THAT TEACH THIS
Foundations of AI InfrastructureTensor cores are not a kind of memory, and the distinction decides your answer13 min→Training at ScaleMixed precision is a numerics contract, not a speed setting14 min→Inference EngineeringThe format that wins is the one with a kernel, not the one with the smallest file15 min→
PRACTICE THIS IN REAL QUESTIONS
GPU & Accelerator ArchitectureWhat is a tensor core, and what does a kernel have to do to actually use one?→GPU & Accelerator ArchitectureWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.→GPU & Accelerator ArchitectureWhy are GPUs so much faster than CPUs for deep learning? Be specific about what the silicon is doing differently.→CUDA, Triton & Kernel EngineeringTake a GEMM from naive to 70% of peak: the steps, the speedup at each, and the arithmetic that says why.→GPU & Accelerator ArchitectureExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?→Distributed Training & ParallelismWhy is fp8 training hard, and how did DeepSeek-V3 make it work?→