tensor cores

Not 'more cores.' The real answers are a 10x memory system, a 16x matrix datapath, and a design that spends transistors on lanes and registers instead of on making one thread wait less. With the numbers for a two-socket server against one H100.

A tensor core multiplies small matrix tiles in one instruction and is 16x faster than the regular lanes, but only for dense matmul at the right precision, with the right shapes and layouts. What the instruction looks like, what it refuses, and how to tell from a profile whether you are on it.

A linear layer at batch 1 runs at a third of a percent of peak and at batch 4,096 near 70%. The curve between has two regions and a sawtooth: the memory-bound slope where intensity equals M, the plateau where the tensor cores are the limit, and the dips where the tile count does not divide the SM count.

The naive kernel reads two bytes per multiply-add; a 128x128 tile with register blocking reads a few hundredths of that. Each step (shared-memory tiles, register blocking, vectorized loads, double buffering, tensor cores) with its intensity arithmetic and speedup, and where a hand kernel stops and cuBLAS begins.

Each version fixed a different bottleneck: version 1 fixed memory traffic, version 2 fixed non-matmul work and parallelism, version 3 fixed the fact that softmax and matmul were waiting for each other on Hopper. The arithmetic that shows why a softmax costing one percent of the FLOPs can cost sixty percent of the time.

Three mantissa bits and a range of 448 make fp8 unusable with one scale per tensor. The arithmetic of what an outlier channel destroys, the 128-element block scaling and fp32 promotion that the DeepSeek-V3 report used, the ops that stayed in bf16, and what the 2× peak bought in practice.