GPU memory hierarchy
The GPU memory hierarchy combines thread registers, shared memory, caches and device memory. These levels differ in scope, capacity, latency and how software controls them.
How it works
A value may move from device memory through L2 and L1 into a register, then be written into shared memory for reuse by other threads. The actual route depends on the instruction and cache policy. Address spaces and physical storage levels are related but not identical.
Work through an example
For a tile reused eight times, eight source reads do not necessarily mean eight HBM fetches. Name the boundary before counting traffic: requested bytes, cache sectors or physical DRAM bytes.
Where the term can mislead
Bandwidth figures from different levels cannot be substituted into one calculation without changing the byte count to match.
Connect the idea
Take it into practice
Continue with the existing concept, apply it to an interview problem, or follow a course lesson. Access requirements are shown before you open a page.
New to GPUs? Start with the free execution-model concept.
Sources and scope
Examples here are teaching calculations, not measured GPU performance. Consult the linked documentation for the exact device, instruction and tool version. Source review: 25 October 2026.