GPU device memory
GPU device memory stores allocations such as model weights, activations and output buffers. HBM and GDDR are memory technologies used by different GPU products.
How it works
Capacity determines whether allocations fit. Bandwidth describes the rate of moving bytes along a memory interface. A workload can fit comfortably while waiting on bandwidth, or have plenty of arithmetic capacity but fail allocation before it starts.
Work through an example
One billion two-byte weights require two billion bytes before runtime buffers and request state. That is 2 GB in decimal units, about 1.86 GiB. Use the memory exposed by the actual device or partition.
Where the term can mislead
CUDA global memory names an address space, while HBM names hardware. Managed allocations and cache behavior make those descriptions different.
Connect the idea
Take it into practice
Continue with the existing concept, apply it to an interview problem, or follow a course lesson. Access requirements are shown before you open a page.
New to GPUs? Start with the free execution-model concept.
Sources and scope
Examples here are teaching calculations, not measured GPU performance. Consult the linked documentation for the exact device, instruction and tool version. Source review: 25 October 2026.