Free learning guide · reviewed 2026-09-20
You see an RTX 4090 in a desktop, an H100 in a cloud menu, and a GB200 NVL72 in the news. They all help run AI, but the names describe very different amounts of hardware.
Start by placing each name: what kind of machine is it, how much can it hold, and what work is it built for? The specifications become easier once you have that picture.
A GPU is a processor that performs many suitable calculations in parallel. It also needs memory to hold the model and the data being worked on.
A server is a computer containing CPUs, memory, networking and one or more accelerators. A rack holds servers and other equipment. Some AI racks connect many GPUs through a fast internal network so they can cooperate on a large job.
For example, B200 is a GPU name. DGX B200 is a server containing eight. GB200 NVL72 is a 72-GPU rack system with Grace CPUs. “72” counts GPUs; it is not a generation number.
8 × 180 GB = 1,440 GB total
“4 × RTX 5090” means four cards. It does not name a special GPU model. DGX H200 and DGX B300 name server products; DGX Spark and the GB300-based DGX Station name desktop systems. Always read the complete product name.
Read from the machines you might use at a desk toward the systems that fill a data center. The last two groups explain shared-memory computers and other accelerator designs. These are useful landmarks, not every GPU ever made.
Open any card for its role and sources. Memory figures describe the stated device or system. A documented product is not a claim that a provider has stock today.
A useful place to learn local inference, image generation and small training jobs. The model must fit the card, and the runtime must support it.
24 GB GDDR6X per card
An older 24 GB card can still run a model that fits. Moving to a 4090 keeps the same stock memory capacity; generation and capacity answer different questions.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
24 GB GDDR6X per card
A 24 GB desktop GPU. Two cards need software that can split the model; installing them does not create one ordinary 48 GB allocation.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
32 GB GDDR7 per accelerator
A 32 GB desktop GPU. More memory than a stock 3090 or 4090, but still much less per device than the large server accelerators below.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
24 GB per card
An AMD desktop GPU. Check support for the exact card, operating system and runtime before choosing a model recipe.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
12 GB GDDR6 per card
A 12 GB desktop GPU using an Intel-compatible software path. A CUDA-only example needs adaptation or a different runtime.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
16 GB GDDR6 per card
A newer AMD desktop generation than RX 7900 XTX, with less memory on this particular card. Newer architecture does not necessarily mean more room for weights.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
Professional graphics cards can hold larger models without splitting them across devices. Read the full name: A6000, 6000 Ada and PRO 6000 Blackwell are different generations.
48 GB GDDR6 per card
A 48 GB professional GPU. The extra memory can matter more than the generation when your first problem is fitting the model.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
48 GB GDDR6 per card
Also 48 GB, with a newer architecture than A6000. Similar names and equal capacity do not imply equal application performance.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
96 GB GDDR7 ECC per accelerator
The workstation and server editions both advertise 96 GB. Cooling, power and deployment support depend on the edition. Several cards still need an explicit multi-GPU plan.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
These cards are worth considering when each model fits on one device. More replicas may serve your users better than one large multi-GPU job.
16 GB GDDR6 per accelerator
A small 16 GB server GPU. Useful to recognize in older cloud machines; check whether your software still supports its architecture.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
24 GB GDDR6 per accelerator
A 24 GB server card for jobs that fit on a single device. Its memory capacity is very different from an A100.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
24 GB GDDR6 per accelerator
A 24 GB server GPU used for inference and video workloads. The L4 name in networking can also mean layer four; context matters.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
48 GB GDDR6 per accelerator
A 48 GB server GPU for inference, graphics and suitable training work. Check the interconnect before planning to split a large model across cards.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
High-bandwidth memory and fast connections between GPUs help when one card is no longer enough. The server configuration is part of the choice.
40 or 80 GB per GPU; specify the variant
Comes in 40 GB and 80 GB versions, with different form factors. An answer that says only A100 has not yet specified its memory budget.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
80 GB (SXM/PCIe) or 94 GB (NVL) per GPU
A generation after A100. SXM, PCIe and NVL variants differ in memory, bandwidth and connections. Ask which one before using a specification.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
141 GB HBM3e per accelerator
Still Hopper, with more memory capacity and bandwidth than H100 SXM. That can help memory-limited work without giving every workload the same speedup.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
180 GB HBM3e per accelerator
A newer large-model GPU. Read B200 as a GPU name; DGX B200 is an eight-GPU system.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
288 GB HBM3e per accelerator
More memory per GPU than the B200 HGX configuration in our reference. That may fit a job on fewer devices; measure speed separately.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
192 GB HBM3 per accelerator
A large-memory AMD data-center GPU. ROCm is the software ecosystem to check when moving a workload from NVIDIA.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
256 GB HBM3e per accelerator
A later CDNA 3 accelerator with more memory than MI300X. Keep model fit and runtime support in the same comparison.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
288 GB HBM3e per accelerator
A newer AMD data-center generation. Compare it with NVIDIA on the same model, precision, request load and response-time target.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
432 GB HBM4 per GPU
Launched July 2026; confirm system configuration and availability
The newer AMD family used in Helios racks. The GPU and the rack have different memory and communication boundaries; read each specification at its stated scale.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
These names describe complete systems. Software divides the model or the work among their GPUs. A rack specification is much larger in scope than a card specification.
640 GB total across 8 GPUs
Eight H100 GPUs in one server. The total memory is distributed across the devices, and NVLink/NVSwitch connects their work.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
1,128 GB total across 8 GPUs
Eight H200 GPUs in one server. The total memory is distributed across the devices, and NVLink/NVSwitch connects their work.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
1,440 GB total across 8 GPUs
Eight B200 GPUs in one server. Its 1,440 GB total is distributed across eight devices; placement and communication still matter.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
8 GPUs; see system specifications
An eight-GPU Blackwell Ultra server. Keep the server total separate from the memory of one B300 GPU.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
186 GB HBM3e per accelerator in the linked cloud configuration
A connected rack with 72 GPUs and Grace CPUs. NVLink carries communication within the GPU group. The rack is a different unit of comparison from one B200.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
279 GB HBM3e per accelerator in the linked cloud configuration
The Blackwell Ultra rack family. The GB300 superchip, DGX Station and GB300 NVL72 rack are different product boundaries.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
288 GB HBM4 per Rubin GPU; 20.7 TB published rack total
Vendor reports production and system shipments; local availability unverified
72 Rubin GPUs and 36 Vera CPUs. NVIDIA describes production ramp and shipments; that does not establish stock or rental capacity in your region.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
72 GPUs; memory is distributed across devices
Vendor reports production; provider availability unverified
A rack system containing 72 MI455X GPUs and 18 EPYC CPUs. AMD reports production; announced customer rollout dates do not establish rental availability.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
The CPU and GPU can access a shared address space. Check which physical memory holds the data: capacity alone does not tell you how quickly the GPU can read it.
Unified system memory; capacity varies by configuration
Mac Studio and MacBook Pro are computers, not plug-in GPU cards. M1/M2/M3 Ultra and M3/M4 Max identify different chips and configurations. MLX and Metal are useful names to recognize for local AI.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
96 GB base; configurations up to 512 GB shared memory
Announced / preorder; September 22 availability date is still ahead of this review
Apple lists up to 512 GB of unified memory. Its announcement schedules availability from September 22, after this review date; check the chosen configuration separately.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
128 GB shared system memory
A compact development computer with 128 GB shared by CPU and GPU. That capacity can be useful locally, but does not give it the bandwidth of a large HBM server GPU.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
252 GB GPU HBM3e + 496 GB CPU LPDDR5X
Documented system; confirm OEM configuration and availability
A desktop superchip system with one Blackwell Ultra GPU and a Grace CPU. Its current specification lists GPU and CPU memory separately; coherent access does not make both equally fast.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
Google TPUs, AWS chips and Tenstorrent processors perform AI computation without being NVIDIA GPUs. Choose the device and its supported software together.
32 GB HBM per accelerator
A Google AI accelerator accessed through Cloud TPU. TPU chip counts and slices describe a different system from an eight-GPU NVIDIA server.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
192 GiB HBM per accelerator
A newer Google TPU family with more memory per chip than v6e. Use the exact TPU version and slice configuration in a deployment plan.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
96 GiB HBM per accelerator
An AWS accelerator used through the Neuron software stack. EC2 instance memory and per-chip memory are different scopes.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
144 GiB HBM per accelerator
A newer AWS accelerator. Check the supported model recipe, compiler and instance topology together.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
32 GiB HBM per accelerator
An inference accelerator, also using Neuron. A CUDA deployment does not transfer unchanged.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-12.
24 GB GDDR6 per board across 2 processors
A PCIe board with two Tensix processors. Its software and supported model implementations differ from CUDA GPUs; the board total is split between processors.
Orientation reviewed 2026-09-20. Linked specification evidence reviewed 2026-09-20.
What about older cards and modified cards? GTX 1080 Ti and RTX 2080 Ti belong to older generations. Their usefulness depends on model fit and current runtime support. A label such as “RTX 4090 48 GB mod” describes a modified card; NVIDIA’s standard RTX 4090 specification is 24 GB. Verify that particular board instead of applying stock specifications to it.
The broader specification catalogue includes additional variants, such as RTX 5070/5080, A30, H200 NVL, MI350X and announced TPU 8t/8i. A BlueField DPU handles infrastructure work such as networking and storage offload; it belongs alongside the model accelerator in a system diagram.
Follow each row from earlier to later. The rows are separate product lines. A workstation GPU and a rack do not become comparable just because both use Blackwell chips.
At the newest end of this map: Rubin follows Blackwell in NVIDIA’s data-center line. AMD’s MI455X and Helios extend its data-center line beyond MI350. Both vendors report production of their new rack families. For a purchase or a rental, the next check is the exact provider, configuration and region.
Older hardware remains useful. A model that fits on a 24 GB RTX 3090 can be a sensible learning project even when newer cards exist. An upgrade earns its place when it fixes a measured limit: memory, response time, throughput, power use or software support.
VRAM means the GPU’s memory. GB measures its capacity. The model’s learned numbers, or weights, must live somewhere before the GPU can use them. An 8B model has about eight billion parameters; at 16 bits each, its weights alone need about 16 GB.
70 billion × 4 ÷ 8 = 35 GB for uniformly stored weights alone. Actual quantized files add metadata and may keep some values at higher precision.
Weights exceed capacity by 11 GB.
Weights exceed capacity by 3 GB.
13 GB remains before request state and runtime buffers.
45 GB remains before request state and runtime buffers.
106 GB remains before request state and runtime buffers.
145 GB remains before request state and runtime buffers.
253 GB remains before request state and runtime buffers.
A 70B model at uniform 4-bit starts at 35 GB. Two 24 GB cards have 48 GB in total, but each still has a 24 GB limit. The runtime must split the model and move data between devices.
Add request state and safety margin in the full memory planner →
HBM and GDDR are memory technologies used by accelerators. Shared system memory, as in a Mac or DGX Spark, has a different boundary: the operating system, CPU and GPU share resources. Reserve room for the rest of the computer as well as the model.
A larger memory can hold a larger model or more request state. It can also avoid splitting a model across GPUs. Capacity is measured in GB or GiB; check the units before doing arithmetic.
Bandwidth measures bytes moved per second. A GPU may spend much of a small-batch generation step reading weights. Faster memory can help that workload even when arithmetic units are waiting.
Training and processing large batches can put the calculation units to work. Peak FLOPS need a format attached: FP4, FP8 and BF16 use different numbers of bits. A sparse peak also assumes a particular pattern of zeros.
If the model is divided across devices, they exchange data during execution. NVLink, PCIe and the server network are different parts of that path. More GPUs can add communication as well as compute.
A useful speed comparison fixes the model and its quality, then measures time to first token, time between output tokens, and completed work at the target load. A gaming frame-rate chart or an advertised AI TOPS number cannot answer those questions by itself.
Start with a small supported model on a machine you already have, or a hosted endpoint. A compatible desktop GPU, Mac or small cloud GPU is enough to learn the request path. Learn model sizes and formats first.
Check the model’s actual storage and runtime support. Desktop cards and shared-memory computers are natural candidates. Move to more memory when the full request no longer fits; measure responsiveness before buying.
Compare small server GPUs when a model fits on one device. Compare H100/H200, Blackwell or AMD Instinct when the model or load needs more. One large rack is not required merely because a service has many users. Work through accelerator selection.
Fine-tuning a small adapter and training every parameter need very different amounts of memory. Start with the training method, then budget weights, gradients, optimizer state and saved activations. Large multi-GPU jobs also need an efficient communication plan. Compare the memory requirements.
No. Both have 24 GB. A generation change can improve other capabilities while leaving the capacity unchanged.
No. The runtime must support distributing that model, each device must fit its assigned work, and you still need space for request state and buffers. The link between cards can affect speed.
No. It describes a rack system containing 72 GPUs and Grace CPUs. Read per-GPU and whole-system figures separately.
Each landmark links to its primary evidence. This orientation was reviewed on 2026-09-20; the detailed profiles retain their own specification review dates. Product announcements and availability are labelled separately. The memory exercise is arithmetic, and the workload suggestions are starting points for evaluation. We have not benchmarked these machines.
In lessons and answers, tap a GPU name with a dotted underline to see its family, memory boundary and place on this map. That explanation comes from the same reference, so updates can reach readers wherever the name appears.
