NVIDIA Vera Rubin NVL72
NVIDIA's rack-scale successor to GB300: 72 Rubin GPUs and 36 Vera CPUs. Each GPU has 288 GB of HBM, the same figure as the B300 profile here, and published memory bandwidth rises to 19.2 TB/s.
New to GPU names? Place this hardware on the beginner’s map →
HBM4
Published peak, not measured application throughput
The 50 PFLOPS NVFP4 headline is a sparse inference figure; dense NVFP4 is 35, FP8 is 17.5 and BF16 is 4. Rack totals are sums across 72 GPUs and 36 CPUs, not one memory a tensor can see. NVIDIA rounds the rack bandwidth to 1,400 TB/s; 72 × 19.2 TB/s is 1,382.
When this is a sensible choice
Start here if…
Consider Rubin when decode is limited by weight and KV reads rather than by capacity. A step that reads 100 GB takes at least 5.2 ms at 19.2 TB/s, against 12.5 ms at B300's 8 TB/s. Confirm that your engine release has kernels for the part before planning on that floor.
Choose another configuration if…
If the model already fits a B300 or GB300 allocation and the work is BF16 prefill, the gain is smaller than the bandwidth step: 4 PFLOPS dense BF16 per GPU against 2.25 to 2.5. Price and delivery date belong in the decision, and this reference records neither.
Specifications with their boundaries attached
- Architecture
- Vera Rubin
- Memory
- 288 GB HBM4
- Memory bandwidth
- 19.2 TB/s per accelerator
- Peak compute
- 4,000 TFLOPS BF16; 17,500 FP8/FP6; 35,000 NVFP4, dense per GPU (NVFP4 inference headline of 50,000 is sparse)
- Scale-up interconnect
- NVLink: 3 TB/s per GPU; 216 TB/s across the 72-GPU rack
- Host attachment
- Vera Arm CPUs, 36 per rack; up to 54 TB LPDDR5X per rack
- Power
- Not listed on the checked page; use the system vendor's rack power plan
- Partitioning
- Not listed on the checked page
- Catalogue status
- NVIDIA lists it as available and ramping into full production; provider capacity unverified
Compute figures are theoretical peaks at the stated precision. Dense and structured-sparse rates must not be mixed. Bandwidth labelled bidirectional combines both directions. See the source documents.
Three bandwidths, three different jobs
NVLink: 3 TB/s per GPU; 216 TB/s across the 72-GPU rack
InfiniBand, RoCE, EFA or provider-specific transport
Where it appears in provider documentation
No provider configuration has been verified for this exact variant in this reference. Consult the linked manufacturer documentation; catalogue absence here is not evidence that the hardware is unavailable.
Model fit and software support
These publisher or serving-engine documents mention this hardware family. They have not been reproduced on our machines.
No model-specific recipe in our reviewed set certifies this exact hardware. The memory calculator can narrow candidates, but it cannot establish software support. Read the model register.
What is the memory floor?
Start with total parameters, then add the memory the workload needs. This arithmetic does not certify a serving configuration. All output sizes below are decimal GB.
Override device capacity with the memory exposed by your allocation, particularly for cloud B300 and partitioned devices. The starting 32 GB budget and 15% reserve are editable teaching assumptions. They are not measurements for the selected model. Mixed-precision tensors, quantization scales, vision encoders and draft models can increase the weight payload.
Raw weights only
320B × 8 bits ÷ 8
Budget per device
288 GB × (1 − 15%)
Arithmetic lower bound
round up ((weights + 32) ÷ budget)
This assumes perfectly balanced sharding. A result of three does not prove that a three-device parallel layout is supported. Check layer/expert divisibility, actual allocatable memory, precision kernels and the fabric before renting.
320B is the model card’s total; 18B active is not its storage size. The vLLM recipe reports about 306 GiB for the native FP8 checkpoint. Hopper requires BF16 KV for this model; the documented ROCm path targets gfx950, not every Instinct GPU. Read the model source ↗
For full training, also budget gradients, optimizer states, activations and communication buffers. The inference weight estimate above is insufficient.
Tokens per second, TTFT and TPOT are not certified for this hardware in our reference. Use the benchmark checklist to compare an exact model, software revision and workload.
Nearby memory capacities, different tradeoffs
These are comparison candidates selected by memory capacity, not performance rankings or drop-in replacements.
NVIDIA B300 HGX
Blackwell Ultra’s larger HBM budget helps large models and long context. A cloud allocation may expose less memory than the 288 GB physical-product figure.
AMD Instinct MI355X
288 GB of HBM on an AMD CDNA 4 accelerator. The memory is useful only after the model’s kernels and serving engine work on the ROCm target.
AMD Instinct MI350X
MI350X and MI355X share 288 GB of HBM3e and 8 TB/s peak bandwidth. MI350X has a lower board-power specification and lower compute peaks.
Sources and review scope
Manufacturer and cloud documentation checked 2026-11-06. We reviewed specifications and the stated product boundaries; we did not run training or serving benchmarks on this device. Cloud memory and availability can differ by configuration.
- NVIDIA Vera Rubin NVL72 specifications ↗ · checked 2026-11-06
Cite this reference
AI Infra Interviews. NVIDIA Vera Rubin NVL72: specifications and workload fit. Reviewed . https://aiinfrainterviews.com/hardware/rubin-nvl72
Include your access date when citing a changing specification. Link to the specification section for hardware figures or the explanation for a sizing or workload decision.
Research method and limits
We compile manufacturer specifications, cloud documentation, model cards and serving recipes. The reference preserves source units and distinguishes individual devices from nodes and racks. Conflicting figures and unknown fields remain labelled.
Our contribution is the comparison, unit reconciliation, worked arithmetic and workload explanation. Published peaks are vendor specifications. Calculator results are estimates under the displayed assumptions. Neither is a measurement from our own accelerator lab.
For a manufacturer’s specification, consult the original source documents. Cite our page when using its analysis, and retain primary-source attribution for underlying figures. This curated reference does not establish market share, live availability or a universal performance ranking.
Found a discrepancy? Send a correction with the page URL, exact variant, disputed figure and supporting primary document.