RTX PRO 6000 Blackwell Server Edition
96 GB of GDDR7 makes this an interesting inference and mixed AI/graphics card. Its memory capacity sits above H100, but its bandwidth does not.
GDDR7
Published peak, not measured application throughput
RTX 6000 Ada, RTX PRO 6000 Blackwell Workstation, Max-Q and Server Edition are separate listings. Cooling, power and supported deployment features matter as much as the shortened “6000” name.
When this is a sensible choice
Start here if…
Try it for models needing more than 48 GB, video pipelines, rendering and enterprise consolidation. A workload using one card avoids many of the topology questions of a sharded replica.
Choose another configuration if…
For bandwidth-heavy decode or tightly coupled training, compare HBM and a purpose-built GPU fabric. Equal memory capacity is not equal service capacity.
Specifications with their boundaries attached
- Architecture
- Blackwell
- Memory
- 96 GB GDDR7
- Memory bandwidth
- 1.6 TB/s per accelerator
- Peak compute
- GCP dense: 467.8 TFLOPS BF16; 935.6 FP8; 1,871.2 FP4
- Scale-up interconnect
- PCIe platform; verify peer-to-peer topology, not an HGX NVSwitch assumption
- Host attachment
- PCIe 5.0 x16
- Power
- Up to 600 W, configurable
- Partitioning
- Universal MIG; verify available profiles
- Catalogue status
- Documented product
Compute figures are theoretical peaks at the stated precision. Dense and structured-sparse rates must not be mixed. Bandwidth labelled bidirectional combines both directions. See the source documents.
Three bandwidths, three different jobs
PCIe platform; verify peer-to-peer topology, not an HGX NVSwitch assumption
InfiniBand, RoCE, EFA or provider-specific transport
Where it appears in provider documentation
| Provider / machine | Network scope | What changes the decision |
|---|---|---|
AWS G7e ↗ | Check the selected instance size | RTX PRO 6000 Blackwell Server Edition. Do not substitute workstation power or clock specifications. |
Google Cloud G2 / G4 ↗ | VM-size-specific network | G2 uses L4; G4 uses RTX PRO 6000. Check exact GPU count and guest-visible memory. |
CoreWeave US East region catalogue ↗ | InfiniBand appears on designated instance types | The regional catalogue identifies which configurations carry InfiniBand. Availability and reservations remain zone-specific. |
Runpod GPU type catalogue ↗ | Host-specific; require evidence for multi-node fabric | Server, workstation and Max-Q RTX PRO names differ. A marketplace GPU listing is not a topology guarantee. |
Model fit and software support
These publisher or serving-engine documents mention this hardware family. They have not been reproduced on our machines.
No model-specific recipe in our reviewed set certifies this exact hardware. The memory calculator can narrow candidates, but it cannot establish software support. Read the model register.
What is the memory floor?
Start with total parameters, then add the memory the workload needs. This arithmetic does not certify a serving configuration. All output sizes below are decimal GB.
Override device capacity with the memory exposed by your allocation, particularly for cloud B300 and partitioned devices. The starting 32 GB budget and 15% reserve are editable teaching assumptions. They are not measurements for the selected model. Mixed-precision tensors, quantization scales, vision encoders and draft models can increase the weight payload.
Raw weights only
320B × 8 bits ÷ 8
Budget per device
96 GB × (1 − 15%)
Arithmetic lower bound
round up ((weights + 32) ÷ budget)
This assumes perfectly balanced sharding. A result of three does not prove that a three-device parallel layout is supported. Check layer/expert divisibility, actual allocatable memory, precision kernels and the fabric before renting.
320B is the model card’s total; 18B active is not its storage size. The vLLM recipe reports about 306 GiB for the native FP8 checkpoint. Hopper requires BF16 KV for this model; the documented ROCm path targets gfx950, not every Instinct GPU. Read the model source ↗
For full training, also budget gradients, optimizer states, activations and communication buffers. The inference weight estimate above is insufficient.
Tokens per second, TTFT and TPOT are not certified for this hardware in our reference. Use the benchmark checklist to compare an exact model, software revision and workload.
Nearby memory capacities, different tradeoffs
These are comparison candidates selected by memory capacity, not performance rankings or drop-in replacements.
RTX PRO 6000 Blackwell Workstation Edition
A desktop card with 96 GB of memory for local model work and graphics. Its 1.792 TB/s specification differs from the Server Edition’s 1.597 TB/s.
NVIDIA H100 NVL
H100 NVL is a 94 GB PCIe GPU often discussed as a two-card, 188 GB pair. Always count how many cards the listing includes.
NVIDIA A100 80 GB SXM
80 GB of HBM on the SXM4 form factor. A100 remains useful for BF16 training and supported inference, but cannot execute Hopper’s native FP8 Tensor Core path.
Sources and review scope
Manufacturer and cloud documentation checked 2026-09-12. We reviewed specifications and the stated product boundaries; we did not run training or serving benchmarks on this device. Cloud memory and availability can differ by configuration.
- NVIDIA RTX PRO 6000 Blackwell Server Edition ↗ · checked 2026-09-12
- Google Compute Engine GPU machine types ↗ · checked 2026-09-12
Cite this reference
AI Infra Interviews. RTX PRO 6000 Blackwell Server Edition: specifications and workload fit. Reviewed . https://aiinfrainterviews.com/hardware/rtx-pro-6000-blackwell
Include your access date when citing a changing specification. Link to the specification section for hardware figures or the explanation for a sizing or workload decision.
Research method and limits
We compile manufacturer specifications, cloud documentation, model cards and serving recipes. The reference preserves source units and distinguishes individual devices from nodes and racks. Conflicting figures and unknown fields remain labelled.
Our contribution is the comparison, unit reconciliation, worked arithmetic and workload explanation. Published peaks are vendor specifications. Calculator results are estimates under the displayed assumptions. Neither is a measurement from our own accelerator lab.
For a manufacturer’s specification, consult the original source documents. Cite our page when using its analysis, and retain primary-source attribution for underlying figures. This curated reference does not establish market share, live availability or a universal performance ranking.
Found a discrepancy? Send a correction with the page URL, exact variant, disputed figure and supporting primary document.