Calculators · free · formulas shown
Plan the budget. Check the capacity. Understand the maths.
Start with the inference economics workbench for a business budget, or use the technical tools to inspect memory, training and compute limits. Every tool shows its formula and assumptions. Hardware specs in the technical calculators are vendor dense peaks as of September 2026; model presets come from each model's published config.
Inference economics workbenchSeven free calculators for API budgets, fleet cost, peak capacity, API-versus-fleet break-even, cache reuse, human review and hardware ownership. Change the assumptions and save your scenario.Open →KV cache & memory-fit calculatorEstimate KV cache per token, sequence and batch for a model or custom architecture. Compare weights, cache and an adjustable reserve against total device memory, with formulas and placement limitations.Open →GPUs and time-to-train calculatorEstimate dense-model training FLOPs, time, GPU rental cost and a memory lower bound. Change the assumed MFU, fleet size and deadline; check the formula and its limits.Open →Cost per million tokens calculatorDerive the cost of a million output tokens from GPU price, replica size, batch and context, using bandwidth-bound and compute-bound decode throughput estimates, or plug in your measured tokens per second.Open →Roofline calculatorDraw the roofline for H100, H200, B200, MI300X, TPU and Trainium, find the ridge point, and place a kernel by its arithmetic intensity to see whether it is memory-bound or compute-bound and how far below the roof it sits.Open →
The spec sheet behind the tools
Dense tensor peaks only. Vendor headline numbers usually include 2:1 structured sparsity, which doubles them and which no dense GEMM reaches. The ridge point is the peak divided by memory bandwidth, the intensity a kernel needs before compute becomes the limit.
| Accelerator | Memory | Bandwidth | bf16 dense | fp8 dense | Ridge (bf16) | Note |
|---|---|---|---|---|---|---|
| A100 80GB SXM | 80 GB | 2.04 TB/s | 312 TFLOPS | n/a | 153 FLOP/B | No FP8 tensor cores; INT8 is 624 TOPS dense. |
| H100 80GB SXM | 80 GB | 3.35 TB/s | 989 TFLOPS | 1,979 TFLOPS | 295 FLOP/B | |
| H200 141GB SXM | 141 GB | 4.8 TB/s | 989 TFLOPS | 1,979 TFLOPS | 206 FLOP/B | Same compute die as H100; the upgrade is memory capacity and bandwidth. |
| B200 (HGX) | 180 GB | 8 TB/s | 2,250 TFLOPS | 4,500 TFLOPS | 281 FLOP/B | Announced at 192 GB; the HGX B200 datasheet lists 180 GB usable. Numbers here are dense; NVIDIA's headline figures include 2:1 sparsity. |
| B300 (Blackwell Ultra) | 288 GB | 8 TB/s | 2,250 TFLOPS | 4,500 TFLOPS | 281 FLOP/B | HGX B300 dense peaks: eight-GPU sparse BF16/FP8 totals divided by 16; dense FP4 total divided by eight. Do not substitute GB300 rack peaks. TDP is a planning assumption; verify the server configuration. |
| MI300X | 192 GB | 5.3 TB/s | 1,307 TFLOPS | 2,615 TFLOPS | 247 FLOP/B | Measured LLM inference throughput has trailed the spec sheet more than on NVIDIA parts; treat the peak as a ceiling, not a forecast. |
| MI325X | 256 GB | 6 TB/s | 1,307 TFLOPS | 2,615 TFLOPS | 218 FLOP/B | Announced at 288 GB, shipping at 256 GB. |
| MI355X | 288 GB | 8 TB/s | 2,500 TFLOPS | 5,000 TFLOPS | 313 FLOP/B | CDNA 4, shipping since Q3 2025. AMD's peaks are dense (it does not market sparsity figures). Liquid-cooled; the MI350X is the 1,000 W air-cooled sibling at about 2.3 PFLOPS bf16. |
| TPU v5e | 16 GB | 0.8589934592 TB/s | 197 TFLOPS | n/a | 229 FLOP/B | INT8 is 393 TOPS. Pods of 256 chips over ICI; sized for serving and small-model training. ICI is 400 GB/s bidirectional per chip. |
| TPU v6e (Trillium) | 32 GB | 1.64 TB/s | 918 TFLOPS | n/a | 560 FLOP/B | INT8 is 1,836 TOPS. 256-chip pods. ICI is 800 GB/s bidirectional per chip. |
| TPU v7 (Ironwood) | 206.158430208 GB | 7.38 TB/s | 2,307 TFLOPS | 4,614 TFLOPS | 313 FLOP/B | The specification table uses 192 GiB; the narrative says 192 GB. Use runtime allocation for deployment. Two framework devices per chip; JAX and PyTorch supported, TensorFlow unsupported. |
| Trainium2 | 103.079215104 GB | 2.9 TB/s | 667 TFLOPS | 1,299 TFLOPS | 230 FLOP/B | Eight NeuronCore-v3 per chip; Trn2 UltraServers link 64 chips over NeuronLink. |
| Trainium3 | 154.618822656 GB | 4.9 TB/s | 671 TFLOPS | 2,517 TFLOPS | 137 FLOP/B | AWS specifies 144 GiB per chip and 2.56 TB/s NeuronLink-v4. Adds MXFP8/MXFP4; verify the chosen UltraServer layout and Neuron runtime support. |
As of September 2026. Refreshed on each new generation; older parts stay because fleets keep them.