mi300x

192 GB and 5.3 TB/s per card beat the H100 on both memory numbers at a lower hourly price. Whether that wins in production is a question about achieved bandwidth and ROCm kernel coverage, and there is a short list of numbers to run before you sign the order.

The source translation is a script and takes an afternoon. The assumptions underneath it are the problem: a wavefront is 64 lanes rather than 32, so a warp-level reduction written for 32 ports cleanly and reduces half the data. What breaks silently, what breaks loudly, and the numbers that change every tuning decision.

Decode is bandwidth-bound, so cost per token tracks dollars per terabyte per second, and on spec the MI300X wins by 2x. The table, the memory-capacity term that changes the replica shape, and the software-efficiency discount that decides whether the spec ranking survives a benchmark.