nvl72

Three things change at once: dense compute more than doubles with an fp4 tensor core, HBM3e reaches 8 TB/s, and NVLink stops at 72 GPUs instead of 8. Work through what each does to tensor-parallel degree, per-step weight reads and expert placement for a 671B-parameter mixture of experts.

Four separate facility limits have to be cleared and total building capacity is not one of them. Power per rack, heat rejection, floor loading and network reach each fail independently, and each has a different remedy with a different lead time. The options that exist, priced, and the one that is usually taken.

Same GPU count, same silicon, and a completely different machine. The domain size is the whole argument, it is worth an order of magnitude on specific traffic, and it costs a failure domain nine times larger plus a facility that most halls do not have.

Two all-to-alls per layer across sixty layers is over a hundred collectives on the critical path of every decode step, each one small and latency-bound. The per-GPU volume worked out, the time on each link type, and why a single large NVLink domain changes the design rather than just improving it.

A trillion parameters with 32 billion active per token: the weights need a rack, the compute needs a fraction of it, and the all-to-all between them decides everything. The memory plan across 72 GPUs, the decode step from bandwidth and two all-to-alls, the compressed attention cache, and what imbalance costs.