Weights are the floor, not the budget

A 70-billion-parameter model needs 140 GB for two-byte weights alone. Cache, temporary allocations and reserve add to that budget, and a total that fits across GPUs can still overload one device.

13 MIN · PLUS