compute budget

A 400B dense model on 10 trillion tokens is 2.4 × 10²⁵ FLOPs, 43 days of pure compute on 16,384 H100s, and about 380 interruptions along the way. The order in which to derive every number, the layout and the data rate, the checkpoint and failure budgets, and the schedule that survives its own arithmetic.

The 6ND rule, where the 6 comes from, and the 6.3e24 that every training-time and GPU-count question in the loop is built on.