data loading

The token stream is tens of megabytes a second and the checkpoints are tens of gigabytes a second; the two workloads differ by a thousand and want different tiers. The chain from tokens per second to bytes, the checkpoint burst, and the restore that sets the real spec.

Reading straight from NVMe into GPU memory skips a copy through host memory, which is worth a lot for one workload and nothing for another. The bandwidth arithmetic for both paths, the two cases where the bounce is the bottleneck, and the far more common case where the CPU has to touch the data anyway.

One gives you POSIX semantics and hundreds of gigabytes per second at a price per terabyte that hurts at petabyte scale. The other is cheap and durable and answers a first byte in tens of milliseconds. The cost arithmetic that makes most clusters run both, and the metadata limit that decides how you shard.

For text training the bytes are trivial and the bottleneck is never bandwidth, so the usual advice about faster storage misses. What the loader actually has to deliver per second, the four things that consume the time instead, and the order to fix them with the measurement that proves each.