upgrades

A fleet-wide version change is the most likely cause of the next unexplained performance regression, so the rollout is designed to make that attributable. Cohorts, a canary that measures rather than boots, and the rollback that has to be real before the first node is touched.

A driver upgrade touches every layer at once: kernel module, CUDA runtime, container toolkit, NCCL, fabric driver, and every job's image. The compatibility matrix that decides whether a job can run on the new node, the canary that proves it, the wave arithmetic for 2,000 nodes, and the rehearsed rollback.