xid

On an HBM part ECC costs almost nothing; on a GDDR part about 6% of capacity and bandwidth. What it buys is the difference between a corrected bit and a silently flipped exponent that turns 1.0 into infinity across a 512-GPU all-reduce. The mechanisms, the signals, and the expected-loss arithmetic.

Half the expected bus bandwidth has four plausible causes needing different fixes: a link trained at reduced width, a fabric that never came up, a topology the job did not expect, or a test that measured latency. The sequence of commands that tells them apart, and what each output looks like on a healthy node.

The driver logs a numbered code when something goes wrong on a GPU, and the number tells you whether to retry the job, drain the node or file a hardware return. The codes worth memorizing in three groups, the action each implies, and the automation that turns a log line into a drained node.