mediumNewCoreWeaveLambdaMicrosoft

What does ECC on a GPU cost you, and why do you keep it on across a fleet?

On an HBM part ECC costs almost nothing; on a GDDR part about 6% of capacity and bandwidth. What it buys is the difference between a corrected bit and a silently flipped exponent that turns 1.0 into infinity across a 512-GPU all-reduce. The mechanisms, the signals, and the expected-loss arithmetic.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

ECC, Row Remapping and Memory ErrorsHBM stacks flip bits, and the difference between a fleet that shrugs and one that loses a training step to corruption is error-correcting codes plus the machinery that retires bad memory before it produces a double-bit error. A single-bit error is corrected silently and counted; a double-bit error is detected, kills the process, and on Ampere and later triggers the row remapper to swap the failing row for a spare at the next reset. This page explains the codes, the remapper's states, how to read the counters as a prediction of failure, and the RMA rules a fleet applies.

Foundational

🩺 Fleet Reliability & Observability

GPU Failure Modes and XID ErrorsWhen a GPU misbehaves, the NVIDIA driver writes an XID line to the kernel log, and the number on that line is the first and often the only clue to what happened. Fleet engineers learn a dozen of them the way doctors learn a dozen lab values: 13 and 31 are almost always the application, 48 and 95 are memory that needs a reset, 63 and 64 are the row remapper reporting or failing, 74 is the NVLink fabric, 79 is a GPU that has vanished from the PCIe bus. This page gives the taxonomy, the decision for each (retry, reset, drain, RMA), and the derivation of how often a big fleet should expect each.

Foundational

🩺 Fleet Reliability & Observability

Alert Design and On-Call LoadAn alert exists to change what a human does, so any alert that fires without a decision attached is a false alarm regardless of whether its condition was true. GPU fleets generate a specific set of noisy signals that look serious and are not, and separating those from the ones that need a person at three in the morning is what keeps a rotation sustainable. The measure of an alerting system is the fraction of pages that led to an action.

Foundational

🧭 Ownership & Judgment

The Reliability Pushback StoryEvery AI infra loop has a behavioral round, and the story it wants most is the one where you stopped something (a launch, a run, a hardware admission) because the data said to, and you were accountable for the cost of stopping. This page gives the skeleton that works: the situation, the signal you read, the decision and who owned it, the evidence you brought, and what changed afterward. It also gives the follow-up interviewers hold back, the version that sounds right and fails, and the line between a senior telling and a staff telling of the same story.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackExplain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?Next in this trackCompare A100, H100 and B200 for a 70B serving fleet. Which gives the most tokens per dollar, and where do fp8 and fp4 change the ranking?Next in this trackIf you could change one thing about GPU architecture for LLMs, what would it be, and what argues against it?

FEDITOR'S NOTE

Scored on distinguishing detection from correction, knowing the HBM-versus-GDDR cost difference, naming the row-remapping and Xid signals an operator watches, and reasoning about silent corruption as a fleet-scale expected cost.

DISCUSSION · 0

No comments yet — be the first to share your approach.