mediumCoreWeaveLambdaMicrosoft

What does ECC on a GPU cost you, and why do you keep it on across a fleet?

Trace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Trace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

ECC, Row Remapping and Memory ErrorsECC and row remapping detect or manage some GPU memory faults. Recovery depends on error containment, remapping state and the device’s supported procedure.

Foundational

🩺 Fleet Reliability & Observability

GPU Failure Modes and XID ErrorsXID messages report GPU errors that need interpretation alongside logs and workload behavior. Use the code, recovery guidance and recurrence pattern to choose the next check.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

Tracing Inference RequestsConnect gateway, queue, model execution and streaming delivery without confusing overlapping spans or GPU activity with user-visible latency.

Foundational

🩺 Fleet Reliability & Observability

Alert Design and On-Call LoadUseful alerts identify a condition that needs a timely response. Connect pages to user impact or imminent risk, and reduce duplicate noise without hiding failures.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackExplain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?Next in this trackCompare A100, H100 and B200 for a 70B serving fleet. Which gives the most tokens per dollar, and where do fp8 and fp4 change the ranking?Next in this trackIf you could change one thing about GPU architecture for LLMs, what would it be, and what argues against it?

FEDITOR'S NOTE

Separate error correction, fault containment and job recovery. Scope memory-overhead claims to the device, and treat error counters as evidence to investigate rather than a forecast of the next failure.

DISCUSSION · 0

No comments yet — be the first to share your approach.