A GPU computes the wrong answer and reports no error. How would you detect that, and what does it look like in a training run?

A successful operation can still produce a wrong value. Explain what comparisons can detect, what they miss, and how to investigate without mistaking ordinary numerical variation for corruption.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

A successful operation can still produce a wrong value. Explain what comparisons can detect, what they miss, and how to investigate without mistaking ordinary numerical variation for corruption.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Foundational

🖧 Hardware & Cluster Build-Out

Rack Power Delivery and BuswaysRack power determines current, distribution equipment and redundancy requirements. Training workloads can also create load swings that average power hides.

Foundational

🩺 Fleet Reliability & Observability

GPU Failure Modes and XID ErrorsXID messages report GPU errors that need interpretation alongside logs and workload behavior. Use the code, recovery guidance and recurrence pattern to choose the next check.

Core

🩺 Fleet Reliability & ObservabilitySign in

DCGM and GPU TelemetryDCGM provides GPU telemetry and diagnostics. Combine activity, errors, environmental readings and application progress to understand a fleet’s health.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

ECC, Row Remapping and Memory ErrorsECC and row remapping detect or manage some GPU memory faults. Recovery depends on error containment, remapping state and the device’s supported procedure.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign the alerting rules for a GPU platform so that a page is always worth waking for. What are the mechanics?Next in this trackCoolant flow to a rack stops. What happens in the next sixty seconds, and what has to be automatic because a human cannot act in time?Next in this trackTrace one request through an inference platform. What are the spans, what does each cost, and where does latency actually hide?

FEDITOR'S NOTE

Look for incomplete hardware coverage, valid comparison invariants, propagation through training, and evidence that separates a suspicion from a confirmed device fault.

DISCUSSION · 0

No comments yet — be the first to share your approach.