What do NVLink errors look like in telemetry, when is a link degrading rather than broken, and when do you drain the node?

Read link counters with their generation-specific units and traffic exposure. Correlate changes with actual communication performance, then choose monitoring, maintenance or immediate recovery from the supported fault state.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🩺 Fleet Reliability & Observability🔒 Premium

NVLink and Fabric FaultsGPU link faults can slow collectives or stop distributed progress. Correlate application timing with link counters and controlled path tests to locate the problem.

Advanced

🕸️ Distributed Training🔒 Premium

Tensor ParallelismTensor parallelism splits operations within a layer across GPUs. Derive the column/row matrix split, count training and inference collectives, and choose a group size from memory, latency and actual interconnect performance.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

NVLink, NVSwitch and PCIeNVLink, NVSwitch and PCIe provide different GPU communication paths. Learn how their topology and bandwidth affect the placement of parallel work.

Foundational

🔌 Networking & Storage

NCCL and Collective AlgorithmsNCCL implements GPU collectives using algorithms, protocols and channels over available hardware paths. Learn what each choice changes, how to read bandwidth results, and which evidence separates a communication problem from late application work.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWrite the postmortem for a training run that lost twelve hours. What goes in it, and what makes the action items stick?Next in this trackYou own the on-call rota for a GPU fleet. What is allowed to wake someone at 3am, and what must not?Next in this trackDesign the controller that decides whether a GPU node is healthy. What are its states, and how do you tune its false-positive rate?

FEDITOR'S NOTE

A replay-event count is not a retransmitted-byte fraction. State the denominator in any throughput model, distinguish bandwidth loss from transfer-time increase, and avoid unique diagnoses from counters.

DISCUSSION · 0

No comments yet — be the first to share your approach.