Core

DCGM and GPU Telemetry

DCGM provides GPU telemetry and diagnostics. Combine activity, errors, environmental readings and application progress to understand a fleet’s health.

This concept is included with a free account. Sign in to continue reading.

free account · no card needed

RELATED CONCEPTS

GPU Failure Modes and XID Errors ECC, Row Remapping and Memory Errors Thermal, Power and Cooling Events MFU and HFU Node Health Checks and Burn-In SLOs for AI Systems

LESSONS THAT TEACH THIS

From Model to ServiceScale from evidence and keep the bill bounded17 min→Foundations of AI InfrastructureConnect request delays to scheduler and device evidence19 min→Inference EngineeringRoute models before replicas22 min→

PRACTICE THIS IN REAL QUESTIONS

GPU Fleet Reliability & ObservabilityWith DCGM available on every node, what do you actually collect, what do you alert on, and what do you deliberately ignore?→GPU Fleet Reliability & ObservabilityDesign observability for a large training cluster. What do you collect, what does each signal answer, and what pages someone?→LLM Inference & Servingnvidia-smi shows 30 percent utilization on our serving fleet. Is that a problem, and what would you look at instead?→AI Infrastructure System DesignDesign observability for an inference fleet: per-request spans, GPU metrics, and cost attribution per tenant.→GPU Fleet Reliability & ObservabilityDefine the service level objectives for an LLM serving fleet, and the alerting that tells you when one is about to be missed.→GPU Fleet Reliability & ObservabilityHow would you detect that GPUs are thermally throttling, and what is the right response when they are?→