health checks
AI infra interview questions tagged health checks, across every topic.
2 questions · 0 unlocked for you
Concepts behind "health checks"
The curriculum that explains the ideas these questions test.
Foundational
Node Lifecycle: Drain, Upgrade and ReturnA node moves through a fixed cycle between provisioning and decommissioning, and most fleet operations are one lap around it: cordon so nothing new lands, drain so running work finishes or moves, act, validate, then return to the pool. The wall-clock cost of a fleet-wide change is dominated by draining rather than by the change itself, which makes the plan a scheduling document rather than a technical one.🗂️ Scheduling & Orchestration
Advanced
Node Health Checks and Burn-InThe cheapest failure is the one caught before a job starts. A GPU fleet runs three kinds of check: a burn-in of days on new or repaired hardware to shake out the parts that die young, a pre-flight suite of minutes before a node is handed to a job, and continuous checks during the job. Each test has an expected number (a GEMM within a few percent of the fleet median, an all-reduce at rated bus bandwidth, a NIC at line rate, a memory test with zero errors), and a node that misses any of them goes to quarantine. This page gives the suites, the numbers, and the arithmetic that justifies the time they cost.🩺 Fleet Reliability & Observability🔒 Premium