27Write the policy for when a GPU is replaced rather than returned to service. What are the triggers and what do they cost?▼hardNewCoreWeaveLambdaMeta4 replies◆ premiumEvery replacement costs a spare, a maintenance window and a return process; every device kept costs the risk of a job it will fail. Four triggers that decide it without a case-by-case argument, the arithmetic that sets the repeat threshold, and why the device's history beats any diagnostic result.Open full answer →
20Implement a circuit breaker in front of a model endpoint. What are the states, and what does the half-open state buy?▼mediumNewBasetenOpenAI4 replies○ sign inThree states and one insight: after a sustained failure, retrying is worse than not calling at all, and the only way to learn whether the backend recovered is to send a small number of probes. The state machine, the two thresholds, and why a rate over a window beats a consecutive-failure count.Open full answer →
21Given per-rank heartbeats from a training job, detect which rank has stopped and when. What produces false positives?▼mediumNewMetaAnthropic4 replies◆ premiumA few lines of comparison, and every difficulty is in the threshold and the clock. Why the collector's receive time rather than the sender's, the boundary case that decides whether an exactly-late rank is reported, and the arithmetic that turns a false-positive rate into a threshold.Open full answer →
