Together AI GPU Fleet Reliability & Observability interview questions

Compare equivalent requests, inspect the replica’s software and device state, and reduce new traffic only after checking where that load will go.

Use the Llama 3 interruption record without turning job interruptions into device lifetime, predictable failures or a universal spare percentage.

Handle an inaccessible GPU with bounded recovery and preserved evidence, without treating a reboot or repeat count as proof of root cause.

Use DCGM fields with their actual semantics, combine device health with workload progress, and calibrate alerts on correlated fleet data.

Interpret XID codes with device and software context, separating immediate recovery from evidence-based root-cause investigation.

Find where useful progress stopped by comparing per-rank traces, host stacks and device events. Distinguish late arrival, mismatched collectives and stalled execution before choosing a recovery action.

Build observability around durable useful progress and aligned phase timing, showing why full-step timers can hide the late rank.

Investigate rising corrected-memory errors without promising a failure prediction, and follow recovery flags and vendor repair criteria.

Find the critical-path delay using local phase timing and controlled comparisons, rather than diagnosing a node from utilization or clock ratios.

Derive a checkpoint interval from measured job interruptions and exposed save cost. Trace durability, lost work and recovery before treating an estimate as goodput.

Promise measurable job outcomes: time to useful work, recoverable progress and successful restoration. Define the population and exclusions before choosing targets, and give each error budget an agreed operating policy.

Define request success, first-output latency and streaming continuity for each service class. Count bad outcomes against an explicit budget, then use paired long and short windows to detect significant ongoing degradation.

Build an acceptance suite around the node’s intended workload and supported hardware limits. Combine configuration, correctness, communication and sustained-load evidence, then record a baseline and an explicit return-to-service decision.

Correlate clock-event reasons, temperature, configured limits and workload timing. Use the scope of the slowdown to choose the next investigation, and distinguish intentional power limits from unexpected thermal constraints.

Read link counters with their generation-specific units and traffic exposure. Correlate changes with actual communication performance, then choose monitoring, maintenance or immediate recovery from the supported fault state.

Use one consistent timeline to separate discarded progress, recovery delay and detection. Explain the contributing conditions, then assign corrective work with owners, dates and tests that demonstrate the intended behavior.