Expert AI infra interview questions

A proposal for more memory bandwidth needs a workload that benefits, a budget for what it costs, and a reason to abandon it. Work through a serving-specific design without assuming all LLM work has the same bottleneck.

Compare the same benchmark on a healthy node, then follow the evidence through NCCL transport choices, device topology and fabric health. Half the measured bandwidth is a symptom, not a unique diagnosis.

Plan a GPU-generation migration around usable workload capacity, site readiness, compatibility and a tested rollback path.

Diagnose a rack outage through independent evidence, preserve safety interlocks, and restore capacity only after its failed paths pass acceptance checks.

Longer sequences exercise more state updates, different boundaries and sometimes different kernel paths. Use a two-tile example to check online softmax, then test masking, indexing and precision against a stable reference.

Turn the daily loss into recovery and replay costs, then identify which part is measured and which is assumed. Compare checkpoint cadence, recovery work and failure prevention without promising one universal floor.

Build a training schedule from explicit model, token, state and recovery assumptions. Keep active compute time separate from availability and contingency.

Translate request demand into prompt work, generated tokens and retained state, then size a measured serving configuration and its failure reserve.

Define cold and reused-context objectives, verify the checkpoint's real state layout, and size per-rank memory and transfer demand. Caching and phase separation help only when their compatibility, retention and capacity assumptions hold.

Validate the exact model, accelerator generation and serving features. Separate packed storage from kernel execution, derive the real memory fallback, and compare qualified throughput and total migration cost.

Define what a throughput claim counts, then compare its required arithmetic and memory traffic with a feasible deployment and matched latency targets.

Investigate a firmware-correlated regression with matched metrics, mapped paths and a controlled canary. Separate a configuration hypothesis from evidence that confirms it.

Trace the job from quota reservation through admission to physical placement. Count eligible node shapes, not just free GPUs, and use controller and scheduler evidence to choose the least disruptive fix.

A successful operation can still produce a wrong value. Explain what comparisons can detect, what they miss, and how to investigate without mistaking ordinary numerical variation for corruption.

Determine what the replacement count measures, compare equivalent device exposure and investigate differences by cohort, location, age, software and policy. Job-interruption statistics cannot supply a hardware replacement baseline.

Size a hypothetical trillion-parameter MoE from its weight placement, attention-state layout and expert traffic. Use rack specifications as bounds, then measure the serving workload before promising throughput.

Check useful progress before interpreting GPU activity. Run a bounded trace exercise that exposes a rank-conditional call, equal-count ordering errors and matching shapes with the wrong tensor meaning.

Build a five-year infrastructure outlook from explicit assumptions, competing explanations and evidence that could change your view. Connect each prediction to a practical engineering decision.