AI Infra Interviews logo
← Distributed Training & Parallelism / 21
hardMetaOpenAIAnthropic

A 16,384-GPU training job’s steps grow from 6.0 to 6.9 seconds. How would you isolate and fix the slowdown?

Separate a longer step from lower throughput, trace the dependency that changed, and distinguish input, workload, device and path causes. Then calculate whether a repair recovers more time than it costs.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Separate a longer step from lower throughput, trace the dependency that changed, and distinguish input, workload, device and path causes. Then calculate whether a repair recovers more time than it costs.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

Compare equal work and communication-aware timings, then test whether the delay follows work, a resource or a path. Tie replacement to evidence, recovery cost and remaining work.

DISCUSSION · 0

No comments yet — be the first to share your approach.