AI Infra Interviews logo
← CUDA, Triton & Kernel Engineering / 27
hardMetaOpenAIAnthropic

A training step runs at 20 percent model FLOPs utilization. Profile it and find where the missing time goes.

Compute MFU with a stated work model, then follow the training step’s critical path across input, host dispatch, computation and communication. Rank experiments by exposed time instead of assigning universal gains to fixes.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Compute MFU with a stated work model, then follow the training step’s critical path across input, host dispatch, computation and communication. Rank experiments by exposed time instead of assigning universal gains to fixes.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced
⚡ Kernels & Compilers🔒 Premium
Profiling with NsightNsight Systems shows where an application spends time; Nsight Compute investigates individual kernels. Use the timeline to choose what deserves detailed profiling.
Advanced
🕸️ Distributed Training🔒 Premium
Forward Pass, Backpropagation and Optimizer StepsFollow one parameter through prediction, loss, gradient and update. Understand why backward does not itself train the weights, why training needs extra memory, and how microbatches differ from optimizer steps.
Core
🕸️ Distributed TrainingSign in
MFU and HFUMeasure useful model work, separate recomputation, and align token counts, timing and hardware peaks. Learn what activity counters can and cannot tell you.
Advanced
🔌 Networking & Storage🔒 Premium
Data Loading Pipelines for TrainingTrace batches from storage to GPU readiness, budget workers and prefetch memory, and resume from the input position saved with training state. An executable example shows how a read-ahead cursor can silently skip samples.
Explore the full curriculum →
UP NEXT ON YOUR JOURNEY
Next in this trackPort a hand-written CUDA kernel to MI300X. What translates mechanically, and what silently computes the wrong answer?Next in this trackDesign a kernel benchmarking setup that will not lie to you. What does it control for, and how do fake speedups get published?Next in this trackYour fused attention kernel matches the reference at 512 tokens and drifts at 8,000. Find the bug.
FEDITOR'S NOTE

Keep useful model FLOPs distinct from busy time, avoid double-counting overlap, and use representative multi-rank traces to test the proposed bottleneck. Recompute the metric from measured step time after each change.

DISCUSSION · 0

No comments yet — be the first to share your approach.