The student generates faster, but distillation throughput is flat. How do you find the bottleneck?

Budget teacher input positions, full-vocabulary feedback and accepted learner work. Diagnose a growing scoring queue and decide when more student GPUs would only increase waste.

Updated Nov 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Budget teacher input positions, full-vocabulary feedback and accepted learner work. Diagnose a growing scoring queue and decide when more student GPUs would only increase waste.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 318 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

GPUs and time-to-train calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🕸️ Distributed Training🔒 Premium

On-Policy DistillationOn-policy distillation asks a teacher to guide the student on prefixes the student generated. Understand distribution matching, token alignment, teacher-scoring cost and the evaluation needed to trust a smaller policy.

Advanced

🧮 Open Weights & Serving Engines🔒 Premium

Pretraining, Fine-Tuning and DistillationUnderstand how a base model becomes a useful application model. Separate training objectives from LoRA and quantization, compare distillation with retrieval, and choose the smallest intervention supported by your evaluation.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Capacity Planning and UtilizationCapacity planning connects demand, useful throughput and acceptable waiting time. Serving and training need different models of bursts, job shape and spare capacity.

Core

🕸️ Distributed TrainingSign in

MFU and HFUMeasure useful model work, separate recomputation, and align token counts, timing and hardware peaks. Learn what activity counters can and cannot tell you.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Closest to this topicDesign a distillation pipeline that improves a target task without hiding capability regressions Closest to this topicSize an inference fleet for a 70B model serving 1,000 concurrent users Closest to this topicYour team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?

FEDITOR'S NOTE

An authored capacity scenario, not a reported company interview. Separate supplied rates from measured hardware results and preserve the training objective when reducing feedback bytes.

DISCUSSION · 0

No comments yet — be the first to share your approach.