Design a distillation pipeline that improves a target task without hiding capability regressions

Version teacher outputs, separate learning from evaluation, and catch a release whose domain score improves while a protected capability regresses.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Version teacher outputs, separate learning from evaluation, and catch a release whose domain score improves while a protected capability regresses.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 318 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

GPUs and time-to-train calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🧮 Open Weights & Serving Engines🔒 Premium

Pretraining, Fine-Tuning and DistillationUnderstand how a base model becomes a useful application model. Separate training objectives from LoRA and quantization, compare distillation with retrieval, and choose the smallest intervention supported by your evaluation.

Advanced

🕸️ Distributed Training🔒 Premium

On-Policy DistillationOn-policy distillation asks a teacher to guide the student on prefixes the student generated. Understand distribution matching, token alignment, teacher-scoring cost and the evaluation needed to trust a smaller policy.

Advanced

📐 AI Systems Design🔒 Premium

Evaluation and Data Pipeline InfrastructureData pipelines prepare traceable training inputs; evaluation pipelines compare model behavior. Both need versioned inputs, reproducible execution and useful failure reporting.

Advanced

🚀 Inference & Serving🔒 Premium

Reasoning Budgets and Answer LatencyReasoning tokens consume generation time and retained state before an answer becomes usable. Separate the first generated token from the first answer token, reserve output capacity, and evaluate reasoning limits against task success.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackA PyTorch team wants to train on TPUs. Do they rewrite in JAX with MaxText, use PyTorch/XLA, or wait for TorchTPU?Next in this trackAgent SFT loss fell after a data-pipeline change, but tool success regressed. What do you inspect?Next in this trackThe student generates faster, but distillation throughput is flat. How do you find the bottleneck?

DISCUSSION · 0

No comments yet — be the first to share your approach.