Agent SFT loss fell after a data-pipeline change, but tool success regressed. What do you inspect?

Trace a misleading loss improvement through role masks, next-token alignment, truncation and normalization. Build a controlled comparison before changing the optimizer.

Updated Nov 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Trace a misleading loss improvement through role masks, next-token alignment, truncation and normalization. Build a controlled comparison before changing the optimizer.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 318 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

GPUs and time-to-train calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

📐 AI Systems Design🔒 Premium

Evaluation and Data Pipeline InfrastructureData pipelines prepare traceable training inputs; evaluation pipelines compare model behavior. Both need versioned inputs, reproducible execution and useful failure reporting.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.

Foundational

🕸️ Distributed Training

Data Parallelism and DDPReplicas combine gradients from different data. Follow a two-worker update, fix unequal-token weighting, and see why queued gradient buckets can leave more than one reduction exposed.

Core

🕸️ Distributed TrainingSign in

ZeRO and FSDPZeRO and FSDP reduce replicated training state by distributing it across workers. The memory saving comes with communication and temporary-buffer costs.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackThe student generates faster, but distillation throughput is flat. How do you find the bottleneck?Closest to this topicDesign a distillation pipeline that improves a target task without hiding capability regressions Closest to this topicTraining loss went flat at step 40k after descending normally. Walk me through how you debug it.

FEDITOR'S NOTE

An authored diagnostic scenario, not a reported company interview. Explain the actual prediction-target pairs and preserve the held-out task denominator.

DISCUSSION · 0

No comments yet — be the first to share your approach.