AI Infra Interviews logo
← Distributed Training & Parallelism / 28
hardDeepSeekOpenAINVIDIA

We are running GRPO at scale. What does the infrastructure have to do that plain RLHF did not, and where do the GPUs sit idle?

GRPO replaces a learned value baseline with comparisons within response groups. Size generation and verification from the actual group workload, then distinguish continuous batching from changes to the training algorithm.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

GRPO replaces a learned value baseline with comparisons within response groups. Size generation and verification from the actual group workload, then distinguish continuous batching from changes to the training algorithm.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced
🕸️ Distributed Training🔒 Premium
RL Post-Training InfrastructureRL post-training connects response generation, scoring and policy updates. The infrastructure must coordinate model versions while handling variable response lengths and uneven phase costs.
Foundational
🕸️ Distributed Training
Data Parallelism and DDPReplicas combine gradients from different data. Follow a two-worker update, fix unequal-token weighting, and see why queued gradient buckets can leave more than one reduction exposed.
Core
🕸️ Distributed TrainingSign in
ZeRO and FSDPZeRO and FSDP reduce replicated training state by distributing it across workers. The memory saving comes with communication and temporary-buffer costs.
Advanced
🕸️ Distributed Training🔒 Premium
Tensor ParallelismTensor parallelism splits operations within a layer across GPUs. Derive the column/row matrix split, count training and inference collectives, and choose a group size from memory, latency and actual interconnect performance.
Explore the full curriculum →
UP NEXT ON YOUR JOURNEY
Next in this trackWhy is fp8 training hard, and how did DeepSeek-V3 make it work?Next in this trackDesign a checkpoint format for thousands of GPUs: no gather on write, resumable at a different world size, no stall.Next in this trackYour 4,096-GPU run loses about 2% of every day to restarts. Fix it, and tell me where the floor is.
FEDITOR'S NOTE

Count responses and tokens under explicit group assumptions. Separate batching from policy staleness, define zero-variance and timeout behavior, and compare accepted learning progress across the complete pipeline.

DISCUSSION · 0

No comments yet — be the first to share your approach.