Advanced

RL Post-Training Infrastructure

Reinforcement-learning post-training alternates between generating long responses with an inference engine and updating the policy with a training framework, and the two phases want different hardware, different parallel layouts and a copy of the same weights each. The infrastructure problem is keeping the generators busy while the learner trains, syncing terabytes of weights between them every step, and surviving the long tail of response lengths.

Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew

RELATED CONCEPTS

Prefix Caching and KV Reuse ZeRO and FSDP Checkpointing and Resumption at Scale Bandwidth-Bound Decode Throughput Serving Engines: vLLM, SGLang and TensorRT-LLM Inference Platform Architecture

PRACTICE THIS IN REAL QUESTIONS

Distributed Training & ParallelismDesign the infrastructure for RLHF on a 70B: where do rollouts and the learner run, and how do weights move between them every step?→Distributed Training & ParallelismWe are running GRPO at scale. What does the infrastructure have to do that plain RLHF did not, and where do the GPUs sit idle?→Distributed Training & ParallelismYour team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?→Distributed Training & ParallelismIn data-parallel training, what actually gets communicated between GPUs, and how much is it per step?→Distributed Training & ParallelismCompare data, tensor and pipeline parallelism. What does each one shard, what does each one communicate, and where does each one live?→Distributed Training & ParallelismWhat is MFU, how do you compute it from a running job, and what counts as a good number?→