Advanced

RL Post-Training Infrastructure

RL post-training connects response generation, scoring and policy updates. The infrastructure must coordinate model versions while handling variable response lengths and uneven phase costs.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Prefix Caching and KV Reuse ZeRO and FSDP Checkpointing and Resumption at Scale Bandwidth-Bound Decode Throughput Serving Engines: vLLM, SGLang and TensorRT-LLM Inference Platform Architecture

LESSONS THAT TEACH THIS

From Model to ServicePublish a model without mixing its versions22 min→

PRACTICE THIS IN REAL QUESTIONS

Distributed Training & ParallelismDesign the infrastructure for RLHF on a 70B: where do rollouts and the learner run, and how do weights move between them every step?→Distributed Training & ParallelismWe are running GRPO at scale. What does the infrastructure have to do that plain RLHF did not, and where do the GPUs sit idle?→Distributed Training & ParallelismYour team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?→Distributed Training & ParallelismIn data-parallel training, what actually gets communicated between GPUs, and how much is it per step?→Distributed Training & ParallelismCompare data, tensor and pipeline parallelism. What does each one shard, what does each one communicate, and where does each one live?→Distributed Training & ParallelismWhat is MFU, how do you compute it from a running job, and what counts as a good number?→