Design the infrastructure for RLHF on a 70B: where do rollouts and the learner run, and how do weights move between them every step?

Separate generation, scoring, learning and policy publication. Size the pools from matched workload measurements, and make policy versions explicit before overlapping rollout generation with updates.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Separate generation, scoring, learning and policy publication. Size the pools from matched workload measurements, and make policy versions explicit before overlapping rollout generation with updates.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

GPUs and time-to-train calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🕸️ Distributed Training🔒 Premium

RL Post-Training InfrastructureRL post-training connects response generation, scoring and policy updates. The infrastructure must coordinate model versions while handling variable response lengths and uneven phase costs.

Core

🚀 Inference & ServingSign in

Continuous BatchingContinuous batching changes which requests run between model iterations. Learn how a waiting request joins, why an empty slot is not enough, and how to balance throughput against each user’s token latency.

Advanced

🚀 Inference & Serving🔒 Premium

Chunked PrefillFollow a long prompt through a shared token budget, including its final partial chunk. Learn why chunking can reduce stream pauses without guaranteeing a fixed token gap or a universal throughput gain.

Foundational

🧮 Open Weights & Serving Engines

vLLM Server Arguments That MatterSeparate executor memory, allocated cache pools and logical occupancy. Map context, sequence and token-batch limits to the constraints they control, then tune a pinned deployment against per-rank memory and latency.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYour team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?Next in this trackWe are running GRPO at scale. What does the infrastructure have to do that plain RLHF did not, and where do the GPUs sit idle?Next in this trackWhy is fp8 training hard, and how did DeepSeek-V3 make it work?

FEDITOR'S NOTE

Define the RL algorithm and its state before placing components. Account for token workloads, policy versions and complete weight publication; compare synchronous and asynchronous plans without assuming quality-neutral staleness.

DISCUSSION · 0

No comments yet — be the first to share your approach.