Advanced

Disaggregated Prefill and Decode

Separate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Prefill vs Decode Chunked Prefill The KV Cache Inference Platform Architecture RDMA, InfiniBand and RoCEv2 Request Routing and Load Balancing for LLMs

LESSONS THAT TEACH THIS

Inference EngineeringChunked prefill controls interference on a shared execution path21 min→Inference EngineeringSplitting prefill from decode answers a problem you must have first30 min→

PRACTICE THIS IN REAL QUESTIONS

LLM Inference & ServingWhen does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?→Open-Weights Models & Serving EnginesWould you split prefill and decode onto separate pools for a 753B model? Work the decision.→AI Infrastructure System DesignDesign prefill and decode disaggregation for a 70B serving fleet: pool sizes, KV transfer, routing, and when to turn it on.→Networking, Interconnects & StorageDoes multi-node inference need the same fabric as training, or can it run on ordinary networking?→LLM Inference & ServingWhy do prefill and decode behave so differently, and why does that matter for the hardware you serve on?→Napkin Math, Cost & CapacityA vendor claims 10,000 tokens per second per GPU. Sanity-check it.→