Would you split prefill and decode onto separate pools for a 753B model? Work the decision.

Long prefills coincide with decode stalls. Compare a tuned combined pool with separate phases, derive transfer and pool capacity, and test whether the improvement survives connector failures.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Long prefills coincide with decode stalls. Compare a tuned combined pool with separate phases, derive transfer and pool capacity, and test whether the improvement survives connector failures.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🚀 Inference & Serving🔒 Premium

Disaggregated Prefill and DecodeSeparate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.

Foundational

🧮 Open Weights & Serving Engines

Multi-Node Serving TopologiesChoose what to distribute: complete replicas, tensor operations, layers, experts or request phases. Compare rank-local fit, communication dependencies, workload concurrency and failure domains. A node boundary is not a universal performance boundary, and multiple layouts can be combined.

Foundational

🚀 Inference & Serving

Prefill vs DecodeFollow one prompt through its first prediction and later decode steps, then separate GPU work from the wait a user sees. Learn why batch size and context change the likely bottleneck.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

Memory-Bound vs Compute-Bound KernelsUse a worked timing example to distinguish a bandwidth ceiling from a measured bottleneck. Learn which profiler evidence to inspect and why decode, prefill and a whole serving step need separate diagnoses.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackA multi-node vLLM deployment starts, forms a cluster, and hangs on the first request. Debug it.Next in this trackYour product forecasts 12,800 output tokens per second at peak. Size the fleet.Next in this trackIs speculative decoding worth enabling on a trillion-parameter mixture-of-experts model?

FEDITOR'S NOTE

Scored on evidence of phase interference, actual transferred state, joint capacity and a reversible deployment decision under explicit latency objectives.

DISCUSSION · 0

No comments yet — be the first to share your approach.