inference

It depends entirely on which dimension you split the model along, and the three choices differ by two orders of magnitude in what they demand. The per-token traffic for each, the arithmetic that puts tensor parallelism on the fast link and pipeline parallelism anywhere, and where disaggregated serving lands.

A shared pool recovers the GPUs inference holds for its peaks, but a preempted training gang takes minutes to give them back and an SLO breaks in seconds. The utilization of each design, the reclaim-time arithmetic against the traffic ramp, and the floor-plus-borrow split most fleets land on.

A plain Deployment reports ready before it can serve and scales on a signal that never moves. What an inference operator adds: weight delivery, readiness tied to warm-up, scaling on queue depth with cold starts accounted for, a prefix-aware router, and rollouts that keep streams alive. KServe and llm-d as of 2026.