model loading

141 gigabytes have to move from somewhere to eight GPUs, and every hop has a bandwidth. Add the CUDA init, the engine warm-up and the graph capture, and the minute is gone unless you design each step.

A plain Deployment reports ready before it can serve and scales on a signal that never moves. What an inference operator adds: weight delivery, readiness tied to warm-up, scaling on queue depth with cold starts accounted for, a prefix-aware router, and rollouts that keep streams alive. KServe and llm-d as of 2026.