A replica takes 70 seconds to become useful and the burst arrives in 60, so the autoscaler cannot win the first minute; the warm pool has to. The leading signal, the cooldowns derived from the cold start, the warm-pool size from Little's law and the burst profile, and the cost of every second of cold start left in.
Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.
A replica takes 70 seconds to become useful and the burst arrives in 60, so the autoscaler cannot win the first minute; the warm pool has to. The leading signal, the cooldowns derived from the cold start, the warm-pool size from Little's law and the burst profile, and the cost of every second of cold start left in.
Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.
Scored on deriving the warm pool from the burst profile and the cold start, on a leading signal with cooldowns longer than the cold start, on shrinking the cold start as the change with the largest effect, and on the cost trade stated as a number.
No comments yet — be the first to share your approach.
