replicas

Six steps forward from traffic, never backward from an available GPU count. The one input that has to be measured rather than derived, the headroom that is not optional, and the utilization term that moves cost per token more than any tuning flag.

Identical replicas serving identical traffic should produce identical numbers, so a difference is a defect and the fleet median is the tool that finds it. Five causes, the three that are about the replica and the two that are about what it was sent.