fairness

One tenant can consume a shared deployment's capacity without exceeding any limit you set, because the limits are usually on requests and the cost is in tokens. What actually needs limiting, the three isolation levels by strength and price, and the fix that works in minutes.

A request can cost 50 tokens or 50,000, so a request limit protects nothing. The two buckets per tenant per model, the reservation-then-settle scheme for output tokens you cannot count in advance, the distributed counter fast enough for the gateway, and how the limits map onto the fleet's real capacity.

A heap orders the queue and the interesting logic is elsewhere: which running jobs may be evicted, how many, and what happens to them. The victim-selection rule that avoids evicting more than necessary, the equal-priority case that must not preempt, and the starvation the design creates if nothing ages.

Fair sharing of one resource is obvious and fair sharing of three is not, because a GPU-heavy job and a memory-heavy job are not comparable. The rule that makes them comparable, the greedy loop that implements it, and the tie-breaking bug that gave one user everything.