quotas

One tenant can consume a shared deployment's capacity without exceeding any limit you set, because the limits are usually on requests and the cost is in tokens. What actually needs limiting, the three isolation levels by strength and price, and the fix that works in minutes.

Every tenant bursts independently, so the shared tier sees the sum of uncorrelated peaks rather than the sum of averages. The provisioning arithmetic that follows, the three isolation boundaries, and why per-tenant local storage solves most of it before any policy is written.

Static quotas waste half the fleet and a free-for-all starves the small teams. The four-layer policy (guaranteed quota, borrowing, fair-share ordering, preemption) with the pooling arithmetic that justifies it, the fair-share ratio worked by hand, and the five ways teams game it.

Allocated means a scheduler handed the GPU out; utilized means it did work. A 60-point gap is jobs holding GPUs they do not use: idle notebooks, placeholder jobs, a loader-bound run at 20% tensor-active. The three metrics that separate the causes, the per-tenant table that names them, and the policies that close it.

A notebook holds a GPU for eight hours and uses it for twenty minutes, which is what sharing is for and what makes it hard. The sharing mode to pick and the one to refuse across users, the idle-reclaim loop that keeps sessions alive without holding hardware, the quota arithmetic, and how many GPUs 300 users need.

A GPU credit system is an accounting ledger with a scheduler attached: who may spend, how fast, what happens when the account is empty and what happens when everyone spends at once. The data model, the debit path, the fairness rule and the arithmetic that keeps 4,096 GPUs busy without starving a team.

A request can cost 50 tokens or 50,000, so a request limit protects nothing. The two buckets per tenant per model, the reservation-then-settle scheme for output tokens you cannot count in advance, the distributed counter fast enough for the gateway, and how the limits map onto the fleet's real capacity.

The playground is the API's most public client and its most abused one: free, barely identified, streaming. The transport and its cost per connection, session state that survives a refresh with no GPU holding anything, quotas per user and per fleet, and the abuse controls that stop the free tier becoming a free API.

A research cluster is shared by teams whose managers bought it and by researchers who want it now. A quota tree with guarantees and borrowing, a fair-share formula that decays so last week's usage does not cost you today, preemption that is cheap because checkpoints are, and the incentives that make hoarding lose.

Saying no is easy and it costs you the relationship. The move that works is to make the queue visible, give the researcher something they control, and offer a smaller thing today. The three mechanisms that turn this from a recurring argument into a system.

A capacity fight is almost never about capacity. It is two teams with different information and no shared number. The move that ends it is publishing utilization both teams can see, and the escalation that works is a written proposal with options rather than a request for a ruling.