bin packing

Fractions are either hardware slices with fixed shapes or soft shares with no isolation, and the scheduler has to know which. The representation, the packing score, the fragmentation arithmetic that spread placement produces, and the defragmentation step that fixes it.

Allocated is not usable: a cluster full of half-empty nodes has hundreds of idle GPUs that no gang can take. The arithmetic of how mixed job sizes fragment a fleet, the packing policies that prevent it, the defragmentation moves that repair it, and what each costs the jobs already running.

A serverless GPU is a promise to run a customer's model within seconds of a request while charging only for the seconds it runs. The cold-start chain from image pull to first token, the snapshot that collapses it, the warm pool sized by Little's law, and the isolation and packing decisions that set the margin.

Three one-line policy differences produce different fragmentation, and the metric that separates them is not utilization. The measured outcome on the same job sequence, why the largest free block is what matters on a training cluster, and the case where the intuitive policy is exactly wrong.