fine tuning

The part with no NVLink can be the correct choice, and the reason is the workload shape rather than the specification. What fits on one card, what MIG partitioning buys a multi-tenant service, and the exact point where the missing scale-up link makes the decision flip.

LoRA shrinks the trainable state and leaves 141 GB of frozen weights untouched. QLoRA attacks the frozen half: NF4 takes it to 35 GB, double quantization compresses the scales themselves, paged optimizers absorb the spikes. The byte accounting, what dequantization costs, and where DoRA adds a term.

Fifty full copies is fifty deployments and is unaffordable. Fifty low-rank adapters over one shared base is one deployment, because an adapter is a tiny fraction of the weights and adapters from different tenants can share a batch. The arithmetic that makes it work and the two limits that decide how far it goes.

Sixteen bytes per parameter is 1.1 TB of static state before a single activation; LoRA keeps the 141 GB of frozen weights and shrinks the rest to a few gigabytes. The derivation, the GPU counts, and the activations that both still pay.

Spot is 60% cheaper until the preemptions eat the discount. The expected-loss formula (waste per preemption = half the checkpoint interval plus restart), the break-even worked at three cadences, the checkpoint cost that pushes back the other way, and the gang arithmetic that says which jobs must never be on spot.

The expensive thing is a base model in GPU memory, not the customer's job. Keep bases resident, pack LoRA jobs onto them, isolate tenants at the process and device level, checkpoint adapters in seconds so jobs run on cheap capacity, and attribute cost per tenant. The arithmetic behind each rule.

A replica per variant is 100 copies of a 140 GB base and 800 GPUs mostly idle; multi-LoRA holds the base once and hundreds of adapters in a few gigabytes. The memory arithmetic, the throughput cost of mixed batches, the routing that keeps hot adapters resident, and the two cases where a variant earns its own replica.