Two thousand GPUs are being replaced. What do you do with the old ones?

Older parts are rarely worthless and often better suited to work the new fleet should not be doing. The four options ranked by what they return, the data handling that has to happen before any hardware leaves, and the reason the answer is usually not one option.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🗂️ Scheduling & Orchestration

Node Lifecycle: Drain, Upgrade and ReturnA node moves through a fixed cycle between provisioning and decommissioning, and most fleet operations are one lap around it: cordon so nothing new lands, drain so running work finishes or moves, act, validate, then return to the pool. The wall-clock cost of a fleet-wide change is dominated by draining rather than by the change itself, which makes the plan a scheduling document rather than a technical one.

Advanced

📐 AI Systems Design🔒 Premium

Multi-Tenant Fine-Tuning ServiceA fine-tuning service takes a customer's dataset and a base model and returns a model, and the design problem is that many customers want this at once, cheaply, without seeing each other's data, on GPUs that must not sit idle between jobs. LoRA changes the shape: an adapter is a few hundred megabytes rather than a copy of the base, so many jobs can share a base in memory and many adapters can be served from one replica. This page designs the service end to end: the pipeline, the LoRA arithmetic that sets memory and cost, the isolation, the scheduler that packs jobs, and the serving path.

Foundational

🖧 Hardware & Cluster Build-Out

Colocation, Power Contracts and Site SelectionFor most organizations the constraint on deploying GPUs is not the GPUs. It is finding a hall that can deliver 100 kilowatts or more per rack, reject that heat with liquid, and sign a contract for the power years before the hardware exists. Colocation contracts price reserved capacity rather than consumption, cooling capability is what eliminates most sites, and the lead time on new electrical supply is measured in years while GPUs arrive in months.

Foundational

🧮 Napkin Math & Capacity

KV Cache SizingThe KV cache is the memory that decides how many users a serving replica can hold and how long their context can be. Its size per token comes from four numbers in the model's config file (layers, KV heads, head dimension, bytes per element) and one formula; multiplied by context and concurrency it is the number every capacity plan is built on. This page derives it, works it for four models including an MLA one, and shows the two places candidates get it wrong by a factor of eight.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackAn entire rack stops responding at 2 a.m. Walk me through the first thirty minutes.Closest to this topicSize the storage for a 2,048-GPU training cluster. What numbers actually drive it?Closest to this topicHow much power does a 512-GPU cluster actually draw, end to end?

FEDITOR'S NOTE

Scored on redeployment being evaluated before resale, on the data-handling steps that gate any disposal, and on the reclaimed power and space being part of the return.

DISCUSSION · 0

No comments yet — be the first to share your approach.