You have one week to get a newly released trillion-parameter model serving production traffic. Plan it.

Two steps consume most of the week and neither is the one people plan for. The gate that can end the project on day one, the multi-hour download, the correctness check almost nobody runs, and the canary that has to see real traffic before Friday.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🧮 Open Weights & Serving Engines

Model Onboarding: From Hugging Face to ProductionA new open-weights model lands and someone asks how long until it is serving traffic. The answer depends on a sequence that is the same every time: read the card and the config, check engine support for the exact attention and quantization combination, size it, pull the weights, bring up one replica, validate correctness against the authors' own outputs, benchmark, then roll out behind a flag. The steps that surprise people are the download, which is hours for a trillion-parameter model, and the correctness check, which almost nobody does and which catches the wrong template.

Foundational

🧮 Open Weights & Serving Engines

Capacity Planning for Open-Weights FleetsPlanning a fleet for a sparse open-weights model works differently from planning one for a dense model, because memory follows total parameters and throughput follows active parameters, and those now differ by more than twenty times. The sizing goes in one direction only: from a traffic forecast to tokens per second, to replicas at a measured operating point, to GPUs, to racks and kilowatts. Doing it in the other direction, from an available GPU count, produces a fleet that fits the hardware rather than the demand.

Foundational

🖧 Hardware & Cluster Build-Out

Burn-In and Acceptance TestingNew hardware fails early or it fails late, and burn-in exists to move the early failures before the cluster is handed over rather than after. A proper acceptance test runs every layer under sustained load for days, compares every node against its siblings rather than against a specification, and produces a signed number the buyer and the vendor both agree on. The comparison is the important part: identical hardware running identical work should produce identical numbers, and the outliers are the finding.

Advanced

🧭 Ownership & Judgment🔒 Premium

Migrations and DeprecationsEvery infrastructure career contains a migration nobody wanted: the scheduler swap, the driver upgrade across a live fleet, the storage move while training runs are in flight, the deprecation of the launcher every team's scripts depend on. The behavioral round asks about one because it tests the skills that matter most and show least on a résumé: sequencing under risk, keeping a rollback real, moving people who have no reason to move, and knowing when to stop. This page gives the shape of a migration story that lands, two worked narratives from GPU fleet work, and the answers that sound like leadership and fail.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackSelf-host a 753B open-weights model or call a hosted API? Work the crossover.Next in this trackOne replica in a serving fleet returns tokens slower than its siblings. Find out why.Next in this trackThe model advertises a million tokens of context. What actually breaks when you serve it?

FEDITOR'S NOTE

Scored on the engine-support gate coming first, on budgeting the download and the correctness validation, and on a canary with quality monitoring rather than error rate.

DISCUSSION · 0

No comments yet — be the first to share your approach.