A replica restarts and loads different weights than its siblings. How did that happen?

A model reference that names a branch points at whatever is current, so a restart months after a deploy loads whatever the authors published since. What that produces, why the fleet ends up half-updated, and the three changes that make a deployment reproducible.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

📐 AI Systems Design🔒 Premium

Evaluation and Data Pipeline InfrastructureBehind every model release is a pipeline that turns raw text into training shards and a harness that runs thousands of evaluation prompts against every checkpoint, and both are infrastructure problems with GPU-sized budgets. The data side is a batch system: dedup, filter, tokenize and shard petabytes with lineage. The eval side is a serving system in disguise: run a benchmark suite against a checkpoint in minutes, on shared GPUs, reproducibly, with results a researcher can trust. This page designs both, derives the compute and storage they need, and gives the reproducibility rules that separate a real harness from a script.

Foundational

🧮 Open Weights & Serving Engines

Serving Benchmarks That Do Not LieMost published serving numbers are not comparable to each other and not predictive of production, because they differ in the input distribution, the concurrency, whether the cache was warm, and which of several very different metrics is being reported. A benchmark that supports a decision has to fix all four, report a distribution rather than a mean, and be run against the traffic shape you actually serve. The single most useful discipline is to compute the bandwidth bound first, so you know what fraction of the possible you achieved.

Foundational

🧮 Open Weights & Serving Engines

Capacity Planning for Open-Weights FleetsPlanning a fleet for a sparse open-weights model works differently from planning one for a dense model, because memory follows total parameters and throughput follows active parameters, and those now differ by more than twenty times. The sizing goes in one direction only: from a traffic forecast to tokens per second, to replicas at a measured operating point, to GPUs, to racks and kilowatts. Doing it in the other direction, from an available GPU count, produces a fleet that fits the hardware rather than the demand.

Foundational

🧮 Open Weights & Serving Engines

Open-Weights Models of 2026The open-weights frontier moved from dense models of tens of billions of parameters to sparse mixtures of experts measured in trillions, and the serving problem changed with it. As of September 2026 the releases an infrastructure engineer is asked about are Z.ai's GLM-5.3 at 753B, Moonshot's Kimi K3 at 2.8T, and DeepSeek's V4 family. What matters for deployment is not the headline count but three other numbers: active parameters per token, the attention design, and the format the weights actually shipped in.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYou must serve a frontier open-weights model on non-NVIDIA accelerators. Plan it.Closest to this topicYour agent product's tool-call success rate fell from 94 to 61 percent after a deploy. Debug it.Closest to this topicThe model authors publish a new revision. How do you roll it out without a quality regression?

FEDITOR'S NOTE

Scored on mutable references as the root cause, on the half-updated fleet being the symptom, and on pinning plus mirroring plus a response fingerprint as the fix.

DISCUSSION · 0

No comments yet — be the first to share your approach.