sizing
AI infra interview questions tagged sizing, across every topic.
7 questions · 4 unlocked for you
Concepts behind "sizing"
The curriculum that explains the ideas these questions test.
Foundational
Reading config.json to Size a Model You Have Never RunEvery Hugging Face model ships a config.json, and it contains enough to compute the weight footprint, the KV cache per token, the parallel degrees that divide cleanly and the minimum GPU count, before downloading a byte. Doing that derivation is a standard whiteboard exercise in serving interviews because it is exactly what an engineer does on the morning a new model lands, and the fields that matter are the same across every recent architecture.🧮 Open Weights & Serving Engines
Foundational
Capacity Planning for Open-Weights FleetsPlanning a fleet for a sparse open-weights model works differently from planning one for a dense model, because memory follows total parameters and throughput follows active parameters, and those now differ by more than twenty times. The sizing goes in one direction only: from a traffic forecast to tokens per second, to replicas at a measured operating point, to GPUs, to racks and kilowatts. Doing it in the other direction, from an available GPU count, produces a fleet that fits the hardware rather than the demand.🧮 Open Weights & Serving Engines