Runtime recipe
8 × H200 SXM
141 GB per GPU
INT4 experts / mixed
vLLM model-specific guide
TP8 on an H200 node; multimodal encoder placement and request state still consume memory. Follow the exact checkpoint loader.
Kimi / MoE
Kimi-K2.5
The multimodal starting point for the K2 series
New to model sizes or GPU memory? Start with weights, parameters and quantization →
K2.5 is worth understanding as the early multimodal K2 baseline, even if you deploy a newer checkpoint. It makes the distinction between active compute, expert storage and vision processing concrete.
1,000Breported total parameters
32Bactive parameters per token
262,144configured token positions
Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size.
Evaluate it for
Reproducing K2-era evaluations and understanding large multimodal MoE deployment.
Choose another path when
A fresh deployment where a newer K2 variant has already demonstrated better task outcomes at the same operating cost.
Release context
Why this release matters
The released artifact combines quantized experts with other components at different precision. Its 32B active count never meant that the whole model could occupy a 32B-sized memory budget.
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
8 × MI355X
288 GB per GPU
W4A16 experts / mixed
vLLM + ROCm 7.2.1
Use AITER and the guide’s gfx950/FlyDSL settings. This is a separate AMD kernel path.
Runtime recipe
4 × GB200 GPUs
GPUs within an NVLink system; this is not a count of whole racks.
NVFP4 variant
vLLM + Blackwell kernels
Four Blackwell GPUs are documented; this is a re-quantized artifact, not the original checkpoint at a new flag.
595.15 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
What the serving engine has to do
Most experts are stored. Only a few execute.
Compute follows selected experts.
Weight memory follows the full expert store.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | moonshotai/Kimi-K2.5 |
|---|---|
| Text model type | kimi_k2 |
| Layers | 61 |
| Hidden width | 7,168 |
| Attention / KV heads | 64 / 64 |
| Head dimension | Not verified |
| Routed / selected experts | 384 / 8 |
| Configured positions | 262,144 |
| Documented extension | Not recorded for this checkpoint |
| Inputs → output | text, image, video → text |
| License metadata | modified-mit · custom terms |
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
K2.6 is the direct successor; GLM-5 offers a text-focused sparse-attention comparison.
20 Apr 2026
Kimi-K2.6
Keep the hardware fixed; compare the completed work
1,000B reported total · text + image + video
12 Feb 2026
GLM-5
A large sparse-attention foundation for the GLM line
744B reported total · text
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Budget the vision encoder separately and explain why sparse expert routing does not remove inactive weights.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Kimi-K2.5: release, architecture and deployment”, checked 2026-09-13. Preserve this date and the exact checkpoint when citing.