Runtime recipe
8 × GB300 GPUs
GPUs within an NVLink system; this is not a count of whole racks.
Mixed MXFP4
vLLM K3 image
Minimum documented GB300 configuration. CUDA 13 image requires an r580+ driver; production concurrency may need more GPUs.
Kimi / MoE
Kimi-K3
A cluster model with a new attention design
New to model sizes or GPU memory? Start with weights, parameters and quantization →
K3 belongs on a shortlist for long-context coding and multimodal agents when a dedicated cluster is already a realistic option. The interesting change is the combination of Kimi Delta Attention, gated latent attention and Attention Residuals; scaling from K2’s 1T to 2.8T is only part of the story.
2,800Breported total parameters
104Bactive parameters per token
1,048,576configured token positions
Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size.
Evaluate it for
Long repository tasks and multimodal agent evaluations on a high-memory cluster.
Choose another path when
A single workstation, a small H100 allocation, or a deployment that requires a mature generic attention backend.
Release context
Why this release matters
K2 used latent attention throughout. K3 combines a recurrent state with selective attention, changing how an engine manages cached prefixes and long requests. Its open release includes supporting kernels and communication work, so adopting the model also means adopting a newer runtime stack.
The July 16 announcement preceded the public weights by eleven days.
Release evidence · Announced 2026-07-16 · Weights 2026-07-27 · Follow the Kimi timeline →
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
8 × MI355X
288 GB per GPU
Mixed MXFP4
vLLM ROCm K3 image
MI350X is also documented. Use the K3-specific ROCm path; a generic CUDA launch command does not transfer.
1,560.86 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
What the serving engine has to do
Two kinds of layers retain two kinds of state.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | moonshotai/Kimi-K3 |
|---|---|
| Text model type | kimi_linear |
| Layers | 93 |
| Hidden width | 7,168 |
| Attention / KV heads | 96 / 96 |
| Head dimension | Not verified |
| Routed / selected experts | 896 / Not verified |
| Configured positions | 1,048,576 |
| Documented extension | Not recorded for this checkpoint |
| Inputs → output | text, image, video → text |
| License metadata | kimi-k3 · custom terms |
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
K2.7-Code is the lower-storage Moonshot coding baseline; Qwen’s 2.4T checkpoint is a text-only cluster alternative.
12 Jun 2026
Kimi-K2.7-Code
A coding upgrade without a new size class
1,000B reported total · text + image + video
12 Aug 2026
Qwen3.8-2.4T-A95B
Large-scale text reasoning, with large-scale residency
2,400B reported total · text
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Explain why 104B active parameters can reduce token compute while 2.8T stored parameters still determine cluster residency. Then describe the recurrent-state and attention-cache split.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Kimi-K3: release, architecture and deployment”, checked 2026-09-13. Preserve this date and the exact checkpoint when citing.