Runtime recipe
8 × B300 HGX
288 GB per GPU
MXFP4 routed experts, BF16 elsewhere
vLLM 0.34.0+ (recipe baseline)
TP8 baseline on one B300 node or two GB300 trays. The recipe states throughput was not tuned.
Kimi / MoE
Kimi-K3.1-Pro
K3's stack with sparse attention in place of full attention
New to model sizes or GPU memory? Start with weights, parameters and quantization →
K3.1 Pro belongs on the same shortlist as K3 for long-context agent work on a dedicated cluster. The change to study is architectural: the 24 full-attention layers now use gated sparse attention with a top-2048 index, which changes what a long request reads, while the 2.8T expert store and the cluster requirement stay.
2,800Breported total parameters
104Bactive parameters per token
1,048,576configured token positions
Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size.
Evaluate it for
Long-context coding and multimodal agent evaluations on a multi-GPU Blackwell or MI355X node.
Choose another path when
A single node smaller than eight current-generation GPUs, or a runtime without kernels for both recurrent-state and sparse attention.
Release context
Why this release matters
Moonshot keeps K3's 69 Kimi Delta Attention layers and replaces the gated latent-attention layers with gated DeepSeek-style sparse attention. Selected experts per token are 16 of 896, with two shared experts.
Announced with API access on 24 October with weights promised by 4 November; the weights repository's first commit is dated 2 November.
Release evidence · Announced 2026-10-24 · Weights 2026-11-02 · Follow the Kimi timeline →
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
8 × GB300 GPUs
GPUs within an NVLink system; this is not a count of whole racks.
MXFP4 routed experts, BF16 elsewhere
vLLM 0.34.0+ multi-node TP8
Default command spans two GB300 trays with TP8 across nodes; network interfaces must be set for Gloo and NCCL.
Runtime recipe
8 × MI355X
288 GB per GPU
MXFP4 routed experts, BF16 elsewhere
vLLM ROCm K3.1 image
One node at TP8 with the AMD AITER profile. The recipe marks it for throughput measurement only and asks for about 6 GiB free per GPU.
1,561.22 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
What the serving engine has to do
Two kinds of layers retain two kinds of state.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | moonshotai/Kimi-K3.1-Pro |
|---|---|
| Text model type | kimi_linear |
| Layers | 93 |
| Hidden width | 7,168 |
| Attention / KV heads | 96 / 96 |
| Head dimension | Not verified |
| Routed / selected experts | 896 / 16 |
| Configured positions | 1,048,576 |
| Documented extension | Not recorded for this checkpoint |
| Inputs → output | text, image → text |
| License metadata | kimi-k3.1-pro · custom terms |
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
K3 isolates the attention change on the same expert store; DeepSeek V4.1 Flash-1017 is a smaller conditional-memory alternative for similar agent tasks.
27 Jul 2026
Kimi-K3
A cluster model with a new attention design
2,800B reported total · text + image + video
17 Oct 2026
DeepSeek-V4.1-Flash-1017
Same footprint, new weights: a clean post-training comparison
552B reported total · text + image
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Compare K3 and K3.1 Pro layer by layer. Explain why replacing full attention with sparse attention reduces long-context read traffic without shrinking the expert store or the recurrent state.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Kimi-K3.1-Pro: release, architecture and deployment”, checked 2026-11-06. Preserve this date and the exact checkpoint when citing.