MiniMax M3
MiniMax
Study how sparse attention changes what the GPU reads.
Ling / Lab profile
Hybrid state and checkpoint identity decide the rollout.
First time comparing models? Learn the terms before the architecture →
The covered Ling releases share a 560B expert store and a 25B active path. Their relevance is the combination of recurrent delta attention and gated latent attention, followed by a separately evaluated post-training update.
Architecture trajectory
Count 63 KDA and nine gated MLA layers separately. The nine history-retaining layers prevent a whole-model constant-state assumption.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Newest first / grouped sequence, not a time scale
A post-training update needs its own deployment evidence
The two releases share 560B total and 25B active counts. Their artifact identity, task results and rollout acceptance still need separate records.
The repository records the weight upload on this date. The 1019 suffix is not proof of an October 19 public weight release.
Count recurrent and latent state separately
Sixty-three Kimi Delta Attention layers retain recurrent state; nine gated multi-head latent-attention layers retain history. Cache reuse must restore both.
Dated weight-upload commit; not inferred from repository creation or an API listing.
Use MiniMax as another hybrid cluster candidate and a smaller Qwen checkpoint when residency is the limiting constraint.
MiniMax M3
Study how sparse attention changes what the GPU reads.
Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
Interview preparation
Reconcile the card’s million-token claim with its 262,144-position pinned config before approving a long-context service.
Study the underlying systems → · Get the illustrated report →
Covered checkpoints carry their own research dates. Assessments and comparison candidates are our interpretation of the cited designs, not measured cross-lab rankings. Each timeline entry links to its release evidence; model pages preserve pinned card and configuration sources.