Qwen
Alibaba Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
Nemotron / Lab profile
Compression and runtime support must travel together.
First time comparing models? Learn the terms before the architecture →
The covered October models combine vision inputs, a Mamba-attention backbone and sparse experts. Super VL provides a mixed-NVFP4 serving path; Bolt further compresses the architecture and packed weights.
Architecture trajectory
Mamba state, attention KV and visual preprocessing are different allocations. Bolt also varies expert counts by block, so inspect nested configuration instead of copying the global default.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Newest first / grouped sequence, not a time scale
A 24 GB checkpoint still needs headroom
Compression changes per-block expert counts, and the quantization config mixes formats. The global 512-expert default does not describe each block in this exported architecture.
The publisher card explicitly dates the Hugging Face release.
Follow the mixed-precision recipe all the way to runtime state
The configuration separates forty Mamba blocks, forty expert blocks and eight attention blocks. Mamba cache precision and restoration are part of the serving contract, not properties inferred from the weight dtype.
The publisher card explicitly dates the Hugging Face release.
Compare parent and compressed checkpoints on identical visual tasks, output budgets and concurrency. A smaller artifact is only useful if the quality and latency requirements still pass.
Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
Gemma
Gemma is a family of different design choices, not just different sizes.
Interview preparation
Reconcile 120B nominal parameters with 75.01 GB mixed payload, then explain why Bolt’s 24.08 GB is not a whole-service peak.
Study the underlying systems → · Get the illustrated report →
Covered checkpoints carry their own research dates. Assessments and comparison candidates are our interpretation of the cited designs, not measured cross-lab rankings. Each timeline entry links to its release evidence; model pages preserve pinned card and configuration sources.