Gemma
Google DeepMind
Gemma is a family of different design choices, not just different sizes.
Qwen / Lab profile
Alibaba Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
First time comparing models? Learn the terms before the architecture →
Qwen offers the widest size ladder in this reference. That makes it useful for a disciplined evaluation: start with a small model, identify concrete failures, and move up only when a larger checkpoint fixes enough of them. The family also offers unusually clear dense-versus-MoE comparisons at similar stored size.
Architecture trajectory
What connects these models?
Qwen3.5 and 3.6 combine gated delta networks with full-attention layers; feed-forward blocks can be dense or MoE. Flash-Next adds a separate n-gram table and MTP component. Those are distinct axes: dense versus sparse compute, attention state, and placement of shared model memory.
Shared tables and request cache have different lifetimes.
Model tables do not multiply with every request.
The lookup and host/device placement are model-specific.
Newest first / grouped sequence, not a time scale
The Qwen timeline
Qwen3.8-Flash-Next
The memory placement is part of the architecture
The reported package separates a 125B backbone, 51B n-gram tables and 4B MTP component. Moving the tables off the accelerator is a deliberate placement strategy; it is not ordinary KV-cache offload.
Date corroborated by the FlagOS team’s account of its same-day integration; architecture is sourced to Qwen’s pinned card.
Qwen3.8-27B
A current dense baseline before you scale out
The configuration has 262,144 positions; the documented extension to one million requires additional setup. A newer release and a larger advertised context do not establish better retrieval on your documents.
Qwen3.8-2.4T-A95B
Large-scale text reasoning, with large-scale residency
The open weights arrived after the hosted Max announcement. The downloadable checkpoint is text-only; hosted Max features should not be copied onto its card.
The hosted Max announcement on August 2 does not date this downloadable checkpoint.
Qwen3.6-27B
A clean dense-generation comparison
The later checkpoint retains hybrid attention in a dense model. Dense feed-forward compute and hybrid attention are separate properties; “dense” does not mean every attention layer retains a full KV history.
Qwen3.6-35B-A3B
A smaller expert store for coding-agent experiments
The 35B/3B shape continues the smaller Qwen MoE line. The interesting evaluation is whether later training improves planning, tool use and recovery at the same resident size.
Qwen3.5-0.8B
A cheap baseline for narrow tasks
The March release brought the Qwen3.5 hybrid multimodal design to four dense sizes. This matters because the same family can support a controlled quality-versus-memory study, rather than a comparison across unrelated prompts and templates.
Qwen3.5-2B
A small step up when the tiniest model misses too much
The March release brought the Qwen3.5 hybrid multimodal design to four dense sizes. This matters because the same family can support a controlled quality-versus-memory study, rather than a comparison across unrelated prompts and templates.
Qwen3.5-4B
A practical small-model starting point
The March release brought the Qwen3.5 hybrid multimodal design to four dense sizes. This matters because the same family can support a controlled quality-versus-memory study, rather than a comparison across unrelated prompts and templates.
Qwen3.5-9B
Test whether the middle size earns its memory
The March release brought the Qwen3.5 hybrid multimodal design to four dense sizes. This matters because the same family can support a controlled quality-versus-memory study, rather than a comparison across unrelated prompts and templates.
Qwen3.5-122B-A10B
A middle step before the largest Qwen3.5 model
The February expansion filled the gap between the first 397B release and small-device models. Ten billion active parameters describes the selected path, not the resident model.
Qwen3.5-27B
Dense compute alongside a sparse sibling
They arrived together in the February expansion. Comparing them keeps the generation closer while exposing why parameter count alone cannot predict latency.
Qwen3.5-35B-A3B
The easiest MoE memory lesson to make concrete
The family combines gated delta networks with full-attention layers. Both the expert store and the two types of attention state matter when you raise context or concurrency.
Qwen3.5-397B-A17B
The launch model that established Qwen3.5’s hybrid design
The first release combined multimodal input, hybrid attention and a large sparse expert pool. Subsequent releases carried the family into much smaller memory classes.
Qwen3-Coder-Next
Executable feedback matters as much as model size
Qwen built it on the hybrid Qwen3-Next base and emphasized executable coding tasks, environment interaction and reinforcement learning. The architecture and the agent-training process both explain its position.
The official blog displays February 2; its citation access date is February 3. These are different events.
Which other labs should you compare?
Pair 27B dense with 35B-A3B to study storage versus active compute. Compare the same dense size across 3.5, 3.6 and 3.8 to study a generation update. For a different lab, Gemma 4 supplies dense and MoE multimodal alternatives, while DeepSeek V4.1 offers another conditional-memory design.
DeepSeek V4 / V4.1
DeepSeek
Flash evolves from a post-training update into a different memory system.
Interview preparation
What you should be able to explain
Explain why an 80B-A3B coder can need more memory than a dense 27B, and why a newer hybrid model may require a new cache implementation even when its active count is small.
Study the underlying systems → · Get the illustrated report →
Sources and scope
Checked 13 September 2026. Assessments and comparison candidates are our interpretation of the cited designs, not measured cross-lab rankings. Each timeline entry links to its release evidence; model pages preserve pinned card and configuration sources.