Qwen
Alibaba Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
LFM / Lab profile
Local agents still need a complete state budget.
First time comparing models? Learn the terms before the architecture →
These checkpoints provide a smaller-device comparison to the frontier cluster releases. LFM2.5-24B-A2B uses sparse experts; LFM3-1.7B uses a much smaller dense model with three kinds of sequence-processing layers.
Architecture trajectory
Short convolutions retain finite history. Sliding attention retains a bounded window when the implementation reclaims it. Full attention keeps growing history. Sum the appropriate state for each family.
Convolution history is not an attention KV array.
Global layers keep growing after local windows fill.
Newest first / grouped sequence, not a time scale
Small weights can still carry a growing cache
The 24-layer stack has 13 short convolutions, five sliding-attention layers and six full-attention layers. Only the local portions can stop growing with context.
Publisher release post; repository creation predates the announcement.
Active parameters are a compute clue, not a device budget
The 40-layer design combines thirty short convolutions with ten grouped-query attention layers. Routing selects four of 64 experts; their stored tensors remain part of residency.
Publisher release post; not the earlier repository creation date.
Compare with small Qwen or Gemma checkpoints under the same local task and memory limit, including output length and tool parsing.
Qwen
One family spans tiny dense models, coding MoEs and conditional memory.
Gemma
Gemma is a family of different design choices, not just different sizes.
Interview preparation
Explain why a 1.73B model can still use 384 MiB for its full-attention KV at 32,768 positions, before other state.
Study the underlying systems → · Get the illustrated report →
Covered checkpoints carry their own research dates. Assessments and comparison candidates are our interpretation of the cited designs, not measured cross-lab rankings. Each timeline entry links to its release evidence; model pages preserve pinned card and configuration sources.