Open-Weights Models of 2026

TL;DR: Read four numbers per model and ignore the headline. Total parameters tell you what the weights weigh and therefore how many GPUs you need to hold them. Active parameters per token tell you what decode actually reads and therefore how fast it can possibly run. The attention design tells you what the KV cache costs per token, which now varies by more than an order of magnitude between models of similar size. And the released format tells you whether you are serving FP8, MXFP4 or bf16, which changes the first number by two to four times. As of September 2026 the three releases that come up are GLM-5.3, which Z.ai published on 14 August 2026 at 753B total with roughly 43B active and a compressed-latent attention with a sparse indexer; Kimi K3, whose weights Moonshot posted on 26 July 2026 at 2.8T total with 104B active across 896 experts; and DeepSeek V4, whose Flash variant went open under MIT on 31 July 2026 at 284B total with 13B active. Every one of them is sparse, and every one of them has an attention design chosen to make the KV cache small.

The four numbers, for the models people ask about

Figures below come from the published model cards and config.json files, checked in September 2026. Model cards change, so re-check before quoting.

Three patterns run across all of them and each has a deployment consequence.

Pattern one: sparsity ratios have become extreme

active fraction = active parameters / total parameters
  GLM-5.3          43 / 753   = 5.7%
  Kimi K3         104 / 2,800 = 3.7%
  DeepSeek V4-Pro  49 / 1,600 = 3.1%
  DeepSeek V4-Flash 13 / 284  = 4.6%
  for comparison, a dense 70B model is 100%

what this does to a deployment
  memory is sized by TOTAL, because every expert must be resident somewhere
  decode speed is bounded by ACTIVE, because only the selected experts are read per token
  so the two constraints have decoupled: you buy GPUs for capacity and get throughput from
    a much smaller number
sanity: Kimi K3 needs memory for 2.8T parameters and reads 104B per token, a ratio of 27 to 1,
        which is why these models are cheap to run per token and expensive to host at all

Pattern two: the KV cache stopped being proportional to size

Every model in that table uses an attention design that compresses or eliminates most of the KV cache. Multi-Head Latent Attention and Sparse Indexers covers the mechanism. The consequence for capacity planning is that KV per token no longer follows from parameter count, and the old habit of estimating it from layers times heads gives an answer that is wrong by more than an order of magnitude.

GLM-5.3, from its published config
  it caches a compressed latent of kv_lora_rank = 512 plus a rope part of 64, per layer
  KV per token = (512 + 64) x 2 B x 78 layers = 89,856 B = 87.75 KB

what the same model would cost with ordinary multi-head attention
  2 x 78 layers x 64 heads x 192 head_dim x 2 B = 3,833,856 B = 3.66 MB per token
  ratio = 3.66 MB / 87.75 KB = 41.7x

and against the corpus anchor
  a 70B GQA model caches 320 KB per token
  GLM-5.3 at ten times the parameters caches 87.75 KB, about a quarter as much
sanity: at 1M context one GLM-5.3 sequence holds 89,856 x 1,048,576 = 94.2 GB of KV, which
        fits in one B300's 288 GB and would need 3.84 TB under plain multi-head attention

Pattern three: the released format is part of the architecture

Serving a model in the format its authors released is almost always the right starting point, because that format is what they evaluated. Weight Formats: FP8 Blocks, MXFP4 and AWQ covers when to deviate and what it costs.

What this landscape means for an infrastructure role

The interview question behind all of this is usually "could you stand this up". The capabilities that answer it are reading a model card and a config.json and deriving the footprint yourself, knowing which engine supports the model's attention and quantization on day zero, and being able to say how many GPUs of which type it takes and why. Kimi K3 is the clearest example: the vLLM project's own launch note for it states that it needs at least one eight-GPU B300 node, or a minimum of sixteen B200 or GB200 GPUs on the previous generation, and that is a sentence an engineer should be able to derive rather than look up.

What interviewers are listening for

Four numbers instead of one. A candidate who says "Kimi K3 is 2.8 trillion parameters" has read a headline; a candidate who says "2.8T total so it needs memory for all of it, 104B active so decode reads a twenty-seventh of that, experts in MXFP4 so the footprint is much smaller than the count suggests, and a hybrid attention so most layers have no growing KV cache" has read the model card. Interviewers also listen for dating, because this landscape turns over every few months and confident numbers with no date attached are a warning sign.

Key takeaways

• Read total parameters for capacity, active parameters for decode speed, attention design for KV, and released format for bytes per parameter.
• As of September 2026: GLM-5.3 is 753B with about 43B active, Kimi K3 is 2.8T with 104B active, DeepSeek V4-Flash is 284B with 13B active, and all are sparse at 3 to 6 percent.
• GLM-5.3's compressed-latent attention caches 87.75 KB per token against 3.66 MB for the same shape under plain multi-head attention, a factor of 41.7.
• One million tokens of GLM-5.3 KV is 94.2 GB, which fits in a single B300.
• Serve in the format the authors released, because that is the configuration they evaluated.