Runtime recipe
8 × H200 SXM
141 GB per GPU
FP8 weights
vLLM model-specific guide
Single-node TP8 path. Long-context limits depend on remaining cache memory; use the guide’s matching parser and MTP settings.
GLM / MoE
GLM-5.2
A million-token setting backed by index sharing
New to model sizes or GPU memory? Start with weights, parameters and quantization →
GLM-5.2 is worth examining when long repository context is central to the workload. The meaningful engineering change is IndexShare across sparse-attention layers, not just the larger context number.
743Breported total parameters
39Bactive parameters per token
1,048,576configured token positions
Approximate 743B total / 39B active, as reported in the model-specific vLLM guide; not an exact tensor inventory.
Evaluate it for
Large text-agent services investigating long-context retrieval and coding.
Choose another path when
Treating sparse selection as proof that retained KV state or prefill work has vanished.
Release context
Why this release matters
Index sharing reuses selection work across layers. It changes the implementation and memory-access pattern, but a million configured positions still require a realistic request budget and a long-context quality test.
This is the dated Z.ai release-note event; a separate first weight-upload day has not been certified.
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
8 × B200 HGX
180 GB per GPU
FP8 weights + FP8 KV
vLLM model-specific guide
Guide documents a full 1M context configuration. That is not a measured TTFT or concurrent-service guarantee.
1,506.66 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
| Representation | Approximate weight floor | What remains to budget |
|---|---|---|
| 16-bit uniform | 1,486 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
| 8-bit uniform | 743 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
| 4-bit uniform | 371.5 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
Enter the total across the whole model replica, not per GPU: KV or recurrent cache for all concurrent requests, image/audio state and execution workspace. There is no universal default. We additionally leave 15% of each GPU unused.
1,486 GB for hypothetical weights
Enter a request-and-buffer budget to calculate the aggregate capacity requirement.
A count of one does not prove the engine supports this GPU. A count above one does not prove the model shards evenly or communicates efficiently. Approximate 743B total / 39B active, as reported in the model-specific vLLM guide; not an exact tensor inventory.
Architecture in practice
What the serving engine has to do
Store the history. Read the selected parts.
Fewer bytes read can reduce bandwidth pressure.
That does not prove an equal reduction in stored bytes.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | zai-org/GLM-5.2 |
|---|---|
| Text model type | glm_moe_dsa |
| Layers | 78 |
| Hidden width | 6,144 |
| Attention / KV heads | 64 / 64 |
| Head dimension | 192 |
| Routed / selected experts | 256 / 8 |
| Configured positions | 1,048,576 |
| Documented extension | Not recorded for this checkpoint |
| Inputs → output | text → text |
| License metadata | mit |
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
5.3 tests later post-training on the same base lineage; MiniMax M3 offers a different sparse-read design.
14 Aug 2026
GLM-5.3
A post-training release with a changed adoption decision
743B reported total · text
1 Jun 2026
MiniMax-M3
Sparse reads do not automatically mean a small cache
428B reported total · text + image + video
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Distinguish index construction, selected KV reads and stored KV capacity. Explain which work can be reused across layers.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “GLM-5.2: release, architecture and deployment”, checked 2026-09-13. Preserve this date and the exact checkpoint when citing.