You need to deploy GLM-5.3. How many GPUs do you need, and what would you verify first?

TL;DR: The pinned GLM-5.3 tensor index contains 755.6 GB before runtime allocations. That is a starting point, not a GPU count. Size the product's cached positions, account for the engine's per-rank cache layout, and reserve measured runtime memory. An eight-GPU candidate can fit a modest workload in the worked example below; the same candidate fails badly at 256 sequences of 32,768 positions if its latent cache is replicated across tensor-parallel ranks.

How to approach it

A product team wants the model running this afternoon. Before choosing GPUs, find out how many requests must run at once and how long their prompts and outputs can be. Those requests need memory alongside the weights. Pin the model revision and engine build, then estimate the largest allocation on any one GPU. Reading config.json to Size a Model You Have Never Run explains the general method.

A strong answer

The official tensor index reports 755,617,140,416 stored tensor bytes, checked September 7, 2026. The release mixes FP8 weights with higher-precision tensors. Do not add an arbitrary 1% for scales: a full 128-by-128 FP8 block with a four-byte scale has 4 / 16,384 = 0.0244% scale overhead, and the index already includes stored scale tensors. Loading can still repack or convert them.

The configuration gives 78 main layers, latent rank 512 and rotary key width 64. This model uses multi-head latent attention: instead of storing full keys and values per head, it caches one compressed latent vector per token (rank 512) plus a small positional key (width 64). For a BF16 compressed-latent cache:

component bytes per position = 78 × (512 + 64) × 2
                             = 89,856 B = 87.75 KiB

Example product envelope: 64 sequences, each reaching 8,192 positions
positions = 64 × 8,192 = 524,288
one copy of this component = 47,110,422,528 B = 47.11 GB

This excludes sparse-indexer state, a possible prediction layer, allocation rounding and other buffers. It also describes one copy, not how an engine distributes it. Multi-Head Latent Attention and Sparse Indexers explains why latent state is not a set of independently shardable KV heads.

Suppose a candidate uses eight tensor-parallel ranks, each retaining that latent cache, and ideally balances the stored weights. For teaching, assume 24 GB per rank for all remaining allocations and use a 260 GB planning budget per rank. Those are explicit assumptions to replace with an engine allocation report, not measurements of this model.

This supports testing an eight-GPU, high-memory configuration. It does not certify eight particular GPUs: unequal tensor placement, backend support, actual allocatable HBM, and the 24 GB assumption still need checking. A hypothetical 160 GB usable budget fails this example by 5.56 GB per rank.

Now change only the product envelope to 256 sequences at 32,768 positions. One latent-cache copy becomes 753.77 GB. Replicating it across eight ranks gives 872.22 GB per rank after the same weights and allowance. Dividing the aggregate cache by eight would incorrectly predict only 212.67 GB per rank.

The reversal condition is concrete: if the required live positions exceed the tested cache pool, reduce admitted workload or evaluate supported pipeline/context/data-parallel layouts. Adding tensor-parallel ranks alone may not reduce a replicated latent cache. Lowering the maximum request length limits allowed growth; paged engines generally allocate cache blocks as needed rather than reserving the maximum separately for every request.

Use the model's engine recipes as versioned starting points. Confirm architecture, quantization, indexer and topology support. Save the reported per-rank weights, cache blocks, and workspace; reconcile the largest rank. Then test long prompts, decode growth, stop behavior, chat/tool parsing and an offered-load sweep against p99 latency and completion targets. A launch command that starts successfully has passed only the first test.

What interviewers probe next

• “Why not seven GPUs?” The chosen parallel implementation imposes divisibility and placement constraints. Check its supported degrees; head counts alone do not define every valid execution layout.
• “Does FP8 KV halve the whole allocation?” Only compatible cache tensors shrink, with scales and auxiliary state still present. Validate quality and backend support; weights and workspaces do not automatically halve.
• “Why not choose GPUs from active parameters?” Routing reduces per-token expert work. Resident serving still stores all required experts somewhere.
• “What observation changes your answer?” A measured peak per-rank allocation or sustainable concurrency outside the assumed envelope. Recompute with that evidence before reserving production capacity.

Common mistakes

• Treating 755.6 GB of checkpoint tensors as either complete runtime memory or 753 GB plus an invented scale percentage.
• Calling 87.75 KiB the entire cache allocation and dividing it by tensor-parallel degree without checking placement.
• Using maximum context, maximum concurrency and a constant percentage overhead without defining their simultaneous workload.
• Promising a throughput SLO from a memory fit.

Key takeaways

• The pinned artifact stores 755.6 GB; a BF16 main-layer latent component uses 87.75 KiB per position.
• At 64 × 8,192 positions, one cache copy is 47.11 GB. At 256 × 32,768 it is 753.77 GB.
• The hardware decision is governed by the largest rank and measured workload capacity, not aggregate HBM alone.