mla

Draw query-head sharing, count actual stored state and compare payloads with explicit model geometry and binary units.

Separate checkpoint memory, expert assignments and retained MLA state. The active parameter count alone cannot predict per-rank fit or token latency.

Derive a deployment candidate from the official tensor index, then account for latent-cache replication and the product workload. A worked example shows why enough aggregate HBM can still leave every GPU out of memory.

Size a hypothetical trillion-parameter MoE from its weight placement, attention-state layout and expert traffic. Use rack specifications as bounds, then measure the serving workload before promising throughput.