moe

Separate Blackwell chip gains from the 72-GPU domain, follow a token to its expert owners, and test placement instead of dividing model bytes by 72.

Compare 64 or 72 useful GPUs, identify the communication that could improve, and test whether a larger NVLink domain justifies the operational requirements.

Use cuBLAS or cuBLASLt when its supported operation fits. CUTLASS exposes kernel building blocks for custom layouts, fusion and scheduling, but each choice needs a whole-operation benchmark and an ownership plan.

Separate per-expert submission, small-grid scheduling and partially filled tiles. A grouped kernel can share work across experts, but routing, packing and output combination still determine the layer’s cost.

Follow one token to two expert ranks and back, then count logical assignments separately from physical network transfers.

Use a ten-expert example to separate assignments, rank load and dropped routes, then compare balancing controls against quality.

Separate checkpoint memory, expert assignments and retained MLA state. The active parameter count alone cannot predict per-rank fit or token latency.

Derive a deployment candidate from the official tensor index, then account for latent-cache replication and the product workload. A worked example shows why enough aggregate HBM can still leave every GPU out of memory.

Turn public metadata into a deployment candidate with explicit uncertainty. Inspect architecture, stored formats and engine support, then work a per-rank memory example that explains why an aggregate fit can fail.

Find whether expert placement, small batches or communication makes the deployment slower. Compare equivalent workloads and inspect the worker that holds up each step.

Use the actual tensor index to size Kimi K3, then separate a published hardware starting point from production capacity. Account for mixed precision, hybrid recurrent state, parser behavior, and the workload behind a throughput claim.

Check the bound’s assumptions, locate the critical path, and quantify which optimization can actually improve one user’s decode latency.

Compare committed output per complete draft-and-verify cycle with ordinary decoding. Derive the acceptance-length model, separate it from engine counters, and test latency, goodput and memory across the actual workload.

Separate resident parameters, active work and the union of experts touched by a batch. Compare toy dense and MoE budgets without promising a fixed speedup.

Follow expert assignments from logical payload to off-rank and cross-node traffic. Compare dispatch and combine dtypes, receiver skew and local-domain placement.

Size a hypothetical trillion-parameter MoE from its weight placement, attention-state layout and expert traffic. Use rack specifications as bounds, then measure the serving workload before promising throughput.

Separate total and active parameters, then size persistent state, approximate compute and token dispatch. Compare supported parallel layouts using per-rank memory, measured communication and recovery.