expert parallelism

Separate per-expert submission, small-grid scheduling and partially filled tiles. A grouped kernel can share work across experts, but routing, packing and output combination still determine the layer’s cost.

Follow one token to two expert ranks and back, then count logical assignments separately from physical network transfers.

Follow expert assignments from logical payload to off-rank and cross-node traffic. Compare dispatch and combine dtypes, receiver skew and local-domain placement.

Size a hypothetical trillion-parameter MoE from its weight placement, attention-state layout and expert traffic. Use rack specifications as bounds, then measure the serving workload before promising throughput.

Separate total and active parameters, then size persistent state, approximate compute and token dispatch. Compare supported parallel layouts using per-rank memory, measured communication and recovery.