tensor parallel

Normalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Build a measured first-token budget and test the smallest serving design that meets it, including cold prompts, queueing and delivery.

Construct concrete TP and DP groups, then verify physical devices and estimate traffic with per-rank rather than whole-model bytes.

Compare tensor, pipeline and expert traffic with per-request KV transfer. Work through declared payload and latency assumptions before choosing a serving network.

Use actual rank placement and measured job duration to decide whether waiting for locality beats starting immediately.