capacity planning

Separate H200 bandwidth and capacity gains from measured speedup, then test realistic memory headroom and cost per useful completed workload.

Compare actual useful work and memory units, validate Neuron support, and calculate when rental savings repay porting and operations.

Work two audio timelines: input accumulation before transcription and playback starvation after synthesis. Compare real-time factor, response delay and safe concurrent capacity without mixing their units.

Translate active streams into replicas using growth reservations and matched latency tests. Show peak and failure headroom separately, with no invented benchmark.

Size for a tested peak and a stated failure, then price an explicit daily scaling schedule. Keep request-slot occupancy separate from GPU utilization.

Count endpoint, leaf, aggregation and core ports for an explicit 4,096-GPU design. Compare 1:1 and 2:1 without confusing switch ports, links or optics.

Derive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.

Compare the cost of committed capacity with the demand each additional GPU would actually serve. Keep billing discounts separate from capacity guarantees, and account for bursts without counting the same work twice.

Derive a checkpoint interval from measured job interruptions and exposed save cost. Trace durability, lost work and recovery before treating an estimate as goodput.

Size a spare pool by separating nodes in repair from ready replacements. Work through a 2,048-node example, then check placement, recovery time and shared failure domains.

Connect POST-and-poll semantics to durable item results, retry ownership and deadline capacity. Explain how cheap batch execution can still miss the delivery contract.

Apply a learned merge order without retraining on the input. Test overlapping pairs, preserve UTF-8 bytes, and turn tokenizer differences into a capacity estimate.