fp8

Use proportional bit layouts and adjacent values around one to distinguish range from precision. Then choose formats from numerical tests and actual kernel support.

Compare the cost of serving the same workload at the same latency and quality. A memory-traffic estimate explains what each GPU might improve; measured throughput and your actual rental price determine the purchase.

Compare the mechanisms in FlashAttention versions 1, 2 and 3: IO savings, work partitioning, asynchronous dependencies and FP8 accuracy.

Derive when operand scales can be applied after a dot product and when they must follow its K blocks. Separate integer zero points, floating-point range, scale granularity and accumulator behavior.

Separate gradient range from update precision using a small numerical example, and make the training-state recipe explicit.

Separate FP8 input quantization from accumulation error. A small representable-value example shows when a shared scale loses small values, while the DeepSeek-V3 report supplies one carefully scoped training recipe.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Separate weight storage, matrix arithmetic and attention-cache precision. Choose a supported candidate using task quality, memory and service measurements.

Build a measured first-token budget and test the smallest serving design that meets it, including cold prompts, queueing and delivery.

Compare actual encoded tensors, per-rank runtime fit, supported kernels and held-out task quality. Four-bit weight-only kernels can accelerate wider arithmetic, and smaller weights need not reduce the GPU count.

Define whether half the footprint means checkpoint bytes or runtime memory. Choose a supported recipe, measure which tensors it changes, separate calibration from held-out evaluation, and validate quality and serving cost.

Map a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.

Calculate FP8 savings by separating weights, KV cache and other step costs. Compare the same batch first, then check whether larger batches meet latency and quality targets.