attention

Follow the running maximum and weighted sum across attention tiles, with executable arithmetic and an honest memory-traffic comparison.

Follow a token into retained attention state, derive bytes per position for a 70B model, and distinguish present cache usage from the room needed to finish a response.

Draw query-head sharing, count actual stored state and compare payloads with explicit model geometry and binary units.

Calculate retained cache and prefill work for a stated 128k model shape, then choose scheduling and memory policies from measured limits.

Turn public metadata into a deployment candidate with explicit uncertainty. Inspect architecture, stored formats and engine support, then work a per-rank memory example that explains why an aggregate fit can fail.

Confirm that requests are waiting for usable state capacity. Separate allocated memory, live blocks and evictable prefixes, then compare admission, lower-precision state, prefix sharing and hardware changes with explicit assumptions.

A context-window limit is not a latency or concurrency guarantee. Budget retained state, prefill and shared service time; distinguish cache reuse from eliminating work; choose isolation only when measured interference justifies its cost.

Build a small CPU reference, overwrite a bounded KV ring, and compare every step with a recomputed attention window. Keep model semantics separate from eviction.