Turn retained context into a cache capacity limit

Full-attention cache size depends on KV heads, layers, head width, precision and retained length. At 320 KiB per token, a 12 GB budget holds nine 4,000-token sequences before allocation and placement overhead.

14 MIN · PLUS