mla
AI infra interview questions tagged mla, across every topic.
4 questions · 2 unlocked for you
Concepts behind "mla"
The curriculum that explains the ideas these questions test.
Foundational
Multi-Head Latent Attention and Sparse IndexersGrouped-query attention shrank the KV cache by sharing key and value heads. Latent attention goes further by caching a single compressed vector per token per layer and reconstructing the heads on the fly, which cuts the cache by tens of times rather than by a small factor. On top of that, sparse indexers pick a few thousand relevant positions per query instead of attending to all of them, turning the quadratic term linear at long context. Both are now standard in open-weights models, and both change how a serving deployment is sized.🧮 Open Weights & Serving Engines
Foundational
KV Cache SizingThe KV cache is the memory that decides how many users a serving replica can hold and how long their context can be. Its size per token comes from four numbers in the model's config file (layers, KV heads, head dimension, bytes per element) and one formula; multiplied by context and concurrency it is the number every capacity plan is built on. This page derives it, works it for four models including an MLA one, and shows the two places candidates get it wrong by a factor of eight.🧮 Napkin Math & Capacity
Core
Attention Variants: MHA, GQA, MQA and MLAThe KV cache scales with the number of key-value heads, and the four attention variants differ exactly there: multi-head keeps one KV head per query head, multi-query keeps one for all, grouped-query shares one across a group, and multi-head latent attention caches a compressed latent instead of keys and values at all. For Llama 3.1 70B that is the difference between 2.6 MB and 320 KB per token; for DeepSeek-V3 it is about 70 KB. The variant a model was trained with is a serving decision made before the first GPU was bought.🚀 Inference & ServingSign in