← Coding for Infra / 37

hardNew

Implement sliding-window attention without confusing token positions with ring-buffer slots

Build a small CPU reference, overwrite a bounded KV ring, and compare every step with a recomputed attention window. Keep model semantics separate from eviction.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Build a small CPU reference, overwrite a bounded KV ring, and compare every step with a recomputed attention window. Keep model semantics separate from eviction.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 303 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🚀 Inference & Serving

The KV CacheThe KV cache saves attention keys and values from processed tokens. Follow one decode step, calculate its memory cost, and see why longer conversations reduce the number of requests a GPU can hold.

💻 Coding for InfraSign in

Rate-Limiting AlgorithmsRate limiters control how much work a caller may submit over time. Token buckets, queues and sliding windows allow different burst and waiting behavior.

🧮 Open Weights & Serving Engines

Multi-Head Latent Attention and Sparse IndexersLatent attention changes what is stored; sparse indexing changes which positions receive expensive attention; recurrent layers summarize history in state. Learn to budget each mechanism separately, including indexer work, cache replication and hybrid prefix snapshots.

🚀 Inference & ServingSign in

Attention Variants: MHA, GQA, MQA and MLACompare MHA, GQA, MQA and compressed MLA by the state each retains. Derive cache payload in consistent units, distinguish storage capacity from serving throughput, and account for hybrid layers and rank replication.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement symmetric INT8 quantization and prove where the reconstruction-error bound applies Closest to this topicImplement a LoRA forward pass and merge two adapters without inventing cross terms Closest to this topicWhat is the KV cache, and why does it keep growing while a request is being served?

DISCUSSION · 0

No comments yet — be the first to share your approach.