AI Infra Interviews logo

Verify KV-cache correctness with a CPU attention lab

Compare full, cached and paged attention on identical synthetic inputs. Reject six deliberate indexing bugs and separate numerical agreement from modeled history-gathering work.

PremiumIntermediate

AI Infra Interviews editorial · Updated

How can you test KV-cache correctness without a GPU?

Compare an independent full-attention reference with incremental and paged execution on the same finite inputs. Vary chunk boundaries, positional offsets, valid lengths and head mappings, and verify that deliberately broken variants fail. This CPU project checks one synthetic attention layer. It does not establish complete-model quality, device memory safety or serving speed.

Read the primary sources
What you build
A cache-evidence.json report, five agreeing execution paths and six rejected faults.
Environment
Python 3.10+ · CPU only · no model download
Plan your session
60–90 minutes. A planning estimate, not measured runtime.
Before you begin
Run a Python script and read nested arrays. Understand prefill, decode and the purpose of the KV cache.
13 CPU tests executed. Executed on Linux with Python 3.12.3. Includes 192 partition/layout/seed comparisons, page and head-layout boundaries, six injected faults and no-overwrite checks. No LLM, MLX or GPU benchmark was run.
  1. 01Reference

    Compute causal attention from the complete synthetic input without mutable cache state.

  2. 02Change execution

    Reuse state across chunks and translate logical positions through permuted physical pages.

  3. 03Reject faults

    Locate the first wrong output and retain the exact source and numerical evidence.

Follow the changing state through the project. Each boundary has a check in the walkthrough.

Continue with Premium

Premium includes this guide’s complete walkthrough and downloadable project files. The technical overview, prerequisites and verification notes above are public.

View Premium access

Start with Train a tiny language model from scratch with PyTorch (Free to read).

Primary sources

Checked . Source review and execution checks are described separately above.