The target runs one forward pass over all k drafted tokens, compares its distribution against the draft's at each position, and keeps the longest prefix that survives a coin flip. The acceptance rule that makes the output exactly a target sample, the resampling at the first rejection, and the one-integer rollback.
Implement the verification step of speculative decoding, including the rollback of the KV cache after a rejection.
The target runs one forward pass over all k drafted tokens, compares its distribution against the draft's at each position, and keeps the longest prefix that survives a coin flip. The acceptance rule that makes the output exactly a target sample, the resampling at the first rejection, and the one-integer rollback.
Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.
The concepts behind this question
Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.
Scored on stating the accept-reject rule correctly including the residual distribution, on knowing the output distribution is exact rather than approximate, and on the rollback being a length update rather than a memory operation.
No comments yet — be the first to share your approach.
