acceptance rate

A small model guesses four tokens and the big one checks them in a single step. That converts idle bandwidth into tokens, and the arithmetic tells you exactly which batch size stops it working.

All three guess tokens for the big model to check. They differ in where the guess comes from, how many candidates they verify per step, and how often the target agrees. Acceptance length per unit of draft cost is the number that decides.

It helps most exactly where these models are weakest, which is single-user latency at low batch, and the published gains are larger than on dense models for a reason. The acceptance arithmetic, what it costs at high concurrency, and the measurement that decides.

Speculative decoding buys latency with spare compute, and a busy fleet has no spare compute. The speedup formula with acceptance rate and draft cost inside it, the batch size where the gain turns negative, where the draft runs and what it costs, the monitoring that catches a silent regression, and rollout per class.

A geometric process with a cap, a closed form that a twenty-line simulation confirms, and a speedup that turns negative when the verification stops being free. The table across acceptance rates and draft lengths, and the crossover from a 2.35 times win to a 0.75 times loss under two stated cost regimes.