Speculative decoding explained: draft heads, what "acceptance" really counts, and why every tok/s figure should say if a drafter was on

A language model writes one token per pass through its weights, and that pass is the slow part. Speculative decoding adds a cheap guesser, a "drafter": it proposes the next few tokens, the model checks them all in one pass and keeps the run of right ones. Guess four and all four are right: five tok…

Read the full story at r/machinelearningnews ↗

Timeline · 1 report

1. 2026-11-13 01:05 · r/machinelearningnews
   Speculative decoding explained: draft heads, what "acceptance" really counts, and why every tok/s figure should say if a drafter was on