NarrowSpec shows draft model width can differ from target in speculative decoding
A new arXiv paper, NarrowSpec, argues that the draft model's hidden width need not match the target model's in speculative decoding, a technique that speeds up LLM generation using a small draft model.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-19 05:00 · arXiv cs.CL
NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding