The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts
arXiv:2609.22156v1 Announce Type: new Abstract: Speculative decoding in Mixture-of-Experts (MoE) models faces the problem of unstable verification cost caused by input-dependent expert loading. To study the physics of this process, we formulate speculation-budget selection as an offline Stochastic…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-09-22 04:00 · arXiv cs.LG
The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts