Four tokens, eight experts each: what should actually be counted during MoE verification?
The usual speculative-decoding intuition is “load the target weights once, check several draft tokens together.” With a sparse MoE, those tokens can ask for different weights. In NVIDIA’s Ling-3.0-flash example, each token selects eight of 512 routed experts per MoE layer, plus a shared expert. Ver…
Read the full story at r/deeplearning ↗
Timeline · 1 report
- 2026-10-29 04:45 · r/deeplearning
Four tokens, eight experts each: what should actually be counted during MoE verification?