Mixture-of-Depths vs Uniform Compute: The Routing Tax on Every Token
Every token in a standard transformer pays for every layer, whether the token is a difficult reasoning step or a comma. Per-token depth routing promises to charge each token only for the layers it needs. On a spreadsheet the saving looks like a straight percentage of FLOPs. On a real accelerator th…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-07 19:28 · DEV Community — AI
Mixture-of-Depths vs Uniform Compute: The Routing Tax on Every Token