Increasing active parameters per token in MOE (Qwen 35B A4B+) reduce reasoning token by 8.5% - and you don't need to train or finetune!
This story is from 2026-09-03. It is preserved in the archive; the latest stories are on the live feed.
I want to share a short paper just published exploring a simple but surprisingly effective optimization for sparse MoE reasoning models. The idea: Instead of retraining anything, we just tweak the router at runtime . Specifically, we expand the expert selection budget (N≥K N ≥ K ) only in the late…
Read the full story at r/LocalLLaMA ↗
Timeline · 2 reports
- 2026-09-05 07:16 · r/learnmachinelearning
Increasing active parameters per token in MOE (Qwen 35B A4B+) reduce reasoning token by 8.5% - and you don't need to train or finetune! - 2026-09-03 21:58 · r/LocalLLaMA
Increasing active parameters per token in MOE (Qwen 35B A4B+) reduce reasoning token by 8.5% - and you don't need to train or finetune!