MoRE: Scaling mixture of experts with hardware-aware low-rank routing
arXiv:2609.36301v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) layers are central to frontier language models, and recent architectures push toward more and smaller experts. In this regime, the standard linear router becomes a bottleneck: with $M$ experts and hidden dimension $h$, its p…
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-09-30 04:00 · arXiv stat.ML
MoRE: Scaling mixture of experts with hardware-aware low-rank routing