New 'Mixture of Wide Experts' Method Aims for Interpretable Sparse Routing in Language Models
A new arXiv paper proposes Mixture of Wide Experts, a sparse routing approach designed to keep language model features in interpretable native coordinates, addressing how concepts spread across many internal dimensions.
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.LG
Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates