Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates
arXiv:2611.03655v1 Announce Type: new Abstract: Language models often spread a concept across many internal dimensions, making their features difficult to interpret. Sparse autoencoders (SAEs) recover clearer features after training. We ask whether models can instead learn interpretable output coor…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.LG
Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates