New 'Mixture of Wide Experts' Method Targets Interpretable Features in Language Models
A paper proposes Mixture of Wide Experts, a sparse routing approach that keeps model features in native coordinates to make language models more interpretable, complementing sparse autoencoders.
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-11-05 05:00 · arXiv cs.LG
Mixture of Wide Experts: Sparse Routing for Interpretable Native Coordinates