DuplexTune: Coupled-State Replay for Memory-Efficient Mixture-of-Experts Adaptation
arXiv:2610.36009v1 Announce Type: new Abstract: Sparse expert execution does not remove the stored states needed to adapt a mixture-of-experts language model. We present DuplexTune, a framework for replaying coupled hidden states through pretrained attention and expert modules. Trainable width brid…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-28 04:00 · arXiv cs.LG
DuplexTune: Coupled-State Replay for Memory-Efficient Mixture-of-Experts Adaptation