MUON-RT Reuses Teacher Trajectories for Momentum-Free Muon in LLM Self-Distillation
A new method, MUON-RT, adapts the Muon optimizer for momentum-free LLM self-distillation by reusing the update history retained by EMA teachers in self-distillation pipelines.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-02 05:00 · arXiv stat.ML
MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation