How is it possible that OKLS can't outperform Muon in terms of step efficiency? [D]
SOAP is more step efficeint than Shampoo, and KL-Shampoo is more step efficient than SOAP. Then how is OKLS getting outperformed by Muon in terms of step efficiency where it's a zero-staleness implementation of KL-Shampoo? It only gets ahead of Muon in terms of parameter efficiency in very long tra…
Read the full story at r/MachineLearning ↗
Timeline · 1 report
- 2026-11-17 09:30 · r/MachineLearning
How is it possible that OKLS can't outperform Muon in terms of step efficiency? [D]