Study Maps When SignSGD Beats SGD Through Gradient Noise and Scaling
A new paper analyzes how the sign operator interacts with gradient noise, deriving conditions under which sign-based optimizers like SignSGD outperform SGD in power-law scaling regimes.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-13 05:00 · arXiv stat.ML
When and How SignSGD Outperforms SGD: A Functional Scaling Law Perspective