Study Derives Scaling Laws for Stochastic Rounding in Low-Precision SGD Training
A new theoretical paper examines how stochastic rounding affects training with low-precision SGD, deriving scaling laws for where, when, and how many bits to use, since reduced precision cuts compute but adds rounding error.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-02 05:00 · arXiv stat.ML
Where, When, and How Many Bits? Scaling Laws for SGD with Stochastic Rounding