The Best Optimizer Depends on Batch Size
arXiv:2610.08975v1 Announce Type: new Abstract: A plethora of new adaptive optimizers are designed to efficiently estimate and use minibatch gradient statistics to shape parameter updates, but they are typically benchmarked at a single batch size. Hyperparameter scaling rules promise to preserve pe…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-08 04:00 · arXiv cs.LG
The Best Optimizer Depends on Batch Size