Correlations Are Ruining Your Gradient Descent
arXiv:2407.10780v5 Announce Type: replace Abstract: Natural gradient descent, and the preconditioned optimizers which have followed from it, correct gradient descent by estimating curvature and applying its inverse to the gradient. For any single layer of a deep neural network, this correction can…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.LG
Correlations Are Ruining Your Gradient Descent