Study Examines SGD Dynamics in Two-Layer Neural Networks on Multi-Index Teacher Model
A new paper analyzes two-layer neural networks trained with SGD on a multi-index teacher model, focusing on how first-layer parameters evolve, including teacher-induced spikes and variance inflation.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-19 05:00 · arXiv stat.ML
Decomposing SGD Dynamics in Neural Networks: Teacher-Induced Spikes and Variance Inflation