Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures
arXiv:2610.02344v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPAs) are prone to representation collapse, typically mitigated through empirical heuristics. We develop an early-training stability theory that unifies these heuristics. Linearising the coupled JEPA gradient…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-05 04:00 · arXiv cs.LG
Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures