SympFormer: Inertial Attention Blocks from Damped Hamiltonian Dynamics on Spaces of Probability Measures
arXiv:2610.31600v1 Announce Type: cross Abstract: The attention mechanism underpins modern large language models, yet their dynamics remain only partially understood. Recent perspectives interpret attention blocks as interacting particle systems, whose mean-field limits correspond to gradient flows…
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-10-23 04:00 · arXiv stat.ML
SympFormer: Inertial Attention Blocks from Damped Hamiltonian Dynamics on Spaces of Probability Measures