How Vision Transformers Represent Position: A Mechanistic Comparison of Absolute and Rotary Positional Encodings
arXiv:2610.13992v1 Announce Type: new Abstract: Positional embeddings encode the spatial position of each image patch in a Vision Transformer (ViT), allowing the model to distinguish different spatial arrangements of visual content. However, we know little about how the choice of embedding shapes a…
Read the full story at arXiv cs.CV ↗
Timeline · 1 report
- 2026-10-14 04:00 · arXiv cs.CV
How Vision Transformers Represent Position: A Mechanistic Comparison of Absolute and Rotary Positional Encodings