Study finds first token acts as attention sink in language and vision-language models
A new arXiv paper examines attention sinks in large language models and vision-language models, finding that particular positions such as the beginning-of-sequence token attract disproportionate attention.
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-19 05:00 · arXiv cs.AI
BOS Summary: The First Token Matters