Study Finds Attention Competition, Not Length, Drives Language Model Retrieval Failures
A new arXiv paper argues that long-context retrieval failures in language models stem from attention competition rather than input length, based on experiments across five open models.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.CL
Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval