NoteBank: How Replaying a Few Saved Snippets Restores Long-Context Recall in Linear-Attention Models
Linear-attention and state-space models are cheap to run on long inputs because they carry a fixed-size state instead of a growing KV cache. The catch is that the state forgets. A new KAUST paper, "History Replays Itself" ( arXiv:2610.15715 ), proposes NoteBank, a training-free add-on that stores a…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-15 19:33 · DEV Community — Machine Learning
NoteBank: How Replaying a Few Saved Snippets Restores Long-Context Recall in Linear-Attention Models