PeekKV: The First Content Token Already Knows Which Heads Matter
arXiv:2610.23939v1 Announce Type: new Abstract: The size of the KV cache grows linearly with context length, making it a major memory bottleneck in LLM serving. Motivated by recent progress in identifying the functional roles of attention heads, a growing body of work compresses the KV cache non-un…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-20 04:00 · arXiv cs.LG
PeekKV: The First Content Token Already Knows Which Heads Matter