AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens
arXiv:2610.20641v1 Announce Type: new Abstract: Long chain-of-thought decoding is memory-bound: the KV cache of the generated tokens grows linearly with the test-time compute being spent. Evicting cache entries does not fix this, because a model never trained on a cache with entries removed generat…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-16 04:00 · arXiv cs.CL
AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens