What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation
This story is from 2026-09-05. It is preserved in the archive; the latest stories are on the live feed.
arXiv:2609.03515v1 Announce Type: new Abstract: Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode steps is often treated as an implementation detail. Under aggressive KV compression,…
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-09-05 04:00 · arXiv cs.AI
What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation