FlagRot: Selective Rotation for Low-Bit KV Caches
arXiv:2611.03992v1 Announce Type: new Abstract: Long-context inference is bound by the KV cache, which 2-bit codes could shrink about sixfold if their error stays in check. The usual remedy rotates keys before quantizing, assuming rotation always helps. However, we observe that it does not: rotatio…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.CL
FlagRot: Selective Rotation for Low-Bit KV Caches