New 'FlagRot' Method Targets Efficient Low-Bit KV Caches for Long-Context Inference
A new arXiv paper, FlagRot, proposes selective rotation to keep 2-bit KV cache quantization errors in check, potentially shrinking long-context inference memory about sixfold.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-04 05:00 · arXiv cs.CL
FlagRot: Selective Rotation for Low-Bit KV Caches