I solved compaction for single-session on consumer-hardware, trust me bro
Not everyone here uses pi, but the concept of re-using kv cache for compaction-summary might help some of you anyway. Took me the better part of a month to reach the state where I can run 40+ compactions without losing model-and-session-coherence.
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-20 23:33 · r/LocalLLM
I solved compaction for single-session on consumer-hardware, trust me bro