Hard numbers on Qwen3.8-27B's KV cache: q8_0 is free, K is ~2x as touchy as V, and a 2k recent-token window makes 6/5-bit basically lossless

Pretty much everyone running Qwen3.8-27B on a 24 GB card is squeezing the KV cache somehow, so here's what it actually costs, measured. This came out of building my own engine for this model ( repo ), but none of it is specific to my engine, it applies to whatever you run the model on. How I measur…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

1. 2026-10-25 15:05 · r/LocalLLM
   Hard numbers on Qwen3.8-27B's KV cache: q8_0 is free, K is ~2x as touchy as V, and a 2k recent-token window makes 6/5-bit basically lossless