What quant do you actually use day to day? Anyone happy with 2-bit or 3-bit?
Curious what everyone runs in practice: - What quant do you use most, and for what (chat, coding, agents)? - 2-bit: do you use it? Is it actually usable or too broken? - 3-bit: good enough for coding and more complex stuff, or do you go back to 4-bit? - If you look at KLD: what number do you call "…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-10-18 13:05 · r/LocalLLaMA
What quant do you actually use day to day? Anyone happy with 2-bit or 3-bit?