Strata looping badly with iq2_xxs
This story is from 2026-10-04. It is preserved in the archive; the latest stories are on the live feed.
Hello, I'm running strata with iq2_xxs quant and indeed I am quite amazed to be able to run it at speeds comparable to vanilla llama.cpp + 3.8 27B. I'm getting about 30 tps on both. Strata seems to offer higher peak tps when working with code, up to 50. Although I do feel that the bigger model offe…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-04 15:30 · r/LocalLLM
Strata looping badly with iq2_xxs