PSA: if you're on an Intel hybrid CPU, run Strata's calibrate - it nearly tripled my decode speed (IQ3_S at 256K, 16 GB card)
This is the third version. In the third version, I added P-core thread pinning. setup: 5070 ti 16gb, 96gb ram, i7-14700kf, windows. qwen3.8-flash-next iq3_s on strata, 262k context. first test: ~17 tok/s at 256k. log screenshot attached, before lines are stock settings. then i changed 3 things: poo…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-04 19:11 · DEV Community — Machine Learning
PSA: if you're on an Intel hybrid CPU, run Strata's calibrate - it nearly tripled my decode speed (IQ3_S at 256K, 16 GB card)