Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact)
(Don't judge by the screenshot, the cache is cold. It hits 24+ tok/s with a warm cache!) About two months ago, I made a post here asking whether predicting which MoE experts would be used on the next token could actually help speed up CPU/GPU offloading. Original post: Tried predicting which MoE ex…
Read the full story at r/LocalLLaMA ↗
Timeline · 5 reports
- 2026-10-11 21:44 · r/LocalLLaMA
I ran Qwen 3.8 Flash-Next on my AMD 7900 XTX at 500k context. All local and what a shift it has been. - 2026-10-11 08:23 · r/LocalLLaMA
What setup do you use, which harness? and how do you customize it to run qwen 3.8 flash next swift with a high context length and a reasonable amount of tok/sec? - 2026-10-10 19:21 · r/LocalLLaMA
Benefits of using bigger models than Qwen 3.8 flash next? - 2026-10-10 00:14 · r/LocalLLaMA
Qwen Flash Next @ 137 tok/s & 3,497 tok/s Prefill w/ 512k context on a 5090, 192gb ram, Windows Build, comparing Strata and Infernix - 2026-10-09 18:41 · r/LocalLLaMA
Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact)