8 GB GPU, 262K context 3.5 BPW Qwen-Flash-Next Coder 70 T/s using llama.cpp
The methodology Qwen-Flash-Next has n-gram, MOE packing above its weight but it's still 6B active params. We exploit this by keeping most MOEs in RAM (Smallest MOEs go in VRAM for speedup) and n-gram on disk - ensure you put the model on your fastest SSD. 8 GB VRAM, 24 GB RAM cost. You can run this…
Read the full story at r/LocalLLaMA ↗
Timeline · 5 reports
- 2026-10-26 10:26 · r/LocalLLaMA
Have 8 GB of VRAM and 32 GB of RAM? You can run Qwen 3.8 Flash Next at up to 900 t/s PP and 30 t/s TG! - 2026-10-26 08:47 · r/LocalLLaMA
Qwen3.8-Flash-Next: Swift 1.5 vs Unsloth on Strix Halo (Halogen, gufo, llama.cpp, UD-Q4_K_XL and IQ4_XS) - 2026-10-25 09:51 · r/LocalLLM
Temperature for Qwen 3.8 Flash Next for creative ideas - 2026-10-24 23:53 · r/LocalLLM
Qwen 3.8 Flash Next on 9060 XT 16GB - 2026-10-24 15:21 · r/LocalLLaMA
8 GB GPU, 262K context 3.5 BPW Qwen-Flash-Next Coder 70 T/s using llama.cpp
More stories
- qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp — r/LocalLLaMA
- I ran a blind test on local AI agents. Here are the results, guess who's who — r/AI_Agents
- I ran local AI inference (AMD GPU) for 30 days and measured every watt — here's the real cost vs cloud APIs — r/LocalLLM
- What's the best front-end for LLM assisted studying? — r/LocalLLM
- It is not clear to me which parameter we should use for LoRA training—shift or sigmoid—in the new models (Qwen 2.1, LLaMA 2, and MOSS) within the AI toolkit. — r/comfyui
- Full DeepSeek-V4.1-Flash-1017 (510 GB, original weights) on a 64 GB RAM + RTX 3060 12 GB PC: up to 7.4 tok/s, 98.3% agreement with the full model. Engine released (Apache-2.0) — r/LocalLLM
- Automated AI 2K documentaries in ComfyUI: Qwen Image 2.1 → MiniMax H3 → Qwen3-TTS → RTX upscale, all in batch (free workflow) — r/comfyui
- Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM
Get the daily brief of stories like this at 6:30 every morning →