Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming
I have been working on implementing adaptive KV streaming and RDNA 4 / ROCm optimizations for llama.cpp to enable running Qwen3.8-27B (IQ4_XS weights, Q8_0/Q4_0 KV cache) with its full 262,144-token context window on a single 16 GB RX 9070 XT GPU at 80+ t/s decode. Qwen 27B IQ4_XS is ~14.3 GB, leav…
Read the full story at r/LocalLLaMA ↗
Timeline · 12 reports
- 2026-10-19 12:22 · r/LocalLLM
Qwen3.8-Flash-Next-FP4-Local: Strada, RTX Pro 4500 32 VRAM, 64 GB DDR5 - 2026-10-19 08:43 · r/LocalLLM
Qwen 35B does 154 tok/s on 2024 Macbook M4 Pro 48GB with Kapitsa: Open-source Apple Silicon optimized custom Metal engine built for Qwen. See benchmarks of Qwen 3.8 27B, Qwen3.6 35B, 3.8 Flash-Next on M1 Pro (16GB), M4 Pro (48GB) and M5 Max (128GB) - 2026-10-19 07:04 · r/LocalLLaMA
Kapitsa: open-source Apple Silicon optimized Metal engine built for Qwen and Nemotron. See benchmarks of Qwen 27B, Nemotron + 8 other models on an M1 Pro (16GB), M4 Pro (48GB) and M5 Max (128GB) - 2026-10-19 05:52 · r/LocalLLM
Jundot--Qwen3.8-Flash-Next-oQ4-mtp running on a Mac Studio M2 192gb with oMLX and OpenwebUI thinks it is Claude??? - 2026-10-18 23:41 · r/LocalLLM
Qwen3.8-Flash-Next 125B Q2_0 on a 24GB M4 Mac mini: ~8.1 tok/s with Strata - 2026-10-18 17:49 · r/LocalLLM
Qwen3.8-Flash-Next UD-IQ4_XS on 64 GB RAM + RX 9070 XT with Strata — around 40 tokens/s - 2026-10-18 05:14 · r/LocalLLM
32GB to 64GB of RAM took Qwen3.8-Flash-Next from IQ2_XS (2-bit) to Unsloth UD-IQ4_XS (4-bit) on a single 4090 - 2026-10-18 00:26 · r/LocalLLM
Basalt: NVFP4 + faster prefill for Qwen3.8 Flash-Next on Blackwell (single 5090) - 2026-10-18 00:26 · r/LocalLLaMA
Basalt: NVFP4 + faster prefill for Qwen3.8 Flash-Next on Blackwell (single 5090) - 2026-10-17 21:13 · r/LocalLLaMA
2 gaming PCs + an M4 Pro MacBook over Ethernet + 2 iPhone 17 Pro Maxes over USB-C: Qwen3.8 Flash Next at 37-39 tok/s at 60k context (262k layout in the works) - 2026-10-17 20:57 · r/LocalLLaMA
32GB to 64GB of RAM took Qwen3.8-Flash-Next from IQ2_XS (2-bit) to Unsloth UD-IQ4_XS (4-bit) on a single 4090 - 2026-10-17 13:01 · r/LocalLLaMA
Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming