Qwen3.8-Flash-Next and GLM-5.3-Flash achieve high speeds on consumer hardware
This story is from 2026-10-10. It is preserved in the archive; the latest stories are on the live feed.
Qwen3.8-Flash-Next and GLM-5.3-Flash demonstrate high inference speeds and long-context capabilities on M5 Ultra Macs and RTX GPUs using llama.cpp and vLLM.
Read the full story at AWS Machine Learning Blog ↗
Timeline · 8 reports
- 2026-10-12 15:51 · AWS Machine Learning Blog
Deploy decision models with llama.cpp on Amazon SageMaker AI - 2026-10-12 08:37 · r/LocalLLaMA
Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included. - 2026-10-12 05:12 · r/LocalLLM
GLM-5.3-Flash on a Mac Studio with oMLX: 61 tok/s and a 29K-token prompt in 15 seconds - 2026-10-12 00:58 · r/LocalLLM
IQ3_XXS Qwen3.8-Flash-Next 125B at full 262K context on one 3090 and 32GB of RAM — my config and my numbers, and I'd like to see yours - 2026-10-11 11:12 · r/LocalLLM
RTX 5090 32GB + 64GB RAM — Qwen3.8 Flash-Next IQ3_XXS Running at 700K Context Without a Single Session Compaction - 2026-10-11 02:20 · r/LocalLLM
Qwen3.8-Flash-Next on an RTX 5090: 128GB RAM: IQ3_XXS vs IQ3_S vs UD-IQ4_XS — Speed and Accuracy with Strata - 2026-10-10 18:07 · r/LocalLLM
GLM-5.3-Flash (320B) on an M5 Ultra Mac Studio with llama.cpp: 61 tok/s, 262K context - 2026-10-10 03:21 · r/LocalLLM
Qwen3.8-Flash-Next (125B) at ~100 tok/s on an M5 Ultra Mac Studio with llama.cpp