AINewsnow

qwen3.8-flash-next-iq3_s vs -27B-GGUF:UD-Q4_K_XL

Sampling: temp 1.0, top_p 0.95, top_k 20 for both. Run with evalscope over the OpenAI-compatible API: Qwen3.8-Next on Strata (MI50 32GB), Qwen3.8-27B on llama.cpp (RX 7900 XTX 24GB). Caveat: These are only a portion of the full sets, since they take days. Edit1: Strata started with --kv fp16 and ll…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-16 07:51 · r/LocalLLM
    moe-cache: MoE models bigger than your RAM, 2-12x faster on stock llama.cpp with identical tokens (Qwen3.8-Flash-Next 68 GB @ 24 GB RAM: 5.2 → 12.5 tok/s, Qwen3.6-35B @ 12 GB RAM: 6.9 → 22.6 tok/s)
  2. 2026-10-14 16:13 · r/LocalLLaMA
    qwen3.8-flash-next-iq3_s vs -27B-GGUF:UD-Q4_K_XL

More stories

  1. llama.cpp has come a long way — r/LocalLLaMA
  2. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  3. I made llama.cpp's slot saves write only the changed chunks: a FUSE filesystem that dedups KV-cache snapshots — r/LocalLLM
  4. Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console — Hacker News Front Page
  5. Fully local, private voice + chat AI assistant (zero cloud) — Qwen3.8-27B on 2× RTX PRO — r/LocalLLM
  6. reflection/Beam-1.0-501B-A23B · Hugging Face — r/LocalLLaMA
  7. 😺 Advertisers can target ChatGPT by prompt — The Neuron
  8. QUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releases — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →