AINewsnow

Betelgeuse running Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 (14 tok/s on a single 16GB stick) + RTX 3060 12GB — 0 dropped experts, 100% byte-exact

In my last thread on Betelgeuse ( bel-cli / bel-server ), the main pushback in the comments was: "Okay cool, 22 tok/s on IQ2_XS , but 2-bit is too low—what happens on an actual 4-bit quant like IQ4_XS where the experts are almost 2x larger?" "Why were you testing on a single 16GB stick of DDR4 inst…

Read the full story at r/LocalLLaMA ↗

Timeline · 3 reports

  1. 2026-11-03 08:12 · r/StableDiffusion
    H3 Long-Gen with Qwen 3.8 Flash Next prompting every 18sec went in a really wild direction (2min14s)
  2. 2026-11-01 23:56 · r/LocalLLaMA
    Qwen 3.8 Flash Next NVFP4 on 64gb RAM and 5090, Basalt. Ditched windows for it. Very hyped for qwen 4 model family now.
  3. 2026-10-31 18:06 · r/LocalLLaMA
    Betelgeuse running Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 (14 tok/s on a single 16GB stick) + RTX 3060 12GB — 0 dropped experts, 100% byte-exact

More stories

  1. Apple Intelligence to launch in China in December, powered by Alibaba's Qwen — Computerworld AI
  2. SillyTavern LLM settings for an OpenAI-compatible API — DEV Community — AI
  3. Astra can read and write weights to improve Qwen — Hacker News Front Page
  4. LLM Hallucinations. Who is to blame and what to do? — DEV Community — Machine Learning
  5. I Ran 6 n8n AI Workflows on a 1.8B Local Model: 192 Runs, 0 Broken JSON, 7 Fake Hot Leads — DEV Community — AI
  6. Alibaba's Qwen Code repo just leaked qwen4-max, qwen4-coder-plus, and qwen4-omni-120b — r/LocalLLaMA
  7. Choosing the Right Base Model for Fine-Tuning LLMs: Strategy Before Execution — DEV Community — Machine Learning
  8. I made ChatGPT, Claude, Gemini, etc. into FREE text-to-speech sites. — r/artificial

Get the daily brief of stories like this at 6:30 every morning →