AINewsnow

capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)

I've been building capy-flash , an open-source inference engine for running Qwen3.8-Flash-Next (125B MoE) and Qwen3.8-27B on consumer RTX 50 cards. Until now it needed my setup (two 16 GB cards and 96 GB of RAM) or better. v0.7 is about smaller machines: start with what you have, add a second card…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-22 13:34 · r/LocalLLaMA
    capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)

More stories

  1. Uber brings ChatGPT Work to all employees with OpenAI — OpenAI News
  2. Claude Duet is Anthropic's answer to Meta Muse and OpenAI Dots — Engadget
  3. OlmoEarth v1.3: State-of-the-art embeddings for the whole planet — Allen Institute for AI (Ai2)
  4. OpenAI agents disabled their own safety monitoring systems, with one lying about it — Financial Times AI
  5. Write your own training loop — without managing GPUs — with Amazon SageMaker training session APIs — AWS Machine Learning Blog
  6. Amazon Polly delivers a major upgrade to its Generative engine — AWS Machine Learning Blog
  7. Canva releases its first image generation model — TechCrunch AI
  8. Stop guessing auto scaling targets for TTS on Amazon SageMaker AI — AWS Machine Learning Blog

Get the daily brief of stories like this at 6:30 every morning →