AINewsnow

Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant

This story is from 2026-10-01. It is preserved in the archive; the latest stories are on the live feed.

I've been working on getting the 95.5 GiB Qwen3.8-Flash-Next model to run fast on a single 64GB Mac. In my earlier post , I shared a custom expert-streaming fork of llama.cpp . It worked, but decode capped out around ~23–27 tok/s and slowed down as context grew. Today I'm releasing Slipstream : a c…

Read the full story at r/LocalLLaMA ↗

Timeline · 5 reports

  1. 2026-10-03 20:15 · r/LocalLLM
    Running a 176B Qwen3.8 Flash Next on a 16GB RTX 3080 Laptop + 32GB RAM + SSD
  2. 2026-10-03 20:06 · r/LocalLLaMA
    Running Qwen3.8 Flash Next 176B on a 16GB RTX 3080 Laptop + 32GB RAM + SSD
  3. 2026-10-03 08:33 · r/ArtificialInteligence
    Qwen3.8-Flash-Next 177B at 11–15 tok/s on a single RTX 5070 12GB with 32GB DDR4 RAM
  4. 2026-10-03 06:49 · r/LocalLLaMA
    Qwen3.8-Flash-Next 177B running at 11–15 tok/s on a single RTX 5070 12GB + 32GB RAM DDR4
  5. 2026-10-01 20:21 · r/LocalLLaMA
    Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant

More stories

  1. How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP) — DEV Community — Machine Learning
  2. Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp — r/LocalLLaMA
  3. Thinking On or Off? Gemma 4 E2B on an 8 GB RTX 3050 — DEV Community — AI
  4. Running 26B Coding Models Locally on a 6GB Laptop GPU at 45 Tokens/Sec with MAPLE and llama.cpp — DEV Community — Machine Learning
  5. Cutting LLM Fine-Tuning Memory in Pure Kotlin: Fused Linear Cross-Entropy and the JVM Vector API — DEV Community — Machine Learning
  6. RAMDeck Architecture Analysis: llama.cpp RPC, Distributed LLM Memory, and SBC Compatibility — DEV Community — Machine Learning
  7. llama.cpp finally has a 1.0 release. Check your old integrations before upgrading. — r/artificial
  8. Signal #136: Qwen4-Max, llama.cpp v1.0.0, and the Git Patch You Need Today — DEV Community — AI

Read the Daily Brief: the day's most important AI stories →