AINewsnow

Qwen3.6-35B-A3B on a laptop RTX 5050 8GB: ~1000 tok/s prefill and ~40-46 tok/s decode (config + steps)

My specs Alienware Aurora 16 (Intel Core 7 240H, 32 GB DDR5) RTX 5050 Laptop, 8 GB VRAM Windows 11 Pro (25H2), CUDA 13.0 Why I started this This is my first Reddit post. I hope it helps someone run larger MoE models on lower-tier laptop hardware. Default Ollama and llama.cpp were giving me roughly…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-11-01 04:15 · r/LocalLLM
    Qwen3.6-35B-A3B on a laptop RTX 5050 8GB: ~1000 tok/s prefill and ~40-46 tok/s decode (config + steps)

More stories

  1. LFM3-1.7b seems impressive — r/LocalLLaMA
  2. 769 tok/s on a Single RTX 4090: Spark-X2.6-1.7B DSpark — DEV Community — Machine Learning
  3. Halving the KV Cache Doubles Capacity and Buys 1% of Speed — DEV Community — Machine Learning
  4. Are bespoke inference engines overhyped? My Strata experience on 2× R9700 — r/LocalLLaMA
  5. How we fit a 27B model on a 12 GB laptop GPU — DEV Community — Machine Learning
  6. Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads — DEV Community — Machine Learning
  7. How to Run LFM3-Omni-2B GGUF Locally (Only llama.cpp Makes It Talk Back) — DEV Community — Machine Learning
  8. A game scripting language that runs speech-to-text 5x faster than llama.cpp on CPU — Lobsters AI

Get the daily brief of stories like this at 6:30 every morning →