AINewsnow

Is anyone working on improving prefill speed? (Qwen 27b, Flash)

I see everyone improving decode with bespoke models, like Sushi or Splash with Qwen 27b and Qwen Flash Next, and I'm running QFN now at 40tk/s on my M3 Max which is awesome. But prefill feels stagnant. I was trying to work on layer / attention propagation / approximating KVcache with QFN and could…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-15 09:38 · r/LocalLLaMA
    Is anyone working on improving prefill speed? (Qwen 27b, Flash)

More stories

  1. Qwen 3.8 Flash Next gains traction with optimized local inference engines — r/LocalLLaMA
  2. Alibaba launches Qwen N1 AI glasses with 50MP cameras and swappable batteries — Mint AI
  3. MBZUAI Ships Jais 3, Six Open Arabic Models Beating Qwen and Claude — AlphaSignal
  4. Apple reportedly plans December China launch for Apple Intelligence using Alibaba models — TechNode
  5. Diffusion without CFG? Correction Training for Diffusion Models — r/StableDiffusion
  6. Has anyone actually managed to get image editing to work when using a gguf quant Qwen 2.1? — r/comfyui
  7. I built Agent Praxis: 6 adversarial environments to test how LLM agents handle traps, metric gaming, and fake authority — r/LocalLLM
  8. Qwen 27B at ~20 tok/s on a jailbroken PS5's GPU. I forked PS5LM and turned it into a native app — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →