AINewsnow

CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp

This story is from 2026-10-28. It is preserved in the archive; the latest stories are on the live feed.

download your CUDA/MoE prompt-processing speedup today: GPU model master PR DGX Spark (GB10) Qwen3-30B-A3B Q4_K_M 2911.4 3410.8 +17.2% Qwen3.6-35B-A3B Q4_K_M 3235.8 3556.7 +9.9% Qwen3.6-35B-A3B UD-Q4_K_M 3264.7 3561.8 +9.1% RTX 5090 Qwen3-30B-A3B Q4_K_M 12611.7 13330.3 +5.7% Qwen3.6-35B-A3B Q4_K_M…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-28 13:40 · r/LocalLLaMA
    CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp

More stories

  1. A 2.6B model on an iPhone that always answers in valid JSON — DEV Community — Machine Learning
  2. Spark-X2.6-4B DSpark Is Here: More Than 2x Average BF16 Inference Speed — DEV Community — Machine Learning
  3. Which coding agent gets the most out of a local Qwen 3.8 27B? I lined up 10 and logged tokens, cache and sandbox blocks — r/AI_Agents
  4. I dpo post-trained llama-3.1-8b to be 100% more robust against jailbreaks (and measured what it costs) — r/huggingface
  5. Run TeleAI's Xing4.0-29B-A4B GGUF via ik_llama.cpp — Full native mHC/MLA runtime + HF converter — r/LocalLLM
  6. PSA: Upgrade Flash Next to DFlash2 if you haven't! +17% on average — r/LocalLLaMA
  7. Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0) — r/LocalLLM
  8. I spent a year making old Tesla P100/V100 cards fast for local LLMs. PXA 4.0 is out today: up to 3x the decode and 8x the prompt speed of mainline llama.cpp on the same box, same files. Numbers, method and the losses inside. — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →