AINewsnow

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows by anujj · Pull Request #29948 · ggml-org/llama.cpp

time to download some speedup DGX Spark (GB10), llama-bench -ngl 999 -fa on -p 16384 -n 0 -b 2048 -ub 2048 -ctk f16 -ctv f16 -r 5 , tok/s: model type master PR Qwen3.8-27B Q4_K_M 840.1 899.9 +7.1% Q5_K_M 840.1 880.6 +4.8% Q6_K 751.2 785.8 +4.6% Q8_0 876.2 940.1 +7.3% IQ4_XS 882.4 948.0 +7.4% Qwen3.…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-25 16:37 · r/LocalLLaMA
    qwen: merge tensors on convert by am17an · Pull Request #31028 · ggml-org/llama.cpp
  2. 2026-10-25 09:35 · r/LocalLLaMA
    CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows by anujj · Pull Request #29948 · ggml-org/llama.cpp

More stories

  1. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  2. I ran a blind test on self-hosted AI agents. Here are the results, guess who's who [D] — r/MachineLearning
  3. I ran local AI inference (AMD GPU) for 30 days and measured every watt — here's the real cost vs cloud APIs — r/LocalLLM
  4. Cohere says Alibaba’s Qwen model tried to send a file to Beijing — TechCrunch AI
  5. Temperature for Qwen 3.8 Flash Next for creative ideas — r/LocalLLM
  6. New 6-step adapters for Qwen-Image-2.1, no upsampling step, ~5x faster — r/huggingface
  7. I tightened the rules on a local Qwen 27B (IQ3 quant) and it stopped taking risks. free design or templates for slides? — r/LocalLLM
  8. Alibaba's latest Qwen models routed some requests through company servers, researchers find — The Decoder

Get the daily brief of stories like this at 6:30 every morning →