AINewsnow

webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp

Tested on an RTX 3060 with native Dawn WebGPU . llama-bench -m MODEL -p 512 -n 128 -r 5 -ngl 99 -dev WebGPU -t 2 -fa off Model Test master (t/s) PR (t/s) Ternary Bonsai 1.7B Q2_0 pp512 35.03 ± 0.46 1915.91 ± 3.36 Ternary Bonsai 1.7B Q2_0 tg128 14.02 ± 0.16 94.75 ± 0.20 Bonsai 2 2B Q2_0 pp512 36.40…

Read the full story at r/LocalLLaMA ↗

Timeline · 3 reports

  1. 2026-10-23 18:43 · r/LocalLLaMA
    qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp
  2. 2026-10-22 18:16 · r/LocalLLaMA
    Massive B70 spec-decoding improvement - Intel XMX matrix engines refactored by budiony · Pull Request #29864 · ggml-org/llama.cpp
  3. 2026-10-22 09:17 · r/LocalLLaMA
    webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp

More stories

  1. Qwen 3.8 Flash Next GSQ-RCO IQ_3S at 70tok/s decode/1900tok/s prefill on 2013 Xeon E5-1680v2, 128GB quad channel DDR3-1866 ECC, and an EVGA 3090 FTW3 — r/LocalLLM
  2. It is not clear to me which parameter we should use for LoRA training—shift or sigmoid—in the new models (Qwen 2.1, LLaMA 2, and MOSS) within the AI ​​toolkit. — r/comfyui
  3. Benchmark: Recursive's Flint 40B A2.5B Q4 achieves 128 tok/s on MacBook Pro M4 Pro 48GB using Kapitsa, an open-source Apple Silicon optimized engine, Claude/Codex/Pi/OpenCode compatible. — r/LocalLLaMA
  4. DeepSeek V4 Flash API prices compared across providers (Oct 2026) — r/huggingface
  5. High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL! — r/LocalLLaMA
  6. AbliteratedOutTheBox Strata Announcement — r/LocalLLaMA
  7. Are there any good finetunes for Blender and/or 3D modeling? — r/LocalLLaMA
  8. Comfy-Org's Qwen-Image-Edit Bundle Hits 1.68M Downloads With Smarter Editing — AlphaSignal

Get the daily brief of stories like this at 6:30 every morning →