Massive B70 spec-decoding improvement - Intel XMX matrix engines refactored by budiony · Pull Request #29864 · ggml-org/llama.cpp
This made a huge improvement to speculate decoding speeds on my B70 running SYCL. Below examples are for Muse Glimmer Q6_K (30B) running with DFlash2. NOTE: Due to the significantly reduced cost of token verification, the n-max was increased which is why the acceptance rate is lower. Before: ``` co…
Read the full story at r/LocalLLaMA ↗
Timeline · 4 reports
- 2026-10-25 09:35 · r/LocalLLaMA
CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows by anujj · Pull Request #29948 · ggml-org/llama.cpp - 2026-10-25 02:16 · r/LocalLLaMA
model: DSV4.1 flash by am17an · Pull Request #31086 · ggml-org/llama.cpp - 2026-10-23 18:43 · r/LocalLLaMA
qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp - 2026-10-22 18:16 · r/LocalLLaMA
Massive B70 spec-decoding improvement - Intel XMX matrix engines refactored by budiony · Pull Request #29864 · ggml-org/llama.cpp