ggml-cpu: tiled mul_mat for k-quants by jbooth · Pull Request #27851 · ggml-org/llama.cpp

faster CPU prompt processing: "TL;DR: 3-7x faster CPU mul_mat using VNNI with IMO minimal complexity"

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

2026-09-26 06:18 · r/LocalLLaMA
ggml-cpu: tiled mul_mat for k-quants by jbooth · Pull Request #27851 · ggml-org/llama.cpp

More stories

Liquid AI Releases LFM2.5-VL-3B-DSpark: Speculative Decoding for Vision-Language Models With Up to 3.13x Faster Decoding — MarkTechPost
Ternary Bonsai 2 27B at up to 532 tok/s on one RTX 4090, native Windows: MTP + n-gram speculative decoding in a from-scratch CUDA engine — r/LocalLLM
Qwen3.8-Flash-Next on 12GB VRAM - 65 tokens per second — r/LocalLLaMA
Qwen3.8-Flash-Next on 12GB VRAM - 65 tokens per second — r/LocalLLM
JiRackUltra_1b Runs AI Routing on Any Laptop Without a GPU — AlphaSignal
Advice on models for RAG use case — r/LocalLLaMA
Targeting Applied AI / ML Engineer roles. Need ruthless feedback on my architecture and metrics. — r/deeplearning
42x Faster Prompt Lookup Drafting in llama.cpp — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →