AINewsnow

High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL!

I've been working on improving llama.cpp support and performance on older NVIDIA GPUs, particularly Kepler and other pre-Ampere architectures. The goal is simple: get more life out of older CUDA hardware instead of treating it as e-waste. Achieved Performance On MTP Qwen3.6-35B-A3B with Q8_0 quanti…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-24 20:01 · r/LocalLLaMA
    High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL!

More stories

  1. DeepSeek V4 Flash API prices compared across providers (Oct 2026) — r/huggingface
  2. NVIDIA's Dory 1.0 4B (looped reasoning model) on Apple Silicon with MLX: 101 tok/s at 1 loop, 69 at 2, on an M5 Max (4.2 GB 8-bit build) — r/LocalLLM
  3. Qwen 3.8 Flash Next GSQ-RCO IQ_3S at 70tok/s decode/1900tok/s prefill on 2013 Xeon E5-1680v2, 128GB quad channel DDR3-1866 ECC, and an EVGA 3090 FTW3 — r/LocalLLM
  4. AbliteratedOutTheBox Strata Announcement — r/LocalLLaMA
  5. It is not clear to me which parameter we should use for LoRA training—shift or sigmoid—in the new models (Qwen 2.1, LLaMA 2, and MOSS) within the AI ​​toolkit. — r/comfyui
  6. 2× Arc Pro B70 on Windows: Qwen3.8-27B Q8_0 at 51 t/s average over 1,030 real agent requests (15K–193K context, temp 1.0). Recipe, patches, and a oneAPI leak fix — r/LocalLLM
  7. webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp — r/LocalLLaMA
  8. Benchmark: Recursive's Flint 40B A2.5B Q4 achieves 128 tok/s on MacBook Pro M4 Pro 48GB using Kapitsa, an open-source Apple Silicon optimized engine, Claude/Codex/Pi/OpenCode compatible. — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →