AINewsnow

P100 (sm_60) patches for llama.cpp: +42% decode and +74% short-prefill on a 2×P100 tensor split

I spent a month making two Tesla P100s fast in llama.cpp — Results : 42% more decode, +74% more on short-prefill. GP100 is an awkward card for llama.cpp: alone among Pascal chips it has no DP4A , so the quantized matmul paths fall back to emulation and cuBLAS, but it does have full-rate HFMA2 — the…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-15 21:02 · r/LocalLLM
    P100 (sm_60) patches for llama.cpp: +42% decode and +74% short-prefill on a 2×P100 tensor split

More stories

  1. Problems with Tesla V100 SXM2 — r/LocalLLaMA
  2. Why long context is expensive: the KV cache bill — DEV Community — AI
  3. QUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releases — r/LocalLLaMA
  4. Qwen3.8FN vLLM vs. llama.cpp vs Ninfer6000 — r/LocalLLaMA
  5. I built Harness Studio — an open-source desktop app for running Hugging Face models locally. Looking for feedback and contributors. — r/huggingface
  6. I built Agent Praxis: 6 adversarial environments to test how LLM agents handle traps, metric gaming, and fake authority — r/LocalLLM
  7. Qwen 27B at ~20 tok/s on a jailbroken PS5's GPU. I forked PS5LM and turned it into a native app — r/LocalLLM
  8. Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules — arXiv cs.CL

Get the daily brief of stories like this at 6:30 every morning →