AINewsnow

llama.cpp has come a long way

Hey everyone, English is not my native language (Dutch), but I write in my own words without assistance. So please let me know if any sentence flows weird, etc. A mate of mine told me how amazing local AI was in January 2025 or so but I was a skeptic who took pride in writing my own code for 10 yea…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-17 01:08 · r/LocalLLaMA
    llama.cpp has come a long way

More stories

  1. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  2. I made llama.cpp's slot saves write only the changed chunks: a FUSE filesystem that dedups KV-cache snapshots — r/LocalLLM
  3. Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console — Hacker News Front Page
  4. Fully local, private voice + chat AI assistant (zero cloud) — Qwen3.8-27B on 2× RTX PRO — r/LocalLLM
  5. reflection/Beam-1.0-501B-A23B · Hugging Face — r/LocalLLaMA
  6. qwen3.8-flash-next-iq3_s vs -27B-GGUF:UD-Q4_K_XL — r/LocalLLaMA
  7. QUASAR 4-bit QAT checkpoints for Qwen3.5, Gemma 4 and Muse Glimmer: closer to BF16 than Google’s and NVIDIA’s 4-bit releases — r/LocalLLaMA
  8. Gemini leaked its thinking tokens by just asking what tag it uses to end thinking — r/ArtificialInteligence

Get the daily brief of stories like this at 6:30 every morning →