AINewsnow

I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest

I wanted to know which low-bit (1-bit / ternary / 2-bit) GGUF files actually give correct output on CPU, and how fast they run, so I compared each runtime against a reference instead of judging by how the text looks. Machines: an ARM server (Neoverse-N1) and x86 VMs with AVX2 (Intel Xeon, AMD EPYC)…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-19 14:19 · r/LocalLLM
    I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest

More stories

  1. MiniMax launches M3.1 Flash, confirming Space Bunny was an early build — Vercel Blog
  2. DeepSeek V4.1 Flash 1017 now available on AI Gateway — Vercel Blog
  3. Google Gemini 4 agents just beat Minecraft with no human intervention — Engadget
  4. Anthropic commits 5% of shares to public benefit fund ahead of planned IPO — TechCrunch AI
  5. Introducing Clairic, a new System One class of extraction models — Cloudflare Blog — AI
  6. People are using AI to make their smart TVs dumb again — TechCrunch AI
  7. Germany Wants to Lead the World in Industrial AI: How NVIDIA Will Help — NVIDIA Blog
  8. Cutting our inference bill by 68% by being obsessive about optimization — Cloudflare Blog — AI

Get the daily brief of stories like this at 6:30 every morning →