AINewsnow

I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest

I wanted to know which low-bit (1-bit / ternary / 2-bit) GGUF files actually give correct output on CPU, and how fast they run, so I compared each runtime against a reference instead of judging by how the text looks. Machines: an ARM server (Neoverse-N1) and x86 VMs with AVX2 (Intel Xeon, AMD EPYC)…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-19 17:57 · r/LocalLLaMA
    I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest

More stories

  1. OlmoEarth v1.3: State-of-the-art embeddings for the whole planet — Allen Institute for AI (Ai2)
  2. Anthropic takes on Muse with Claude Duet, its new AI agent — The Verge AI
  3. Barclays to cut 5,000 jobs as bank steps up AI adoption — Financial Times AI
  4. NVIDIA Unveils Minerva to Bring End-to-End Observability for AI Factories — NVIDIA Blog
  5. Enact, which develops full-stack tech for general-purpose robots, emerges from stealth with a $30M pre-seed from Mistral CEO Arthur Mensch, Yann LeCun, and more — Techmeme
  6. OpenAI agents disabled their own safety monitoring systems, with one lying about it — Financial Times AI
  7. Beyond tree cover: mapping the world’s forest types — Google DeepMind Blog
  8. AI Inbox Management: This Is Now a Thing — Salesforce AI Blog

Get the daily brief of stories like this at 6:30 every morning →