AINewsnow

Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode

Hi folks! We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time — one shared library, a C ABI, and Guile Scheme as the first-class language. What v0.…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-19 03:44 · r/LocalLLM
    Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode
  2. 2026-10-19 03:40 · r/LocalLLaMA
    Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode

More stories

  1. DeepSeek V4.1 Flash 1017 now available on AI Gateway — Vercel Blog
  2. Seamless Model Discovery and Download from HuggingFace Hub in llama.cpp UI — DEV Community — Machine Learning
  3. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  4. Liquid AI's LFM2.5-24B Runs 184 Tokens per Second for Local Agents — AlphaSignal
  5. Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB — AlphaSignal
  6. Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless — r/LocalLLM
  7. Efforts to upstream adaptive MTP depth to llama.cpp abandoned — r/LocalLLM
  8. New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →