AINewsnow

SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.

I’ve been building SonicBoom, a native tensor runtime written in C++/CUDA with a C ABI and no Python anywhere in the stack. I just tagged v0.0.1, and the first target is local LLM inference on consumer GPUs. Numbers (Gemma 4 E4B-it Q3_K_M on a laptop RTX 3050 8 GB, both runtimes using the same GGUF…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-13 09:46 · r/LocalLLaMA
    SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.

More stories

  1. Was My 2,103-Token Tokenizer a Mistake? — DEV Community — Machine Learning
  2. Qwen 27B at ~20 tok/s on a jailbroken PS5's GPU. I forked PS5LM and turned it into a native app — r/LocalLLM
  3. An 80B MoE (Qwen3-Next, 48 GB GGUF) answering in a Chrome tab: llama.cpp's WebGPU backend, experts paged in from disk — r/LocalLLaMA
  4. Models keeps unloading from GPU — r/LocalLLM
  5. train the best model you can on your mac in one hour — r/learnmachinelearning
  6. Running decision models: Picking quant, VRAM usage — r/LocalLLaMA
  7. Problems with Tesla V100 SXM2 — r/LocalLLaMA
  8. I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating) — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →