AINewsnow

Gemma 4 26B-A4B on a 16 GB Mac: 33/40 on TypeScript tasks, 31 tok/s, offline, using Google's stock file

Last time we cut Qwen3.6-35B down to a TypeScript Edition. This time nothing is cut. Kraits, a plugin for stock llama.cpp, loads Google's own 14.4 GB Gemma 4 26B-A4B file (QAT Q4_0) on a 16 GB Mac mini M4. A heat map of the experts each layer uses on TypeScript puts the 80 hottest of 128 per layer…

Read the full story at r/huggingface ↗

Timeline · 1 report

  1. 2026-10-26 06:33 · r/huggingface
    Gemma 4 26B-A4B on a 16 GB Mac: 33/40 on TypeScript tasks, 31 tok/s, offline, using Google's stock file

More stories

  1. Qwen 3.8 Flash Next Gyro Coder is out! — r/LocalLLaMA
  2. Full DeepSeek-V4.1-Flash-1017 (510 GB, original weights) on a 64 GB RAM + RTX 3060 12 GB PC: up to 7.4 tok/s, 98.3% agreement with the full model. Engine released (Apache-2.0) — r/LocalLLM
  3. Goldman Sachs puts $500M into small-business AI with four AI labs — The Next Web
  4. Gartner's AI predictions address the control gap — The Deep View
  5. boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face — r/LocalLLaMA
  6. I built my first RAG project using Whisper, BGE-M3 and Ollama — what should I improve? — r/learnmachinelearning
  7. DSpark on an RX 7900 GRE: Qwen3-8B goes from 59 to 133 tok/s in llama.cpp — r/LocalLLM
  8. qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →