AINewsnow

I removed 85% of gemma's weights (and it is mostly fine)

I pruned Gemma 4 26B-A4B, an MoE model, down to 15% of its original size, and the model still does great on one narrow task. It gets 92% agreement with the original on a classification task. That’s close to how often the 12B agrees with the 26B on the same ticks, which is 90%. This isn’t a model fo…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-11-04 13:53 · r/LocalLLaMA
    I removed 85% of gemma's weights (and it is mostly fine)

More stories

  1. Claude Code and Codex ecosystem expands with relay setups, phone control, and provenance tools — Runway Research
  2. Google to cut off Search for schools that reject AI, drawing backlash — Forbes AI
  3. LLMs shown to read and edit neural network weights directly — Techmeme
  4. Tencent launches Hy4 open-weight AI model with vision and computer-use capabilities — SCMP China Tech
  5. Apple opens Siri AI to all developers with iOS 27.3 beta and new language support — Engadget
  6. OpenAI to acquire developer tools startup Pierre Computer — OpenAI News
  7. Democratic Midterm Wins Cloud Outlook for US AI Data Center Expansion — DigiTimes Asia
  8. UK cyber agency warns AI-powered malware likely to hit critical infrastructure by 2028 — The Guardian AI

Get the daily brief of stories like this at 6:30 every morning →