AINewsnow

Meta's new 816B model only uses ~5% of its parameters per token. I made an animated explainer of how Mixture of Experts works

Muse Spark Open has 816B parameters but only ~42B are active for each token: a router picks 8 experts out of 256. I tried to explain visually why dense models get slow, how the router works, the surprising thing about what experts actually learn, and the big catch (all experts still need to sit in…

Read the full story at r/learnmachinelearning ↗

Timeline · 1 report

  1. 2026-10-21 20:33 · r/learnmachinelearning
    Meta's new 816B model only uses ~5% of its parameters per token. I made an animated explainer of how Mixture of Experts works

More stories

  1. Anthropic takes on Muse with Claude Duet, its new AI agent — The Verge AI
  2. Meta says it will create a new class of nonvoting stock to raise the capital it needs for developing AI models, with an effective date in January 2027 — Techmeme
  3. Meta's Muse hits 10M US downloads in 40 days, a milestone reached by only 11 other apps — TechCrunch AI
  4. Anthropic launches Claude Duet, its answer to Meta and OpenAI's personal agents — Business Insider AI
  5. Mark Zuckerberg says Muse has drawn 10M+ users since its launch and 25%+ of users are making purchases through the app, with an average purchase size of $200+ (Business Insider) — Techmeme
  6. Jundot--Qwen3.8-Flash-Next-oQ4-mtp running on a Mac Studio M2 192gb with oMLX and OpenwebUI thinks it is Claude??? — r/LocalLLM
  7. Meta Muse exposes users' personal and business secrets when they share conversations — Ars Technica AI
  8. Introducing TorchTPU — PyTorch Blog

Get the daily brief of stories like this at 6:30 every morning →