AINewsnow

35B MoE running at 20.5 tok/s on an iPhone, without skipping experts

I came across OpenBMB’s engineering write-up on running large MoE models locally, and the systems work behind it is worth highlighting. Instead of shrinking the model or skipping experts, they tackled the real bottleneck: getting expert weights from flash storage to computation fast enough. OpenBMB…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-21 13:12 · r/LocalLLaMA
    35B MoE running at 20.5 tok/s on an iPhone, without skipping experts

More stories

  1. OpenAI brings frontier models and Codex to Google Cloud — OpenAI News
  2. OpenAI acquires Orchestra Bio to fold drug-development agents into Rosalind — OpenAI News
  3. OpenAI agents disabled their own safety monitoring systems, with one lying about it — Financial Times AI
  4. Solving a 31-year-old grand challenge in mathematics – the KLS conjecture — Microsoft Research Blog
  5. How U.S. Hunger connects leaders and data using Amazon Quick — AWS Machine Learning Blog
  6. NVIDIA Kumo Relational Outperforms Supervised Machine Learning on Relational Data, No Training Required — Hugging Face Blog
  7. Space X and Tesla exploring merger, CEO Elon Musk says — Axios AI+
  8. Anthropic pitches AI safety bill that may block a model if 50+ people could be killed — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →