Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model
TL;DR On a CPU, how fast a language model writes text depends mostly on how many bytes of weights it must read for each token , not on how many parameters it has in total. A sparse Mixture-of-Experts (MoE) model reads only the experts that are active for that token. That is why POCKET-35B , a 35B M…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-01 03:14 · DEV Community — Machine Learning
Why a Sparse 35B MoE Runs Faster on a CPU Than a Dense 27B Model