I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens
This story is from 2026-10-04. It is preserved in the archive; the latest stories are on the live feed.
Coverage of "I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens" from 1 source, with a live timeline of who reported what and when.
Read the full story at r/huggingface ↗
Timeline · 1 report
- 2026-10-04 15:53 · r/huggingface
I trained a 3.87B MoE (1.45B active) from scratch on only 86.5B tokens