AINewsnow

Why Meta trained a small 8B model on 15 trillion tokens, and why that's cheaper than it sounds

This story is from 2026-10-20. It is preserved in the archive; the latest stories are on the live feed.

There's a rule of thumb from a famous paper: for the best result per dollar of training, feed a model about 20 tokens of text for every parameter it has. An 8-billion-parameter model times 20 is 160 billion tokens . Meta's own blog post puts the "Chinchilla-optimal" amount for an 8B model at about…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-20 17:47 · DEV Community — AI
    Why Meta trained a small 8B model on 15 trillion tokens, and why that's cheaper than it sounds

More stories

  1. DeepSeek V4.1 Flash 1017 [GGUF + Abliterated] — r/LocalLLaMA
  2. Introducing TorchTPU — PyTorch Blog
  3. Meta pauses Muse sharing after researchers find AI chats on the web — The Verge AI
  4. Google open-sources Harness²: agents that rewrite their own harness on unseen tasks, with frozen weights and no labels — r/machinelearningnews
  5. 32GB to 64GB of RAM took Qwen3.8-Flash-Next from IQ2_XS (2-bit) to Unsloth UD-IQ4_XS (4-bit) on a single 4090 — r/LocalLLaMA
  6. GenRecs Tokenization: Transforming Ads Recommendation at Meta with Stable Model Vocabulary — Engineering at Meta
  7. The Thin Orchestrator: How We Made Our AI Agents More Reliable by Giving the Router Less to Do — Engineering at Meta
  8. Joel Kaplan, Meta’s Global Policy Chief, to Leave After 15 Years — New York Times AI

Get the daily brief of stories like this at 6:30 every morning →