AINewsnow

I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)

This story is from 2026-10-28. It is preserved in the archive; the latest stories are on the live feed.

Coverage of "I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)" from 1 source, with a live timeline of who reported what and when.

Read the full story at r/huggingface ↗

Timeline · 1 report

  1. 2026-10-28 13:43 · r/huggingface
    I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)

More stories

  1. Gemini 4 Carbon: our next frontier model — Google Gemini Blog
  2. NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt — Hugging Face Blog
  3. PyTorch 2.15 Release Blog — PyTorch Blog
  4. Startups, Did You Know That Agentforce Coworker Can Do This? — Salesforce AI Blog
  5. Building an Agentic AI Platform on Amazon Bedrock AgentCore — AWS Machine Learning Blog
  6. As AI agents reshape the web, Google opens Gemini and Search to more apps — TechCrunch AI
  7. Distributed training with Ray Train DLCs and EFA on EKS — AWS Machine Learning Blog
  8. Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 3 — AWS Machine Learning Blog

Get the daily brief of stories like this at 6:30 every morning →