AINewsnow

The Ultimate MoE Training Handbook: training Mixture-of-Experts models with Megatron-LM, setting by setting [P]

This story is from 2026-11-05. It is preserved in the archive; the latest stories are on the live feed.

Handbook: https://huggingface.co/spaces/huggingface/moe-blogpost Hello, I'm Nouamane, one of the authors. I'm at Hugging Face and also worked on the Ultra-Scale and Smol Training Playbooks, if you've read those. This handbook is for people who have trained a dense model and want to train an MoE nex…

Read the full story at r/MachineLearning ↗

Timeline · 1 report

  1. 2026-11-05 19:13 · r/MachineLearning
    The Ultimate MoE Training Handbook: training Mixture-of-Experts models with Megatron-LM, setting by setting [P]

More stories

  1. olmOCR-bench on Hugging Face: Three Rows Use a Different Average, and One Moves From 13th to 7th or 19th — DEV Community — Machine Learning
  2. Moonshot Just Put a 2.8-Trillion-Parameter Model on Hugging Face — Kimi K3.1 Pro's Open Weights Are Real, and the Fine Print Matters — DEV Community — AI
  3. The Open-Source Paradox: Giving Away AI Makes It Safer (and More Dangerous) — DEV Community — AI
  4. Huggingface - WOW.. 2 ZeroGPU Spaces for all free accounts? — r/LocalLLaMA
  5. [Release] Qwen3.8-Flash-Next DR-V2-Q2_0 GGUF - Fits in 48GB VRAM (38.4 GB Backbone), 83.3% AIME 2025, 74.7% Aider Polyglot — r/LocalLLM
  6. Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure — DEV Community — AI
  7. Viggle Remake MCP now has a ComfyUI worker (int8 + offload). Raw results: what it nails and where it breaks — r/StableDiffusion
  8. Viggle Remake is a beast — r/StableDiffusion

Get the daily brief of stories like this at 6:30 every morning →