AINewsnow

Why MuonGS Beats AdamW and Muon in LLM Pretraining

This story is from 2026-10-29. It is preserved in the archive; the latest stories are on the live feed.

Standard AdamW and vanilla Muon waste millions of GPU hours during LLM pretraining because uncontrolled weight-norm growth throttles the effective learning rate. We cannot keep burning H100 clusters on sub-optimal trajectory dynamics when Gram-Schmidt projection gives us direct mathematical control…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-29 10:15 · DEV Community — AI
    Why MuonGS Beats AdamW and Muon in LLM Pretraining

More stories

  1. Google launches Gemini 4 Carbon with a ‘significant leap’ in cybersecurity — The Verge AI
  2. Sources: DeepSeek and Moonshot AI heavily relied on GPT-6 Astra distillation to train models like Kimi K3.1 Pro and V4.1-Flash 1017, after Anthropic's crackdown (The Information) — Techmeme
  3. 8 examples of Gemini 4 Argon in action — Google Gemini Blog
  4. Alogic Launches Crux Wireless Mouse For Productivity And AI Workflows — Forbes AI
  5. Running Qwen 3.8 Flash Next — r/LocalLLaMA
  6. Tencent targets Southeast Asia as launch pad for AI agent WorkBuddy amid tough local market — South China Morning Post Tech
  7. API credits, now in ChatGPT Pro and Business plans — OpenAI YouTube
  8. OpenAI and ServiceNow expand partnership across enterprise workflows — OpenAI News

Get the daily brief of stories like this at 6:30 every morning →