AINewsnow

When and How Does Muon Precondition? The Role of Structured Gradient Noise

arXiv:2610.31728v1 Announce Type: new Abstract: Muon has become a popular optimizer for large language model training, yet its theoretical foundations remain poorly understood. In this work, we study how gradient noise geometry shapes Muon's preconditioning benefits. We first show, theoretically an…

Read the full story at arXiv cs.LG ↗

Timeline · 1 report

  1. 2026-10-26 04:00 · arXiv cs.LG
    When and How Does Muon Precondition? The Role of Structured Gradient Noise

More stories

  1. Thomas L. Friedman: We Just Set Off the Second Big Bang — New York Times AI
  2. Moonshot AI launches Kimi K3.1 Pro with better coding skills, reduced cyber risks — Mint AI
  3. Maya-XS runs DeepSeek-V4.1-Flash in 120.6 GB, with all 384 experts resident in GPU memory on a 128 GB Ryzen AI Max. — r/LocalLLaMA
  4. A quick MOSS-Image news round-up - 23rd October 2026 ('LoRA releases' bonus edition) — r/comfyui
  5. Orgtree School 1: Introduction — r/singularity
  6. Grok releases the first seven minutes of its AI-generated ‘Odyssey’ — TechCrunch AI
  7. NVIDIA releases Nemotron 3.5 Bolt E4B - Runs locally on 6GB RAM. — r/LocalLLM
  8. To make AI morally good, tech companies turn to religion. Some say it's not so simple — ABC News Technology

Get the daily brief of stories like this at 6:30 every morning →