AINewsnow

I built a small optimizer: AdamW where the weights only move toward a scout that passed a test (on par with tuned AdamW on small tasks, feedback welcome)

I've been experimenting with an optimizer idea and would love feedback. Idea. Two copies of the weights. a1 is the main copy. a2 is a scout that runs ahead with AdamW at its own adaptive learning rate. Every k steps I compare a1 and a2 on one fresh batch (paired per-example losses). If the scout is…

Read the full story at r/learnmachinelearning ↗

Timeline · 1 report

  1. 2026-10-17 06:42 · r/learnmachinelearning
    I built a small optimizer: AdamW where the weights only move toward a scout that passed a test (on par with tuned AdamW on small tasks, feedback welcome)

More stories

  1. DeepSeek Releases V4.1 Flash 1017 with Improved Agentic Performance and Cost Efficiency — Vercel Blog
  2. Community debates Qwen Flash Next performance and quantization options — r/LocalLLaMA
  3. Meta releases Muse Spark Open, an 816B-parameter open-weight AI model — AWS Machine Learning Blog
  4. NVIDIA RTX Spark Laptops Launch Globally With AI Agent Support — NVIDIA Blog
  5. Anthropic pitches AI safety bill to limit government model blocks — Politico Technology
  6. Scarlet and Reach Security each raise $50M Series B for AI expansion — Techmeme
  7. Anthropic offers Blackstone double-digit yield in $60B chip financing deal — Financial Times AI
  8. Google releases Gemini 4 Argon to Pro subscribers with limited access — r/ArtificialInteligence

Get the daily brief of stories like this at 6:30 every morning →