AINewsnow

Is Adam a second order method?

Assume you have two params, and their gradients are x & y. Define V: [[E(x^2), E(xy)], [E(xy), E(y^2)] Define M: [E(x), E(y)]] Adam's update is: M . V^-1/2 Adam just assumes the off diagonal terms like E(xy) are zero. So the update becomes m/sqrt(v) Isn't this basically a second order method like g…

Read the full story at r/learnmachinelearning ↗

Timeline · 1 report

  1. 2026-10-25 10:15 · r/learnmachinelearning
    Is Adam a second order method?

More stories

  1. Kimi K3.1 Pro and Kimi K3.1 Pro Highspeed are available now in AI Gateway — Vercel Blog
  2. Two outsiders found ways OpenAI’s AI agents could slip past the system built to catch them — r/OpenAI
  3. Thomas L. Friedman: We Just Set Off the Second Big Bang — New York Times AI
  4. Qwen 3.8 Flash Next (Q4) got stuck on this loop for hours. I wonder if this is the AI equivalent of having a nightmare and not being able to wake up — r/LocalLLM
  5. GitHub - vincalkr/DS41Engine: DeepSeek-V4.1-Flash 1017 on 1x-2x RX 7900 XTX (gfx1100, ROCm/HIP): max optimizations — r/LocalLLM
  6. Securing our frontier model training environment — OpenAI News
  7. NCERT introduces 20-hour AI module for classes 9-12, covers deepfakes and data privacy — Mint AI
  8. Block unveils Dessa, a "true speech-to-speech" AI model trained on "nothing but human conversation" that it says runs locally on a MacBook Pro M4 Max or newer — Techmeme

Get the daily brief of stories like this at 6:30 every morning →