AINewsnow

Teachers other than EMA for distillation ? [R]

Hi guys, Self-distillation methods, see Self-Distillation Fine Tuning SDFT, usually use an EMA (Exponential Moving Average) teacher to get synced and stable targets. I wanted to ask the community for research papers/ works that consider different types of teachers that also hold these properties (t…

Read the full story at r/MachineLearning ↗

Timeline · 1 report

  1. 2026-10-27 15:53 · r/MachineLearning
    Teachers other than EMA for distillation ? [R]

More stories

  1. DeepMind unveils Genie 4 world model promising more realistic physics and humans — Ars Technica AI
  2. Anthropic reveals plans for blockbuster IPO — Sky News Technology
  3. Muse Is Now Available in Singapore: The World’s First Personal AI Agent Built for Everyone — Meta Newsroom
  4. Introducing GPT-6.1 Terra — OpenAI News
  5. Nvidia-Backed Firm Launches $3.1 Billion GPU Loan into Market — Bloomberg AI
  6. Microsoft to reportedly offer buyouts to 60,000 employees in latest cuts — The Guardian AI
  7. Apple Creator Studio update adds MotionVFX to Final Cut, and AI to everywhere — Engadget
  8. Frontier AI labs say their models refuted the Exponential Time Hypothesis. What does that mean? — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →