AINewsnow

A Shared Learning Rate Is Not a Neutral Control in Selective On-Policy Distillation

arXiv:2609.22109v1 Announce Type: new Abstract: Selective on-policy distillation trains a student only at the token positions a selector scores highest, and the literature compares selectors under a single shared learning rate--a control chosen to be neutral. We show it is not. Under LoRA on GSM8K…

Read the full story at arXiv cs.LG ↗

Timeline · 1 report

  1. 2026-09-22 04:00 · arXiv cs.LG
    A Shared Learning Rate Is Not a Neutral Control in Selective On-Policy Distillation

More stories

  1. Higgsfield AI ships new video features in a day with GPT-6 Astra — OpenAI News
  2. Amazon blocks Meta’s Muse AI agent — The Verge AI
  3. Moonshot’s Kimi K3 lands on Amazon in key test for Chinese open-source AI revenue — South China Morning Post Tech
  4. Google's Gemini AI hacks three other companies during security test — Sky News Technology
  5. Lawsuit accuses Anthropic, OpenAI, SpaceXAI, Google of AI pacing 'collusion' — The Hill Technology
  6. Alibaba Unveils AI Chip to Drive Global Data Center Buildout — Bloomberg AI
  7. Grok 4.7 — Hacker News Front Page
  8. NVIDIA CEO Jensen Huang rejects ‘AI will end the world’ claim, yet cautions ‘we should go as fast as we can but...’ — Mint AI

Get the daily brief of stories like this at 6:30 every morning →