OPMD: On-Policy Mechanism Distillation Beyond Output Supervision
arXiv:2610.14022v1 Announce Type: new Abstract: On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains confined to the output space. Consequently, OPD transfers what a teacher predicts without supervising th…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-14 04:00 · arXiv cs.LG
OPMD: On-Policy Mechanism Distillation Beyond Output Supervision