Token Utility for On-Policy Distillation via Local Trust-Region Optimization
arXiv:2610.18883v1 Announce Type: new Abstract: Token-selection methods for on-policy distillation (OPD) prioritize student-visited positions using diverse signals, including uncertainty, teacher-student disagreement, and mismatch direction. We instead derive token importance from the distillation…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-15 04:00 · arXiv cs.LG
Token Utility for On-Policy Distillation via Local Trust-Region Optimization