Predicting Privileged Information Effectiveness for On-Policy Self-Distillation
arXiv:2610.22512v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) enables large language models (LLMs) to learn from privileged information (PI) without a separate teacher model. However, prior work largely treats PI as a fixed design choice, leaving the effects of its content and…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-19 04:00 · arXiv cs.LG
Predicting Privileged Information Effectiveness for On-Policy Self-Distillation