Study Challenges Assumption That Teacher and Student Representations Must Align in Knowledge Distillation
A new paper argues that knowledge distillation does not require teacher and student representations to occupy compatible or explicitly aligned feature spaces, instead using random prototype spaces.
Read the full story at arXiv cs.CV ↗
Timeline · 1 report
- 2026-10-30 04:00 · arXiv cs.CV
You Don't Need Aligned Representations: Knowledge Distillation via Random Prototype Spaces