Too much of a good thing -- when knowledge distillation promotes overfitting, and how to avoid it
This story is from 2026-08-26. It is preserved in the archive; the latest stories are on the live feed.
arXiv:2608.23752v1 Announce Type: new Abstract: The growing size of Convolutional Neural Networks has led to increasingly large and costly models. Knowledge Distillation (KD) addresses this by transferring knowledge from a large network (teacher) to a small one (student), also reducing the training…
Read the full story at arXiv cs.CV ↗
Timeline · 1 report
- 2026-08-26 04:00 · arXiv cs.CV
Too much of a good thing -- when knowledge distillation promotes overfitting, and how to avoid it