Study finds recursive self-training can shrink LLM vocabulary while boosting frequent words
A new arXiv paper on model collapse shows recursive self-training of language models can reduce vocabulary, with frequent function words dominating. It introduces ordered Renyi crossings to measure this unevenness.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-09 05:00 · arXiv cs.CL
Richness Is Not Evenness: Ordered Renyi Crossings under Self-Training