Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
arXiv:2610.21025v1 Announce Type: new Abstract: When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes through the unembedding matrix, or readout, which is itself still being le…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-16 04:00 · arXiv cs.CL
Learning to Read Out: Unembedding Dynamics in Language Model Pretraining