Why Small LLMs Memorize: I Trained the Same Model at Four Sizes and Watched It Start Reciting
Part 3 of ten. Each post is a short, plain-words tour of one lesson from e=mc² , a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site. The same little language model, trained on the same stories for the same 1,500 steps, either makes things up or recites, a…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-17 12:30 · DEV Community — Machine Learning
Why Small LLMs Memorize: I Trained the Same Model at Four Sizes and Watched It Start Reciting