Update on my tiny from-scratch model - followed your advice, here's what happened
What I did: Scaled the dataset from 60 hand-written functions to 12,000 procedurally generated ones (math ops, list manipulation, string ops, conditionals, and loop algorithms). Built a custom BPE tokenizer (vocab 1024). Funny story: my first tokenizer (vocab 4096) literally memorized entire functi…
Read the full story at r/learnmachinelearning ↗
Timeline · 1 report
- 2026-10-02 15:17 · r/learnmachinelearning
Update on my tiny from-scratch model - followed your advice, here's what happened