Why LLMs Can't Count the R's in Strawberry: A Look Inside BPE Tokenization
This story is from 2026-09-16. It is preserved in the archive; the latest stories are on the live feed.
A model never sees letters. Every prompt gets chopped into tokens before a single parameter runs, and a surprising share of "the model can't do basic things" bug reports trace straight back to where those cuts land. I built an actual tokenizer against OpenAI's published vocabularies to measure this…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-09-16 07:38 · DEV Community — AI
Why LLMs Can't Count the R's in Strawberry: A Look Inside BPE Tokenization