The Byte Tokeniser Posts the Lowest Loss per Token on the Page and Is the Worst Model on It
This story is from 2026-08-25. It is preserved in the archive; the latest stories are on the live feed.
The cross-entropy in your training log is an average over a unit the tokeniser invented: loss = (1/T) * sum_t -log p(token_t | context) The sum is a property of the model and the data. The 1/T is a property of the tokeniser , because T is how many pieces it cut the data into. Two tokenisers divide…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-08-25 13:40 · DEV Community — Machine Learning
The Byte Tokeniser Posts the Lowest Loss per Token on the Page and Is the Worst Model on It