Meta trained small AI models to read raw bytes instead of tokens. They start out worse, but the paper predicts they end up better
https://arxiv.org/pdf/2609.12303 this paper suggests a model that skips models reading tokens that and has the model read raw bytes, basically one letter at a time. what they did: trained small models (~1B params), some on tokens and some on bytes trained them partly by copying a bigger model (llam…
Read the full story at r/LocalLLaMA ↗