Study measures tokenization premium and sovereignty cost of inherited language models
A new arXiv paper argues most language models deployed outside the US and China are fine-tuned descendants of a few pretrained families, quantifying the tokenization premium and sovereignty costs this imposes on the Global South.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-11 05:00 · arXiv cs.CL
Inherited Weights: Measuring the Tokenization Premium and the Sovereignty Cost of Language Models for the Global South