Conceptizer: Next-Token Prediction Is Not All You Need
arXiv:2610.33027v1 Announce Type: new Abstract: Next-token prediction (NTP) rewards only the token that occurs next, penalizing every alternative. But natural language often offers many ways to express the same meaning. We introduce Conceptizer, a training framework that extends next-token predicti…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-27 04:00 · arXiv cs.CL
Conceptizer: Next-Token Prediction Is Not All You Need