Study Examines How Next-Token Training Shapes Latent Fact Composition in Transformers
A controlled study finds that prior work on latent two-hop reasoning trains transformers to predict only the answer, while language models are trained to predict every token, leaving latent fact composition under next-token training less understood.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-18 05:00 · arXiv cs.CL
Latent Fact Composition under Next-Token Training