AINewsnow

Lower loss, worse stories: small NCMs trained on TinyStories

I trained small Next-Concept Modeling (NCM) models from scratch in MLX on TinyStories. They reconstruct held-out stories almost perfectly, but the text they write on their own is a lot worse than text from a plain autoregressive (AR) baseline built from the same transformer blocks. And the part I d…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-02 21:39 · DEV Community — Machine Learning
    Lower loss, worse stories: small NCMs trained on TinyStories

More stories

  1. Anthropic reports $17B quarterly revenue, 1,100% growth and $900B spending commitments ahead of IPO — Reuters AI
  2. Introducing the Grid Protectors Program — OpenAI News
  3. Marvell to acquire Lumentum in $115 billion all-stock AI connectivity deal — DigiTimes Asia
  4. Israeli startup 42AI exits stealth with $62.5M seed for behavioral world model — TechCrunch AI
  5. OpenAI to merge ChatGPT's Chat and Work tabs into a single desktop experience — TechCrunch AI
  6. October slide in AI and tech stocks tests broader US market's resilience — Reuters AI
  7. Malicious Claude Code extension in Cursor marketplace attempted SSH backdoor — Microsoft Research Blog
  8. OpenAI weighs rebranding data centers as 'compute campuses' amid growing public backlash — Techmeme

Get the daily brief of stories like this at 6:30 every morning →