AINewsnow

Built a 30M-parameter GPT-style language model from scratch

Hey everyone I've been learning about Transformers and LLMs, so I built KERN , a 30M parameter GPT-style language model in PyTorch,trained from scratch on TinyStories. GitHub: https://github.com/OmarDawoud4/KERN This is still a work in progress! I'd like to keep improving it by implementing techniq…

Read the full story at r/learnmachinelearning ↗

Timeline · 1 report

  1. 2026-10-21 20:38 · r/learnmachinelearning
    Built a 30M-parameter GPT-style language model from scratch

More stories

  1. OpenAI brings frontier models and Codex to Google Cloud — OpenAI News
  2. Anthropic takes on Muse with Claude Duet, its new AI agent — The Verge AI
  3. Meta says it will create a new class of nonvoting stock to raise the capital it needs for developing AI models, with an effective date in January 2027 — Techmeme
  4. Meta's Muse hits 10M US downloads in 40 days, a milestone reached by only 11 other apps — TechCrunch AI
  5. OpenAI acquires Orchestra Bio to fold drug-development agents into Rosalind — OpenAI News
  6. Securing Quick access from corporate networks with AWS PrivateLink — AWS Machine Learning Blog
  7. 5 production challenges in multi-agent systems and how Deep Insight solved them — AWS Machine Learning Blog
  8. OpenAI, PerturbAI, and Scripps say thousands of GPT-Rosalind agents analyzed data on millions of mouse brain cells in 32 minutes, a job that takes humans months (OpenAI) — Techmeme

Get the daily brief of stories like this at 6:30 every morning →