AINewsnow

A Record of the Process of Increasing MFU from 21% to 50% While Pre-training 21B MoE Using a Single GH200 Chip (Includes Code and Logs)

I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have compiled the metrics I measured during that process into a paper. https://preview.redd.it/77womlml91vh…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-12 12:52 · r/LocalLLaMA
    A Record of the Process of Increasing MFU from 21% to 50% While Pre-training 21B MoE Using a Single GH200 Chip (Includes Code and Logs)

More stories

  1. NVIDIA Opens Vera Rubin Architecture to Scale AI Factories — NVIDIA Blog
  2. Microsoft to deploy Maia 300 AI chips to reduce Nvidia dependence — CNBC Technology
  3. NVIDIA RTX 5080 24GB reportedly launching in Q1 2027, board partners still waiting for details — r/LocalLLM
  4. Microsoft unveils Maia 300 AI chip to reduce Nvidia reliance — The Verge AI
  5. Application for managing LLM sessions in a local environment — r/LocalLLM
  6. Diamond-infused watercooling coldplates for 2,300W Nvidia Rubin drop temps by up to 22C — Tom's Hardware
  7. First LLM Server - Questions — r/LocalLLM
  8. Any takers for local llm? NVIDIA’s CUDA Moat Is Accelerating RTX Spark Laptop Sales, Higher 128GB Unified Memory Options Belonging To Surface Laptop Ultra Already Out Of Stock — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →