AINewsnow

Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine

This story is from 2026-10-03. It is preserved in the archive; the latest stories are on the live feed.

We built an engine, Kyojin, on top of ExLlamaV3 for Strix Halo (gfx1151, ROCm), and packed two 300B-class MoE models so each fits one 128 GB machine. First release, all measured on Ryzen AI Max+ 395. Model GLM-5.3-Flash MiMo-V2.6-Flash-MOPD Size 99.7 GB 105 GB Prefill 580 tok/s at 3.5K, 546 at 64K…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-03 14:45 · r/LocalLLM
    Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine
  2. 2026-10-03 14:16 · r/LocalLLaMA
    Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine

More stories

  1. 27,280 Agent Runs Prove Your Harness Matters More Than Your Model — DEV Community — AI
  2. Baseten Inference Infrastructure Enters Public Preview on Microsoft Foundry — Unite.AI
  3. China AI Highflier Crashes 85% as Chip Fears Hit Model Makers — Bloomberg AI
  4. Alibaba's QwenWork Adds Third-Party Models From Kimi, GLM and DeepSeek — Pandaily
  5. OpenAI launches GPT-6.2 Astra and Sol as Google ships Gemini 4 Neon and Microsoft unveils MAI Code 2 — OpenAI News
  6. Anthropic debuts on Nasdaq with record $81.3 billion IPO, hitting $2 trillion value — Anthropic (X)
  7. Google launches Gemini 4 Neon and opens Genie 4 world model to Gemini API developers — Google DeepMind YouTube
  8. Google books Blue Origin's New Glenn for 2029 launch of AI server satellites — Google: The Keyword

Read the Daily Brief: the day's most important AI stories →