AINewsnow

JetBrains Mellum 2.1 12B A2.5B MoE is a very good model.

It runs at around m2 pro 16 at 60 tps at full 130k context, you can easily run a Q8 if you have a 16 GB GPU, I was able to get 140TPS on 9070xt, It’s an MoE so I think we can run it well on smaller cards too, 8GB, 12GB should be possible to get upwards of 30-40 tps. I have used it a bit, it can cod…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-15 10:50 · r/LocalLLM
    JetBrains Mellum 2.1 12B A2.5B MoE is a very good model.

More stories

  1. DeepSeek Releases V4.1-Flash-1017 with Improved Benchmarks and Open Weights — Vercel Blog
  2. Meta releases 816B-parameter Muse Spark Open model optimized for local Mac deployment — Vercel Blog
  3. Eight US states sue OpenAI over ChatGPT safety concerns — Fortune AI
  4. Apple tightens App Store rules to block AI nudify apps — Engadget
  5. Qwen 3.8 Flash Next gains traction for high-speed local inference — r/LocalLLaMA
  6. OpenMOSS releases Apache-licensed MOSS-Image model — r/StableDiffusion
  7. NVIDIA Launches RTX Spark Laptops for Creators and AI Agents — NVIDIA Blog
  8. Scarlet and Reach Security each raise $50M Series B for AI expansion — Techmeme

Get the daily brief of stories like this at 6:30 every morning →