AINewsnow

A 125B model at 50+ tok/s on a 12 GB gaming card: what I changed in Strata

I run Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model, on one RTX 3060 with 12 GB of video memory and 30 GB of RAM. Nothing leaves my PC. This post is about the fork I made to get there, what I measured, and what I have not tested. Starting point: Strata The model is about 78 G…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-03 15:13 · DEV Community — Machine Learning
    A 125B model at 50+ tok/s on a 12 GB gaming card: what I changed in Strata

More stories

  1. OpenAI launches Grid Protectors Program to cut data center power use during grid strain — OpenAI News
  2. OpenAI hires former Army Secretary Dan Driscoll as senior counselor for national security — OpenAI News
  3. Security flaw found in Claude Code extension listed on Cursor's marketplace — Runway Research
  4. OpenAI raises bug bounty rewards and plans new security sprint with outside researchers — OpenAI News
  5. OpenAI seeks to rebrand data centers as 'compute campuses' amid growing local backlash — Techmeme
  6. Anthropic launches Message Threads in public beta, cutting agent request payloads from 800 KB to 1.8 KB — Anthropic (X)
  7. UK cyber chief warns AI-powered malware likely to hit critical infrastructure by 2028 — The Guardian AI
  8. AI Is Changing How Semiconductor Fabs Work — and Engineers Are Moving Faster Because of It — NVIDIA Blog

Get the daily brief of stories like this at 6:30 every morning →