AINewsnow

Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~40-50 tok/sec decode & 1300-1500 tok/sec prefill at 131k ctx on 12GB VRAM + 32GB RAM + NVME

I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3_S with just 12GB VRAM + 32GB RAM + NVME, achieving 40-50 tok/sec decode & 1300-1500 tok/sec prefill @ 131k context (44k thinking budget), on a custom fork of Strata. Don’t see IQ3_S and instantly dismiss it, this recipe uses an abliterated…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-16 21:18 · r/LocalLLaMA
    Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~40-50 tok/sec decode & 1300-1500 tok/sec prefill at 131k ctx on 12GB VRAM + 32GB RAM + NVME

More stories

  1. Deploy and evaluate models with Amazon SageMaker JumpStart — AWS Machine Learning Blog
  2. Reflection AI releases open-weight Beam model for agentic coding — AWS Machine Learning Blog
  3. Anthropic pitches AI safety bill to limit government model blocks — Politico Technology
  4. Be There for the Beginning: Celebrate the Launch of NVIDIA RTX Spark — NVIDIA Blog
  5. Replit is no longer working with Anthropic, CEO says — TechCrunch AI
  6. AWS introduces evaluation framework for AI agents on Bedrock — AWS Machine Learning Blog
  7. Put your dot on call — OpenAI YouTube
  8. Filings: Meta reduced its $65M midterms budget by $25M through refunds, primarily from a super PAC it launched in 2025 with $45M to fight state AI bills — Techmeme

Get the daily brief of stories like this at 6:30 every morning →