AINewsnow

Qwen3.8-Flash-Next IQ3_XXS - 60GB total memory, 50-70tps, very impressed.

A bit late to the party but I decided to give Strata a try on my 4070Super + 48GB DDR4-3200 system that is my Blenderbox. To sum up I'm quite impressed. Getting 50-70tok/s at ~6k context, tool calls and web searches working well, doesn't seem lobotomized despite being a 3-bit quant. A few niggles I…

Read the full story at r/LocalLLM ↗

Timeline · 2 reports

  1. 2026-10-29 18:33 · r/LocalLLM
    Upgraded from RX 9070 XT (16GB, ROCm) to RTX 5090 (32GB, CUDA) on CachyOS Linux — 236 tok/s Decode & 6,272 tok/s Prefill on Qwen3.8-Flash-Next (IQ3_S) in Strata
  2. 2026-10-29 03:11 · r/LocalLLM
    Qwen3.8-Flash-Next IQ3_XXS - 60GB total memory, 50-70tps, very impressed.

More stories

  1. Sources: Indian AI infrastructure company AM Intelligence seeks to raise as much as $3B to fund a previously announced purchase of Nvidia Vera Rubin systems (Bloomberg) — Techmeme
  2. Canada to Take Equity Stake in AI Firm Cohere — Wall Street Journal Technology
  3. Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS — AWS Machine Learning Blog
  4. Into the Omniverse: How Developers Turn Ideas Into Simulations With Frontier AI Agents — NVIDIA Blog
  5. Could Nvidia control how AI models reach developers? Why DOJ is worried about the $12.93 billion Hugging Face deal — Mint AI
  6. NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters — MarkTechPost
  7. NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt — Hugging Face Blog
  8. Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4 — NVIDIA Technical Blog

Get the daily brief of stories like this at 6:30 every morning →