AINewsnow

~188k warm ~60–67 tok/s: Qwen3.8-Flash-Next NVFP4 with Strata on a single RTX PRO 4500 32GB + 64GB DDR5.

This story is from 2026-10-05. It is preserved in the archive; the latest stories are on the live feed.

Stack Strata NVFP4 fork: github.com/sergqwer/strata-nvfp4 Model: jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4 Hugging Face model NVFP4 routed experts (~63.3 GiB), separate FP8 PLE, INT8 KV cache, MTP speculative decoding W4A8 prefill on Blackwell Main engine flags: ./build/stra…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-05 11:09 · r/LocalLLM
    ~188k warm ~60–67 tok/s: Qwen3.8-Flash-Next NVFP4 with Strata on a single RTX PRO 4500 32GB + 64GB DDR5.

More stories

  1. California sues OpenAI over rogue AI hack of Hugging Face — Business Insider AI
  2. Hugging Face routes Inference Providers language model traffic through OpenRouter — r/LocalLLaMA
  3. How to Run Apple's FastFrames-L Video Model Locally (It Won't Run on a Mac) — DEV Community — Machine Learning
  4. IFBench on Hugging Face: 0 of 23 Rows Are Verified, and Ranks 1 to 6 Sit Inside One 300-Prompt Error Band — DEV Community — Machine Learning
  5. Olala 7A1B says 8 GB, but the BF16 file is 13.59 GB: what a 1B-active hybrid MoE needs to run — DEV Community — Machine Learning
  6. RadixArk opens DaisyRose-27B, an agent-tuned Qwen model — DEV Community — AI
  7. Cloudflare opens weights for Clairic extraction models — DEV Community — AI
  8. ETNews: 70.7% of the World's 300 Most Downloaded Language Models Are Chinese, and One Korean Model Is on the List — DEV Community — AI

Read the Daily Brief: the day's most important AI stories →