AINewsnow

QWEN 3.8 FN - 8bit FP quant - 96.32 tokens/s - Context of 95k - $1500 hardware

This story is from 2026-11-05. It is preserved in the archive; the latest stories are on the live feed.

Did some extra optimizations and implemented MTP. High number is coding at 96t/s (3 drafts). Natural text is 69 t/s. 128.0 µs to distribute a token-residual to 24 gpus, compute experts, collect result and join in one layer. Probably at peak hardware capability now unless I get some new ideas on how…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-11-05 09:53 · r/LocalLLM
    QWEN 3.8 FN - 8bit FP quant - 96.32 tokens/s - Context of 95k - $1500 hardware

More stories

  1. Config-Verified Comparison Maps How 10 Open MoE Models Route Tokens — MarkTechPost
  2. CyberTiel35BA3B working on Blender Rigging via MCP. 8GB VRAM, 16GB RAM. llama.cpp. — r/LocalLLaMA
  3. ComfyUI Readable Node: an agent skill for turning messy workflows into readable ones — r/StableDiffusion
  4. [Release] Qwen3.8-Flash-Next DR-V2-Q2_0 GGUF - Fits in 48GB VRAM (38.4 GB Backbone), 83.3% AIME 2025, 74.7% Aider Polyglot — r/LocalLLM
  5. My bug reproduction agent trusts the container, not the model — DEV Community — AI
  6. FLUX.3 Klein x Qwen-Image 2.1 in ComfyUI: Batch Pose and Outfit Changes with a Pose x Outfit Matrix (6 Workflows) — DEV Community — AI
  7. When AI Weight Surgery Met My Daughter's Lazy Eye: A Builder's Notes on Dichoptic VR — DEV Community — Machine Learning
  8. Can you run a local LLM offline on a flight? What to check before you board — DEV Community — AI

Get the daily brief of stories like this at 6:30 every morning →