AINewsnow

Qwen3.8-27B at 147 tok/s on one CMP 170HX — 65 tok/s at 250K CTX

I’ve been tuning Qwen3.8-27B + DFlash2 on a single NVIDIA CMP 170HX 64GB (SM80) with vLLM 0.27.1. I’m finally calling the main optimization path done. Results Single-stream decode, greedy, fresh engine per cell, 1350 MHz / 180W : Context Peak decode 4K 134.7 tok/s 32K 147.0 tok/s 65K 100.1 tok/s 12…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-09-20 07:06 · r/LocalLLM
    Qwen3.8-27B at 147 tok/s on one CMP 170HX — 65 tok/s at 250K CTX

More stories

  1. NVIDIA CEO Jensen Huang rejects ‘AI will end the world’ claim, yet cautions ‘we should go as fast as we can but...’ — Mint AI
  2. Building an open-source 500+ language Sparse MoE translation model from scratch (Apache 2.0) — r/huggingface
  3. what's the state of the art recipe for running Qwen3.8-Flash-Next with a pair of 3090s and a ton of system RAM rn? — r/LocalLLaMA
  4. Huawei details AI accelerator roadmap, pulls in next-generation Ascend NPUs by several quarters — FP4 performance of the Ascend 960PR doubles expectations — Tom's Hardware
  5. Flyweight: open-source C++/CUDA engine for running MoE models bigger than your VRAM on one GPU + system RAM. First PyPI release, looking for contributors. — r/LocalLLaMA
  6. Built a home server from an old PC with GPU upgrade. Qwen3.8 27B runs at ~30 tokens per second. — r/LocalLLaMA
  7. FREE AI TRAINING CREDIT — r/learnmachinelearning
  8. No one is surprised that Nvidia's Jensen Huang thinks AI fears are overblown. — The Verge AI

Get the daily brief of stories like this at 6:30 every morning →