AINewsnow

Qwen3.8-27B at 262K context on a Strix Halo + RTX 3090 Ti: 9.5 -> 153 tok/s, and it beats a dual-3090 vLLM box on HumanEval

This story is from 2026-08-21. It is preserved in the archive; the latest stories are on the live feed.

Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128 GB unified) + RTX 3090 Ti on an eGPU link. One llama.cpp process, AMD on Vulkan, NVIDIA on CUDA, o…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-08-21 06:06 · r/LocalLLaMA
    Qwen3.8-27B at 262K context on a Strix Halo + RTX 3090 Ti: 9.5 -> 153 tok/s, and it beats a dual-3090 vLLM box on HumanEval

More stories

  1. Runway News | Runway Dev is in Claude Code, Codex and Cursor — Runway Research
  2. I built a one-click installer for local LLMs on Windows: it picks a model for your hardware and sets up Ollama — DEV Community — AI
  3. SSD MoE finally viable! — r/LocalLLM
  4. Mistral Large 4 Is Open Weight. Read the License and Do the Memory Math Before You Self-Host — DEV Community — AI
  5. Democratic midterm win could bring tighter scrutiny of major AI companies — Mint AI
  6. Now that some time has passed: which Qwen3.8 27B quant do you stick with? — r/LocalLLM
  7. Configuring llama.cpp for Enterprise Environments: A Comprehensive Guide to Concurrency and Stability — DEV Community — AI
  8. Independent study tests Direct Nash Optimization for aligning language models — DEV Community — Machine Learning

Get the daily brief of stories like this at 6:30 every morning →