AINewsnow

Qwen3.8 27B on one RTX 5090: six inference engines benchmarked with and without DFlash2

Disclosure: I’m the author of TensorSharp. This English summary was adapted and edited with LLM assistance from the full benchmark report . The comparison covers llama.cpp, vLLM, SGLang, NInfer, velox-inferer, and TensorSharp on the same RTX 5090 32 GB, including decode throughput, long prompts, co…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-27 13:15 · r/LocalLLaMA
    4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)
  2. 2026-10-27 06:20 · r/LocalLLaMA
    Qwen3.8 27B on one RTX 5090: six inference engines benchmarked with and without DFlash2

More stories

  1. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  2. I trained LLaMa from scratch — r/deeplearning
  3. Basalt (NVFP4) vs Unsloth Studio (llama.cpp IQ4_XS) — 5090 with 64GB RAM — r/LocalLLM
  4. Gemma 3 1B on the Nexus 5X (Directly on the Linux Kernel, No Android) — 2 GB RAM, 4.47 t/s — r/LocalLLM
  5. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  6. A 2013 Mac Pro runs a 35B model at 66 tok/s reading. The first benchmark I trusted was wrong by about 9x. — r/LocalLLM
  7. mtmd: enable video input support for LFM3 model family by tdakhran · Pull Request #31283 · ggml-org/llama.cpp — r/LocalLLaMA
  8. shura.cpp: a llama.cpp fork with a VRAM governor, so a 35B MoE can share one 16 GB card with the desktop you keep using — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →