AINewsnow

Two 16GB V100s, no NVLink: Qwen3.8-27B NVFP4 at 253+ tok/s(DFlash2/7) , 143 tok/s with 85K input (MTP)

I wanted to share a small project I've been working on: NInfer-V100X2, a C++/CUDA inference engine fork focused on getting more performance out of two older V100s. Repo: https://github.com/tuxKOH/ninfer-V100X2 The setup is two Tesla V100-SXM2 16GB cards on SXM2-to-PCIe adapters, working PCIe P2P, n…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-13 15:16 · r/LocalLLM
    Two 16GB V100s, no NVLink: Qwen3.8-27B NVFP4 at 253+ tok/s(DFlash2/7) , 143 tok/s with 85K input (MTP)

More stories

  1. Problems with Tesla V100 SXM2 — r/LocalLLaMA
  2. NVIDIA’s 9-Year-Old Tesla V100 Still Packs Some Inference Muscle; 4 GPUs On An NVLink Board Can Outperform M5 Ultra, DGX Spark In Various AI Benchmarks By Up To 230% — r/LocalLLaMA
  3. Tesla P100 image generation... much better than I expected! — r/LocalLLM
  4. UCLA builds a humanoid from smartphone — The Rundown AI
  5. First LLM Server - Questions — r/LocalLLM
  6. 😸 White House orders AI incident reporting — The Neuron
  7. PXA v3.1: Qwen3.8 Flash-Next at 31 t/s on a single Tesla P100, 55 t/s on two V100s (open source, beats Strata on prose + prefill) — r/LocalLLM
  8. Tesla Powerwall and Car Backup During an Outage — Matthew Berman

Get the daily brief of stories like this at 6:30 every morning →