AINewsnow

V100 16GB: A 2017 GPU running a 35B vision model at 93 tok/s: Tesla V100 16 GB on Windows

Old Tesla V100-SXM2 16 GB (Volta, 2017), plain Windows 11, no WSL/Docker. MiniCPM-V 4.7 35B-A3B: ~93 tok/s, vision + 128k context, one 16 GB card. No custom kernel, no custom quant: stock llama.cpp + OpenBMB's official mix_q3gud_im (15.5 GB). Just tuned settings: --no-mmproj-offload (vision encoder…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-17 08:37 · r/LocalLLM
    V100 16GB: A 2017 GPU running a 35B vision model at 93 tok/s: Tesla V100 16 GB on Windows

More stories

  1. P100 (sm_60) patches for llama.cpp: +42% decode and +74% short-prefill on a 2×P100 tensor split — r/LocalLLM
  2. Unsloth's Llama 3.2 Mirror Hits 800K Downloads by Removing Meta's Login Wall — AlphaSignal
  3. 780m and Gemma 4 26b a4b — r/LocalLLM
  4. llama.cpp has come a long way — r/LocalLLaMA
  5. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  6. I made llama.cpp's slot saves write only the changed chunks: a FUSE filesystem that dedups KV-cache snapshots — r/LocalLLM
  7. Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console — Hacker News Front Page
  8. Fully local, private voice + chat AI assistant (zero cloud) — Qwen3.8-27B on 2× RTX PRO — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →