AINewsnow

Local inference advances: DeepSeek V4.1 Flash runs on single GPUs with streaming experts and faster serving

Community reports show DeepSeek V4.1 Flash running on one RTX 3090 or even a 12 GB RTX 3060 with system RAM via new engines, plus a 3x speedup over vLLM and SGLang on B200 via dynamic topology. Ollama vs llama.cpp gaps were traced to defaults, not the engine.

Read the full story at DEV Community — AI ↗

Timeline · 6 reports

  1. 2026-11-11 22:07 · r/machinelearningnews
    New Inference Optimization technique speeds DeepSeek-V4.1-Flash up 3x on B200 than vLLM and SGLang running on GB300
  2. 2026-11-10 10:19 · DEV Community — AI
    DeepSeek V4.1 Flash in DeepSeek Harness: GenUI praktisch einrichten und prüfen
  3. 2026-11-09 23:32 · r/LocalLLaMA
    Same prompt, DeepSeek V4.1 Flash vs Qwen 3.8 FN: 94 tok/s vs 350 tok/s
  4. 2026-11-09 19:49 · r/LocalLLM
    RustingStrata vs llama.cpp runs DeepSeek-V4.1-Flash Q2 (249 GB) on a single RTX 3060 12 GB with 30 GB RAM
  5. 2026-11-09 13:47 · r/LocalLLM
    DeepSeek V4.1 Flash Unsloth IQ2 at 42 tok/s on one RTX 3090 + 192 GB RAM: Delta Inference Engine 0.94 (native Windows)
  6. 2026-11-09 05:54 · DEV Community — AI
    Ollama vs. llama.cpp on the Same GGUF: The 15% Speed Gap Was the Defaults, Not the Engine

More stories

  1. Comparing 7 Chinese frontier models through one API — a field guide, not a benchmark — DEV Community — AI
  2. How our open source coding agent scored 84.5% on Terminal-Bench 2.1 for $27 — DEV Community — AI
  3. Can't Top Up a DeepSeek API Key From Your Country? Every Workaround, Compared Honestly — DEV Community — AI
  4. DeepSeek Harness' new Creator Mode creates plugins from prompts — Mashable AI
  5. I Can't Code, So I Built a System That Stops an AI From Lying — DEV Community — Machine Learning
  6. Run Gemma 4 31B-it Uncensored on a 12GB GPU: Complete Local Vision & 256K Context Guide with llama.cpp, DeepSeek Harness & Hermes Agent — DEV Community — Machine Learning
  7. Ripple-65B Preview Benchmark: Gauntlet, Forge 2.0, LFM3-1.7B — DEV Community — AI
  8. New ADIndex Method Adds Layer-Adaptive Sparse Attention Indexing via Group-Level Distillation — arXiv cs.LG

Get the daily brief of stories like this at 6:30 every morning →