AINewsnow

RustingStrata vs llama.cpp runs DeepSeek-V4.1-Flash Q2 (249 GB) on a single RTX 3060 12 GB with 30 GB RAM

RustingStrata is a pure-Rust GGUF inference engine with CUDA and Vulkan backends. It now loads DeepSeek-V4.1-Flash at IQ2_XXS (249 GB) and runs it on a 12 GB card with 30 GB of RAM. The experts stream from SSD through the page cache. A 9 GB LRU cache in VRAM holds the experts that get reused. Every…

Read the full story at r/LocalLLM ↗

Timeline · 5 reports

  1. 2026-11-12 04:25 · DEV Community — Machine Learning
    Evaluating DeepSeek-V4.1-Flash-1017 on a 60-Page PDF: What Text Extraction Misses
  2. 2026-11-11 22:07 · r/machinelearningnews
    New Inference Optimization technique speeds DeepSeek-V4.1-Flash up 3x on B200 than vLLM and SGLang running on GB300
  3. 2026-11-10 10:19 · DEV Community — AI
    DeepSeek V4.1 Flash in DeepSeek Harness: GenUI praktisch einrichten und prüfen
  4. 2026-11-09 23:32 · r/LocalLLaMA
    Same prompt, DeepSeek V4.1 Flash vs Qwen 3.8 FN: 94 tok/s vs 350 tok/s
  5. 2026-11-09 19:49 · r/LocalLLM
    RustingStrata vs llama.cpp runs DeepSeek-V4.1-Flash Q2 (249 GB) on a single RTX 3060 12 GB with 30 GB RAM

More stories

  1. Run Gemma 4 31B-it Uncensored on a 12GB GPU: Complete Local Vision & 256K Context Guide with llama.cpp, DeepSeek Harness & Hermes Agent — DEV Community — Machine Learning
  2. Comparing 7 Chinese frontier models through one API — a field guide, not a benchmark — DEV Community — AI
  3. Kinetic: a new open-source LLM engine in Rust. Runs your GGUFs on Windows and Linux, replaces llama-server with the same flags — r/LocalLLM
  4. the latest Nvidia benchmark — r/ChatGPT
  5. Llama App - Windows’ta ücretsiz indir ve yükle — Microsoft Research Blog
  6. Alibaba's T-Head plans Hong Kong IPO amid Nvidia saga, testing China's AI ambitions — South China Morning Post Tech
  7. Paying for DeepSeek API outside China is harder than using it — DEV Community — AI
  8. GPT-6 Astra edits a small model’s weights, pointing to AI debugging — DEV Community — Machine Learning

Get the daily brief of stories like this at 6:30 every morning →