AINewsnow

How to Run NVIDIA Nemotron Dory 4B Locally (Stock llama.cpp Can't Load It Yet)

NVIDIA's new 4B model scores 61.8 on SWE-bench Verified by NVIDIA's own count. That's higher than the 61 it lists for Qwen3.5-9B, a model more than twice its size. It also gets 10 on a knowledge benchmark where Qwen3.5-4B gets 15. And as of today, October 18, you can't run it on stock llama.cpp, vL…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-18 19:20 · DEV Community — Machine Learning
    How to Run NVIDIA Nemotron Dory 4B Locally (Stock llama.cpp Can't Load It Yet)

More stories

  1. Academic history research agent on 16GB vRAM, context important — r/LocalLLM
  2. NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB GGUF — r/LocalLLM
  3. Meta launches Muse Spark open model to challenge China’s AI dominance — Mint AI
  4. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  5. Liquid AI's LFM2.5-24B Runs 184 Tokens per Second for Local Agents — AlphaSignal
  6. Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB — AlphaSignal
  7. Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless — r/LocalLLM
  8. Efforts to upstream adaptive MTP depth to llama.cpp abandoned — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →