AINewsnow

NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB GGUF

hi dudes and duderinas, Has anyone run this model on 32GB VRAM on two RTX with 16G with llama.cpp 0.7.0-dev build 11671 and can tell me its parameters? Or is it not working at the moment? I've got the error: segmentation fault With LM Studio's Llama it works.

Read the full story at r/LocalLLM ↗

Timeline · 2 reports

  1. 2026-10-16 12:09 · r/LocalLLM
    Has anyone tried this Bolt 74B-A9B 24GB on a 5090 yet?
  2. 2026-10-16 07:39 · r/LocalLLM
    NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB GGUF

More stories

  1. How to Run NVIDIA Nemotron Dory 4B Locally (Stock llama.cpp Can't Load It Yet) — DEV Community — Machine Learning
  2. Academic history research agent on 16GB vRAM, context important — r/LocalLLM
  3. Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming — r/LocalLLM
  4. Liquid AI's LFM2.5-24B Runs 184 Tokens per Second for Local Agents — AlphaSignal
  5. Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB — AlphaSignal
  6. Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless — r/LocalLLM
  7. Efforts to upstream adaptive MTP depth to llama.cpp abandoned — r/LocalLLM
  8. New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →