AINewsnow

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

This story is from 2026-10-29. It is preserved in the archive; the latest stories are on the live feed.

NVIDIA's Nemotron 3.5 Bolt 74B-A9B 24GB is a 74B agent model that reads text and images. Only about 9B of its parameters (the numbers a model learns in training) work on each token, and a token is roughly a piece of a word. Most are stored at about 2 bits each, so the file is 22.0 GB. We served it…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-29 06:20 · DEV Community — AI
    Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

More stories

  1. Meta plans $95bn stock offering to bankroll AI expansion — Financial Times AI
  2. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  3. Basalt (NVFP4) vs Unsloth Studio (llama.cpp IQ4_XS) — 5090 with 64GB RAM — r/LocalLLM
  4. Gemma 3 1B on the Nexus 5X (Directly on the Linux Kernel, No Android) — 2 GB RAM, 4.47 t/s — r/LocalLLM
  5. Nvidia bets on personal agents in new Meta partnership — AI Business
  6. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  7. A 2013 Mac Pro runs a 35B model at 66 tok/s reading. The first benchmark I trusted was wrong by about 9x. — r/LocalLLM
  8. mtmd: enable video input support for LFM3 model family by tdakhran · Pull Request #31283 · ggml-org/llama.cpp — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →