NVIDIA's Dory 1.0 4B (looped reasoning model) on Apple Silicon with MLX: 101 tok/s at 1 loop, 69 at 2, on an M5 Max (4.2 GB 8-bit build)
NVIDIA released Nemotron-Labs Dory 1.0 4B on 16 October. It's a looped model: a shared block of layers runs 1 to 4 times per token, so one set of weights gives you a speed-versus-accuracy dial. I ported it to MLX so it runs natively on Macs, and I've published 8-bit (4.2 GB, recommended) and bf16 (…
Read the full story at r/LocalLLM ↗