Stefano Ermon: Autoregressive inference is sequential and memory-bound. Diffusion is built to map to GPUs — that's why it wins.
TL;DR: Augment Code swapped its production coding-agent backend in September. It didn't switch to a bigger model. It switched to a smaller one running on an architecture that doesn't generate tokens one at a time — Stefano Ermon's Mercury 2.5 — and the numbers aren't close: latency down 82%, cost d…
Read the full story at r/artificial ↗