Where does most of your GPU spend actually go: training, inference, or idle?
When people talk about ML compute costs, they almost always mean training: bigger runs, more GPUs, longer jobs. My guess is that a lot of teams have a different bill. Inference runs all day. And reserved GPUs sit idle between jobs because nobody wants to give up the capacity. I'm curious what it lo…
Read the full story at r/learnmachinelearning ↗
Timeline · 1 report
- 2026-10-01 21:52 · r/learnmachinelearning
Where does most of your GPU spend actually go: training, inference, or idle?