Cheapest per token is not cheapest per finished job: what 2,400 runs show
A model with a 33% pass rate turned out to be the cheapest way to finish a task in a recent 10-model benchmark. The same benchmark shows why that result is less of a free lunch than it looks. Here is the metric behind it and how to compute it on your own prompts. The metric Cost per successful task…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-05 00:47 · DEV Community — AI
Cheapest per token is not cheapest per finished job: what 2,400 runs show