Why I made my eval tool refuse to give a score
This story is from 2026-09-03. It is preserved in the archive; the latest stories are on the live feed.
Most eval tools always hand you a number. Mine sometimes refuses — and I think that refusal is the more useful behavior. Here's why. The vacuous green muteval reports a mutation score: of the regressions I injected into your system, what percent did your eval suite catch? Naively that's killed / to…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-09-03 18:27 · DEV Community — AI
Why I made my eval tool refuse to give a score