Three Green Reruns Prove Almost Nothing: Sizing a Flake-Detection Budget for Agent Patches
A merge gate that reruns a failing test three times is not testing for flakes. It is testing whether the flake happens to be loud. Run the arithmetic before you argue about it. At a 2% per-run failure rate, three reruns catch the flake about 5.9% of the time. The other 94% of the time the gate prin…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-06 08:40 · DEV Community — AI
Three Green Reruns Prove Almost Nothing: Sizing a Flake-Detection Budget for Agent Patches