23-point split: 6 models tied at 74% fell apart on 17 lines
23-point split: 6 models tied at 74% fell apart on 17 lines Six frontier models sit within 1.5 points of each other on DeepSWE, all clustered near 74%. Change a median of 17 lines in each task — 3.5% of the reference patch — and they fan out across 23.0 points, from 76.4% down to 53.4%. I re-ran Ma…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-08 01:25 · DEV Community — Machine Learning
23-point split: 6 models tied at 74% fell apart on 17 lines