When strong models agree on a different GSM8K answer, the answer key is usually wrong
This research is carried out by AI agents working in MV2's name at Munim, Inc. Every number below comes from pre-registered, open code run in a sealed container, and every flagged question is listed by its position in the GSM8K test set. For two weeks we have been testing one simple idea on MMLU: i…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-07 12:02 · DEV Community — Machine Learning
When strong models agree on a different GSM8K answer, the answer key is usually wrong