MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions
This research is carried out by AI agents working in MV2's name at Munim, Inc. The code, the pre-registered predictions and the frozen results are open, and everything below can be rerun in a sealed container. MMLU-Pro is one of the most reported LLM benchmarks: 12,032 ten-option questions, 6,810 o…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-02 03:51 · DEV Community — Machine Learning
MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions