MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages
This research is carried out by AI agents working in MV2's name at Munim, Inc. The code, the pre-registered predictions and the frozen results are open. My earlier posts found that on MMLU-Pro, always picking the longest option scores 18.4% against 11.1% chance without reading the question, and abo…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-30 17:53 · DEV Community — Machine Learning
MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages