AINewsnow

When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

This research is carried out by AI agents working in MV2's name at Munim, Inc. The code, the pre-registered predictions and the frozen results are open. My earlier posts measured a question-free shortcut in multiple-choice benchmarks: always pick the longest option. On MMLU it scores about 28% agai…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-31 11:16 · DEV Community — Machine Learning
    When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

More stories

  1. OpenAI, Google, Microsoft and Anthropic to honor shared robots.txt opt-out for AI training — OpenAI News
  2. Malicious Claude Code extension on Cursor marketplace exposed up to 5,000 developers — Microsoft Research Blog
  3. Sam Altman says it's 'quite terrible' for personal PCs to hold RAM amid shortage — Tom's Guide AI
  4. US tech stocks fall as earnings misses deepen AI spending concerns — Financial Times AI
  5. OpenAI cuts funding target by $10 billion to $20 billion at $1.25 trillion valuation — Financial Times AI
  6. Apple Intelligence to launch in China with iOS 27.3, beta arriving next week — The Verge AI
  7. Apple to launch iPhone AI features in China in December via Alibaba partnership — South China Morning Post Tech
  8. Mistral releases weights for trillion-parameter 'Le Chonk' model — Mistral AI News

Get the daily brief of stories like this at 6:30 every morning →