Epoch Finds Half of 21 Major AI Benchmarks Are Flawed

Epoch AI expanded its benchmark audit program, flagging popular evals like SWE-bench Verified and Humanity's Last Exam as Flawed under a public rubric.

Read the full story at AlphaSignal ↗

Timeline · 1 report

2026-10-23 18:47 · AlphaSignal
Epoch Finds Half of 21 Major AI Benchmarks Are Flawed

More stories

Kimi K3.1 Pro and Kimi K3.1 Pro Highspeed are available now in AI Gateway — Vercel Blog
Securing our frontier model training environment — OpenAI News
Anthropic Joins the A.I. Agent Race With Claude Duet — New York Times Technology
Stop guessing auto scaling targets for TTS on Amazon SageMaker AI — AWS Machine Learning Blog
At MIT, Martha Minow speaks on sustaining democracy, both by using and overcoming AI — MIT News AI
Q&A: How AI came to model the “messy” human mind — MIT News AI
LitPivot: Developing research ideas through dynamic contextualization within the literature landscape — Allen Institute for AI (Ai2)
Why Google CEO Sundar Pichai thinks asking 'how much code AI writes' is the wrong question — Mint AI

Get the daily brief of stories like this at 6:30 every morning →