Artificial Analysis updates hallucination benchmark to catch models guessing instead of abstaining

Artificial Analysis refreshed its AA-Omniscience benchmark to version 1.1, cutting it to 3,000 questions and exposing which AI models were quietly overfitting by guessing rather than abstaining.

Read the full story at AlphaSignal ↗

Timeline · 1 report

2026-10-31 00:01 · AlphaSignal
Artificial Analysis' AA-Omniscience v1.1 Catches AI Models Guessing Instead of Abstaining

More stories

Utopai X rises to No. 2 on video leaderboards with new image-to-video support — AlphaSignal
Mistral releases weights for trillion-parameter 'Le Chonk' model — Mistral AI News
Inside a malicious Claude Code-branded extension and its persistent SSH backdoor — Microsoft Research Blog
Anthropic files for blockbuster IPO as Blackstone flags likely junk rating — Fast Company AI
OpenAI cuts funding target by $10bn amid tech anxieties — Financial Times AI
Google unveils Gemini 4 Carbon, a cybersecurity-focused frontier model — Google DeepMind Blog
OpenAI, Google, Microsoft and Anthropic to honor shared robots.txt opt-out for AI training — OpenAI News
S&P cuts Meta's credit outlook to negative on AI spending concerns — Bloomberg AI

Get the daily brief of stories like this at 6:30 every morning →