AINewsnow

Swapping LLM judges can change system rankings, preregistered study finds

A preregistered arXiv study examines cross-provider replication in LLM-judged system comparisons, where practitioners without human labels re-run evaluations using a judge from a different provider to check conclusions.

Read the full story at arXiv cs.CL ↗

Timeline · 1 report

  1. 2026-11-04 05:00 · arXiv cs.CL
    Swapping the Judge Changes the Ranking: A Preregistered Study of Cross-Provider Replication in LLM-Judged System Comparisons

More stories

  1. Postman's Claude Code and Codex installs spark security debate over silent self-upgrades — Runway Research
  2. Google to cut off search access for schools that ban AI use — Forbes AI
  3. Democratic Midterm Wins Cloud Outlook for AI Data Center Expansion — DigiTimes Asia
  4. Anthropic reports 1,100% revenue surge and $895B compute commitments ahead of IPO — Reuters AI
  5. AI inference startup Fal raises $1.3B at $16B valuation in Bessemer-led round — Bloomberg AI
  6. Tencent launches Hy4 open-weight AI model with vision and computer-use capabilities — SCMP China Tech
  7. Musk says Grok Bot costs 'unsustainably high' as usage limits suddenly max out — Forbes AI
  8. Israeli startup 42AI emerges from stealth with $62.5M seed for behavioral world model — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →