Behind Aggregate Scores: What Systematically Varying Arena-Hard's LLM Judge Reveals
arXiv:2610.30512v1 Announce Type: new Abstract: LLM judges are used to determine which of two answers is better. Trusting these verdicts requires knowing how repeatable they are and how much they depend on the judge's instructions. We show that similar aggregate scores can conceal substantial chang…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-24 04:00 · arXiv cs.CL
Behind Aggregate Scores: What Systematically Varying Arena-Hard's LLM Judge Reveals