AGORA method uses item-response theory to aggregate LLM judges with reliability weighting
A new arXiv paper introduces AGORA, a method that aggregates multiple LLM judges using item-response theory to account for each judge's reliability, addressing individual judges' biases in LLM-as-a-judge evaluation.
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-05 05:00 · arXiv cs.AI
AGORA: Reliability-Aware Aggregation of LLM Judges via Item-Response Theory