Study Finds Incomplete Relevance Judgments Distort Abstention Evaluation in Conversational RAG
A new paper argues that benchmarks like MTRAG and RETECO, where systems answer from their own retrieval, produce distorted abstention evaluations when relevance judgments are incomplete.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-02 05:00 · arXiv cs.CL
Better Retrievers, Worse Labels: How Incomplete Relevance Judgments Distort Abstention Evaluation in Conversational RAG