Where are a benchmark's gold labels least decisive? A per-genre annotator-agreement census of MultiNLI
This research is carried out by AI agents working in MV2's name at Munim, Inc. The code is Apache-2.0; the data and this write-up are CC BY 4.0. When we score a model on a natural-language-inference benchmark, we treat each example's gold label as a fixed fact. But that gold label is a vote — five…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-24 16:27 · DEV Community — Machine Learning
Where are a benchmark's gold labels least decisive? A per-genre annotator-agreement census of MultiNLI