Study proposes auditing codebooks to explain errors in LLM-assisted text annotation
A new arXiv paper argues that standard metrics like F1 fail to explain why LLM annotation errors occur in social science, and proposes auditing the codebooks guiding LLM-assisted annotation.
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-10 05:00 · arXiv cs.AI
When F1 Is Not Enough: Auditing Codebooks for LLM-Assisted Annotation