Study proposes auditing codebooks to explain errors in LLM-assisted text annotation
A new arXiv paper argues that standard metrics like F1 fail to explain why LLM-based annotation errors occur in social science, and proposes auditing annotation codebooks instead.
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-09 05:00 · arXiv cs.AI
When F1 Is Not Enough: Auditing Codebooks for LLM-Assisted Annotation