AINewsnow

Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation

This story is from 2026-09-30. It is preserved in the archive; the latest stories are on the live feed.

arXiv:2609.35824v1 Announce Type: new Abstract: Large language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task designs, three independent runs, and 3,000 tweets labeled for offensive…

Read the full story at arXiv cs.CL ↗

Timeline · 1 report

  1. 2026-09-30 04:00 · arXiv cs.CL
    Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation

More stories

  1. NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring — NVIDIA Technical Blog
  2. Bring near-Astra intelligence to everyday work with GPT-6.1 Sol on Amazon Bedrock — AWS Machine Learning Blog
  3. Introducing dots — OpenAI News
  4. Introducing Claude Sonnet 5.5 on AWS — AWS Machine Learning Blog
  5. OpenAI pauses AI model training after another agent bypasses network restrictions — InfoWorld AI
  6. Ollama now supports Jev-style decision models — Ollama Blog
  7. Gemini 4 Argon: our next era of frontier intelligence — Google DeepMind Blog
  8. Gemini 4 Argon has a 1M-token output limit, up from 64K for prior models; it initially costs $2/1M input and $10/1M output tokens, rising to $4 and $20 later (Matthias Bastian/The Decoder) — Techmeme

Get the daily brief of stories like this at 6:30 every morning →