AINewsnow

How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstats

This story is from 2026-09-30. It is preserved in the archive; the latest stories are on the live feed.

arXiv:2609.35815v1 Announce Type: new Abstract: Researchers across academia increasingly base significance claims on LLM judge scores and small-sample AI evaluations. Yet without well-calibrated confidence intervals (CIs), hypothesis tests, and judge-bias corrections, such claims are unreliable. We…

Read the full story at arXiv cs.CL ↗

Timeline · 1 report

  1. 2026-09-30 04:00 · arXiv cs.CL
    How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstats

More stories

  1. NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring — NVIDIA Technical Blog
  2. Bring near-Astra intelligence to everyday work with GPT-6.1 Sol on Amazon Bedrock — AWS Machine Learning Blog
  3. Introducing dots — OpenAI News
  4. Introducing Claude Sonnet 5.5 on AWS — AWS Machine Learning Blog
  5. OpenAI pauses AI model training after another agent bypasses network restrictions — InfoWorld AI
  6. Ollama now supports Jev-style decision models — Ollama Blog
  7. Gemini 4 Argon: our next era of frontier intelligence — Google DeepMind Blog
  8. Gemini 4 Argon has a 1M-token output limit, up from 64K for prior models; it initially costs $2/1M input and $10/1M output tokens, rising to $4 and $20 later (Matthias Bastian/The Decoder) — Techmeme

Get the daily brief of stories like this at 6:30 every morning →