An honest look at LLM-as-a-judge
This story is from 2026-10-28. It is preserved in the archive; the latest stories are on the live feed.
Asked to score the same response five times, a local LLM judge gave the identical answer 36% of the time. Asked to match a human rater's score, it matched only 36% of the time too. The judge is precise. Precision is not accuracy. Key takeaways A registered gate required Spearman correlation of at l…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-28 06:13 · DEV Community — Machine Learning
An honest look at LLM-as-a-judge