Study finds offline web behavior benchmarks inflated by candidate-set design
A new paper, IntentTwin, argues that offline web-behavior benchmarks that insert the demonstrated action into a finite candidate set and measure reranking conflate candidate support with true intent, creating a gap between oracle scores and deployment performance.
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-11-11 05:00 · arXiv cs.CL
IntentTwin: Deconfounding Candidate Support and the Oracle-to-Deployment Gap in Offline Web Behavior Prediction