AINewsnow

browsecomp-plus-mini: 15 of BrowseComp-Plus's 830 queries that rank deep-research agents roughly like the full set, no LLM judge

I needed a cheap way to compare search agents on BrowseComp-Plus without burning hours of agent time every run. So I built a 15-query subset of the full 830 that ranks systems roughly the way the full benchmark does. - weighted Spearman rank agreement 0.88 averaged over 30 splits, versus 0.75 for a…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-18 14:30 · r/LocalLLaMA
    browsecomp-plus-mini: 15 of BrowseComp-Plus's 830 queries that rank deep-research agents roughly like the full set, no LLM judge

More stories

  1. Anthropic to Shift Some Shares Into New ‘Public Good’ Fund — Bloomberg AI
  2. Anthropic proposes AI safety bill with high-risk thresholds and federal preemption — Politico Technology
  3. AI Gateway logs now show session affinity — Vercel Blog
  4. Consumer apprehension grows as US retailers adopt AI-assisted pricing tools — ABC News Technology
  5. DeepSeek releases MIT-licensed V4.1-Flash-1017 with improved agentic coding — Vercel Blog
  6. Anthropic gives third-party auditors ’employee-like access’ in AI industry first — ZDNET AI
  7. NVIDIA RTX Spark Laptops Launch Globally with Local AI Support — NVIDIA Technical Blog
  8. MOSS-Image gains traction with fast generation and ecosystem support — r/StableDiffusion

Get the daily brief of stories like this at 6:30 every morning →