AINewsnow

SWE-Pair: a coding-agent benchmark where every task is two real bugs to fix at once, with results from four models [P]

We built a benchmark of 161 tasks from 69 Python projects where the agent has to fix two real bugs at once. The cheapest setup came out on top. DeepSeek V4.1 Flash at medium effort solved 90% of tasks, at about 6 cents per solved task. GPT-6 Luna at max effort solved 78%, at about 23 cents. More re…

Read the full story at r/MachineLearning ↗

Timeline · 1 report

  1. 2026-11-10 18:33 · r/MachineLearning
    SWE-Pair: a coding-agent benchmark where every task is two real bugs to fix at once, with results from four models [P]

More stories

  1. One Week, Three Public Numbers: 73.68 Open-Weight #1 in Math Reasoning, 290 of 290 Actions Denied, 3.15M On-Device Downloads — DEV Community — AI
  2. DeepSeek's Harness agent app reaches 1.54 million daily users — Bloomberg AI
  3. Local AI inference gains ground as compact desktop hardware and new Mac tooling emerge — DEV Community — Machine Learning
  4. DeepSeek V4.1 Flash draws wave of benchmarks, cost tests and local-run experiments — DEV Community — AI
  5. How Often Do AI Labs Ship New Models? Release Cadence of OpenAI, Anthropic, Google, DeepSeek and Mistral (2024-2026) — DEV Community — AI
  6. GPT 6 Instant is Terrible — r/ChatGPT
  7. How I Cut My LLM Bill by Routing Off-Peak Traffic to Discounted DeepSeek — DEV Community — AI
  8. CXMT chairman reportedly invested in DeepSeek's second funding round — DigiTimes Asia

Get the daily brief of stories like this at 6:30 every morning →