AINewsnow

Alibaba measured it: the best coding agent meets 70% of requirements and finishes 26% of tasks

The Decoder ran a story on Saturday about a benchmark called Co-Design Bench , and the two headline numbers look contradictory until you read the method. The strongest of ten model endpoints meets 69.9% of a task's acceptance criteria on average, and fully completes 26.4% of tasks. Both numbers des…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-11-09 07:35 · DEV Community — AI
    Alibaba measured it: the best coding agent meets 70% of requirements and finishes 26% of tasks

More stories

  1. Nous Research releases Arete-27B, cutting Qwen3.8-27B propaganda rate from 22% to 0.9% — AlphaSignal
  2. Nvidia CEO's alleged affair with Chinese spy spotlights governance at $4.5 trillion chipmaker — Reuters AI
  3. CyberTiel35BA3B working on Blender Rigging via MCP. 8GB VRAM, 16GB RAM. llama.cpp. — r/LocalLLaMA
  4. Boogu 0.1 in ComfyUI: Three Open Models in One Release, Tested Against Qwen-Class Baselines — DEV Community — AI
  5. AgentScope Basics: Building Your First Agentic Application — DEV Community — AI
  6. A New AI Video Model for Christmas 🎅 — r/StableDiffusion
  7. MIT CSAIL: GPT-6 Astra edited 0.016% of a small Qwen model's weights and lifted MATH-500 from 7.9% to 48.1% -- critics say it's standard ML engineering — r/artificial
  8. Apple researchers use a Mac mini running Qwen locally to control a Unitree G1 humanoid robot. Trained on 1,000 simulated homes, it autonomously sits, crawls under tables and navigates unfamiliar rooms, achieving 91% success across 360 real-world trials. — r/singularity

Get the daily brief of stories like this at 6:30 every morning →