AINewsnow

A test that only checks the type passed 3 of my 5 planted bugs

This story is from 2026-10-23. It is preserved in the archive; the latest stories are on the live feed.

A test that only checks the type of a return value passed three of the five bugs I planted underneath it. A test whose only assertion was that an empty list comes back as an empty list passed four of five. Both count as success under the metric most agent benchmarks use, because both pass the refer…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-23 09:20 · DEV Community — AI
    A test that only checks the type passed 3 of my 5 planted bugs

More stories

  1. Chinese state media: DeepSeek and MiniMax completed registration and won approval to open-source and deploy AI models overseas under proposed export controls (Bloomberg) — Techmeme
  2. Canva launches Craft image model with 2K output and API — TestingCatalog AI News
  3. The Signal Chats That Cost OpenAI Researchers Their Jobs — Wall Street Journal Technology
  4. Anthropic says it partnered with Theorem to use Claude to formally prove security properties at the assembly level across 100+ real-world open-source libraries (Anthropic) — Techmeme
  5. Sources: Anthropic and London-based PE firm Hg agree to launch a consultancy by the end of 2026 to embed AI consultants and engineers into European businesses (Financial Times) — Techmeme
  6. Things will get ‘a lot worse before they get better’, experts warn amid AI security panic — The Independent Tech
  7. Anthropic Joins the A.I. Agent Race With Claude Duet — New York Times Technology
  8. Google's Agent Compute Engine gives every Gemini agent its own hardware-isolated micro virtual machine — The Decoder

Get the daily brief of stories like this at 6:30 every morning →