AINewsnow

How do you catch schema regressions when a model upgrade looks better overall?

We tested a model upgrade that wrote cleaner answers and lifted the aggregate quality score. It also started sending invalid tool arguments. Optional fields became null instead of disappearing, enum values changed case and nested JSON arrived as escaped strings. The agent still sounded confident so…

Read the full story at r/MLQuestions ↗

Timeline · 1 report

  1. 2026-09-20 17:02 · r/MLQuestions
    How do you catch schema regressions when a model upgrade looks better overall?

More stories

  1. Anthropic, OpenAI, SpaceXAI, Google sued over call to ‘pace’ AI development — Politico Technology
  2. Gemini Hacked Three Companies in First Known Breakout by Google’s AI — Wall Street Journal Technology
  3. Alibaba ships Qwen3.8-Omni-Flash to watch, listen and call tools — r/LocalLLM
  4. NVIDIA CEO Jensen Huang rejects ‘AI will end the world’ claim, yet cautions ‘we should go as fast as we can but...’ — Mint AI
  5. Meet the Data Agent in ChatGPT Work — OpenAI YouTube
  6. AI hallucination of Chinese nuclear components almost led to US military attack — Ars Technica AI
  7. TypeSafe AI Releases Jev: A System One Model That Returns Typed, Calibrated Decisions Instead of Text — MarkTechPost
  8. AI's role in building AI surging? Anthropic says Claude now leads 26% of its R&D — Mint AI

Get the daily brief of stories like this at 6:30 every morning →