AINewsnow

MaxxBench (RLVR'd LLMs fail easy variants of DeepSWE tasks + obsess over "hidden tests" and pleasing the grader)

Similar to Kimi K3.1 Pro Recalls and Enforces Hidden Test Rules Some of these drops are very big: DeepSeek V4 Flash falls from 53.3% to 28.1% on a lightly different version of Deep-SWE. (how different? "one DeepSWE task asks an agent to write a separate test report for each browser. We added a requ…

Read the full story at r/mlscaling ↗

Timeline · 1 report

  1. 2026-11-08 00:06 · r/mlscaling
    MaxxBench (RLVR'd LLMs fail easy variants of DeepSWE tasks + obsess over "hidden tests" and pleasing the grader)

More stories

  1. Moonshot Just Put a 2.8-Trillion-Parameter Model on Hugging Face — Kimi K3.1 Pro's Open Weights Are Real, and the Fine Print Matters — DEV Community — AI
  2. How Fusion works: DeepSeek V4 Pro plans, V4 Flash writes the code — DEV Community — AI
  3. Blind listening test: 62 people compared 8-bar piano loops written by 8 LLMs (Gemini, Claude, Qwen, Kimi, Grok, GPT-6.1, DeepSeek, Llama). Interim results — r/artificial
  4. DeepSeek V4.1 Flash gains traction across local rigs, benchmarks, and serving platforms — Vercel Blog
  5. Chinese AI lab Moonshot reportedly sought European data centers for Nvidia chip access — CNBC Technology
  6. China's AI models, led by Kimi, dominate open-source enterprise spending — South China Morning Post Tech
  7. Tencent launches Hy4 AI model with visual reasoning and finance benchmark lead — DigiTimes Asia
  8. Homelabber runs 2.8-trillion-parameter Kimi K3 on CPU alone using Optane memory sticks — Tom's Hardware

Get the daily brief of stories like this at 6:30 every morning →