AINewsnow

Evaluating DeepSeek models broke three assumptions my rubric was built on

This story is from 2026-10-15. It is preserved in the archive; the latest stories are on the live feed.

I started evaluating DeepSeek models the lazy way. The API speaks the same /v1/chat/completions shape as OpenAI, so I swapped the base_url , pointed my existing eval suite at R1, and waited for a score. I got one, and it told me almost nothing. That suite was written for a closed model. It scored t…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-15 16:09 · DEV Community — AI
    Evaluating DeepSeek models broke three assumptions my rubric was built on

More stories

  1. Western and Chinese AI search engines cite largely separate webs. Qwen's sources were 96% Chinese even with half the questions in English — r/artificial
  2. Five AI chatbots were tested on accuracy. The results weren’t good — The Independent Tech
  3. Mais de 240 mil linhas de código versionadas, 1 agente central, 2 IAs e 1 buscador no turno inteiro — publiquei a prova de runtime do Worion — r/PromptEngineering
  4. DeepSeek offers user refunds after suffering another major outage — South China Morning Post Tech
  5. Strata might be actually really detrimental for intelligence? or is it Swift and GSQ-RCO? or both? — r/LocalLLaMA
  6. I built Agent Praxis: 6 adversarial environments to test how LLM agents handle traps, metric gaming, and fake authority — r/LocalLLM
  7. DeepSeek implements fix after partial outage affects chat and API services — TechNode
  8. SoSBench Catches DeepSeek-R1 Leaking Dangerous Science at 84.9% — AlphaSignal

Get the daily brief of stories like this at 6:30 every morning →