AINewsnow

The 2B model scored 10%. 93% of its answers were right

I expected small local models to be worse than a hosted decision API at Japanese classification. I did not expect most of the gap to be something other than judgement. On the hardest of three tasks one model scored 10.8% overall while getting 27 of the 29 answers it produced right. The other 221 ro…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-11-13 00:15 · DEV Community — AI
    The 2B model scored 10%. 93% of its answers were right

More stories

  1. Nvidia CEO Jensen Huang Resigns Amid Scandals — CSET Georgetown
  2. Google DeepMind unveils AlphaGenome 2 for cell-type-specific DNA variant predictions — Google DeepMind Blog
  3. Google launches Gemini 4 Hydrogen, a fast, lower-cost model for coding and agents — Google DeepMind Blog
  4. Anthropic IPO about six times oversubscribed as Nvidia pledge and CEO exit draw scrutiny — Reuters AI
  5. Google expands Gemini 4 Carbon access from cybersecurity defenders to Pro users and API — Google DeepMind Blog
  6. Anthropic expands Claude lineup with Fable 5.2, manufacturing tools, and Meetings beta — Anthropic (X)
  7. Google brings Gemini app to Apple CarPlay and unveils Gemini 4 Hydrogen model — Google AI Blog
  8. ElevenLabs launches Scribe v3 and Scribe v3 Live speech-to-text models — Anthropic (X)

Get the daily brief of stories like this at 6:30 every morning →