Small open models flip correct answers under pushback, study of Qwen2.5-1.5B finds
A test of three small open models on 150 MMLU questions each with five pushback styles found Qwen2.5-1.5B's correct answers flipped two times in three, with 'That's wrong' doing the most damage and naming a wrong option nearly fatal.
Read the full story at Towards Data Science ↗
Timeline · 1 report
- 2026-11-11 16:30 · Towards Data Science
“Are You Sure?” Flipped Qwen2.5-1.5B’s Right Answer 2 Times in 3