AINewsnow

Your exact_match metric's normalization options can both invent matches and hide real ones

This story is from 2026-10-18. It is preserved in the archive; the latest stories are on the live feed.

HuggingFace evaluate 's exact_match is one of the most-used metrics in LLM and QA evals. What's easy to miss is that it ships with four normalization knobs -- regexes_to_ignore , ignore_case , ignore_punctuation , ignore_numbers -- and each quietly changes which answer pairs count as equal before t…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-18 13:47 · DEV Community — Machine Learning
    Your exact_match metric's normalization options can both invent matches and hide real ones

More stories

  1. Flux.3 Image Klein is now out — r/LocalLLaMA
  2. Hugging Face details 2.8 trillion token Indic pretraining pipeline — Hugging Face Blog
  3. Hugging Face releases Beam, a 501B parameter open model — Hugging Face Blog
  4. Granite Switch 4.1 offers chat, decisions, and speech in one endpoint — Hugging Face Blog
  5. I built an open-source tool on top of Hugging Face TRL: GRPO, DPO or SFT from one YAML file — r/huggingface
  6. Moss - 8 step turbo lora — r/StableDiffusion
  7. Qwen 3.8 27B finetune did the lava lamp — r/LocalLLM
  8. Mellum2 model from JetBrains — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →