AINewsnow

A Guardrail That Cuts Attack Success 10x Can Still Be the Wrong Guardrail: Reading AgentDojo Numbers Like an Engineer

This story is from 2026-10-20. It is preserved in the archive; the latest stories are on the live feed.

TL;DR Meta's LlamaFirewall paper reports that on AgentDojo static traces, attack success rate (ASR) drops from 17.63% (no guardrail) to 7.53% (PromptGuard 2 alone), 2.89% (AlignmentCheck alone) and 1.75% (both). Task utility drops from 47.73% to 47.01% , 43.09% and 42.68% . An ASR number is half of…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-20 14:23 · DEV Community — Machine Learning
    A Guardrail That Cuts Attack Success 10x Can Still Be the Wrong Guardrail: Reading AgentDojo Numbers Like an Engineer

More stories

  1. DeepSeek releases DeepSeek-V4.1-Flash-1017 open weights under the MIT license — r/huggingface
  2. GenRecs Tokenization: Transforming Ads Recommendation at Meta with Stable Model Vocabulary — Engineering at Meta
  3. The Thin Orchestrator: How We Made Our AI Agents More Reliable by Giving the Router Less to Do — Engineering at Meta
  4. RBC Weighs Blocking Meta’s AI Agent Muse, CEO McKay Says — Bloomberg AI
  5. Meta Muse exposes users' personal and business secrets when they share conversations — Ars Technica AI
  6. Google just open-sourced TorchTPU, a native PyTorch backend for TPUs (52% MFU on Qwen 3 8B) — r/machinelearningnews
  7. vLLM PagedAttention Memory Layout: Why Our 70B Deployment Stopped OOMing — DEV Community — Machine Learning
  8. A World Model of Life: Biohub's Alex Rives on Virtual Cells, Measurement, & the $1.8B Data Push — The Cognitive Revolution

Get the daily brief of stories like this at 6:30 every morning →