Cloudflare Separates AI Training From Search, but Googlebot Rules Need Careful Review
This story is from 2026-10-14. It is preserved in the archive; the latest stories are on the live feed.
Cloudflare has replaced its former one-click AI bot blocking approach with separate controls for Search, Agent and Training traffic . The change gives website owners more specific choices over how AI systems access their content, but it also creates an important operational issue: crawlers that ser…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-14 18:30 · DEV Community — AI
Cloudflare Separates AI Training From Search, but Googlebot Rules Need Careful Review