AINewsnow

We tested 2,000 websites: 45% of GPTBot blockers also opt out of ChatGPT search answers

This story is from 2026-10-29. It is preserved in the archive; the latest stories are on the live feed.

Originally published on the CrawlBeacon blog . Short answer: In October 2026 we tested 2,000 popular websites with the same code behind CrawlBeacon's free AI Bot Readiness Checker . Of the 894 that returned a normal web page, 8.9% block at least one AI search bot in robots.txt , and 45% of sites th…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-29 14:23 · DEV Community — AI
    We tested 2,000 websites: 45% of GPTBot blockers also opt out of ChatGPT search answers

More stories

  1. 8 examples of Gemini 4 Argon in action — Google Gemini Blog
  2. ChatGPT Enterprise at Zendesk: Putting Company Data to Work — OpenAI YouTube
  3. The Key Numbers From Anthropic’s IPO Filing — New York Times Technology
  4. API credits, now in ChatGPT Pro and Business plans — OpenAI YouTube
  5. What is artificial general intelligence – and is it already here? — The Conversation AI (US)
  6. OpenAI and ServiceNow expand partnership across enterprise workflows — OpenAI News
  7. AI may quickly generate ideas, but working with people leads to more creative ones — The Conversation AI (US)
  8. Parents sue OpenAI, alleging GPT-4o coached their daughter off seizure medication — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →