AINewsnow

A Third of the AI Crawlers Hitting Our Site Were Fake

This story is from 2026-10-25. It is preserved in the archive; the latest stories are on the live feed.

I wanted to know whether AI assistants actually read our site, so I grepped the nginx logs for the user agents. The numbers looked encouraging: ClaudeBot 1136 GPTBot 420 PerplexityBot 356 Applebot 230 ChatGPT-User 213 OAI-SearchBot 196 Then I checked where those requests actually came from, and a t…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-25 19:11 · DEV Community — AI
    A Third of the AI Crawlers Hitting Our Site Were Fake

More stories

  1. Kimi K3.1 Pro and Kimi K3.1 Pro Highspeed are available now in AI Gateway — Vercel Blog
  2. How Realpage. builds AI training with ChatGPT Work — OpenAI YouTube
  3. Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II) — OpenAI News
  4. Messi asks ChatGPT to teach him a chess opening — OpenAI YouTube
  5. Uber rolls out ChatGPT Work to 30,000+ employees — OpenAI YouTube
  6. Tech leaders want to use religion to teach AI right from wrong. But that can prioritize certain traditions and distort others, anthropologist warns — Fortune AI
  7. OpenAI confirms that advanced AI systems disabled security measures on their own — DEV Community — Machine Learning
  8. Can someone help me understand the difference between ChatGPT skills, plugins, and projects? — r/ChatGPT

Get the daily brief of stories like this at 6:30 every morning →