How many requests that claim to be a known bot are real?

On four small websites we operate, 7,428 requests in the week of 19 to 25 October 2026 named one of 11 known bots in their user agent. 7,096 of them (95.5 percent) came from the operator's published IP ranges; 332 did not. One hosting network outside the lists claimed two or more bots: 287 of those 332 requests came from it, under 8 different bot names.

By the Crawlerpeek editorial team · Week 2026-W43, computed 2026-10-25 from our own server logs · first week of the study · How we check

Claimed versus verified, per bot

Week 2026-W43, 19 to 25 October 2026.

Only bots with a published, machine-readable IP list are in the table. Fewer than 20 requests in a week say little on their own.

Week by week

A new week is added every Monday for the week before. A partial week is shown, but the headline uses the newest complete one.

One network, many bot names

For every address outside the lists we look up the reverse-DNS name and group addresses by the network that name belongs to (approximately: the last two labels of the name). A network that sends requests under the names of two or more different bots, all outside those bots' own ranges, looks like one client rotating bot names. We publish only the counts, never the names or addresses.

In our first study, the week of 19 to 25 October 2026, 287 of the 332 outside requests (86 percent) came from one hosting network, whose names a forward lookup did not confirm. It claimed 8 different bots: Bingbot, CCBot, ChatGPT-User, ClaudeBot, Googlebot, GPTBot, OAI-SearchBot and PerplexityBot. Those names belong to six different operators, and every one of these requests sat outside the matching operator's published ranges. The remaining 45 outside requests had no reverse-DNS name.

That is why a user-agent allow list is weak protection: the name is free to copy, the address is not.

Method

1. A request claims a bot when its user agent contains the bot's token, case-insensitive, as in the user-agent lookup. The longest matching token wins, so Googlebot-Image counts as its own bot. The first study (week 43) folded the Google variants into Googlebot. In the trailing four-week figure, Googlebot therefore leaves out that first study as soon as a live week is in the window, so the figure never mixes the two ways of counting.
2. It counts as verified when the client IP address is inside the IP lists the operator publishes (all lists of that operator together): Google, OpenAI, Anthropic, Perplexity, Bing, Apple, Common Crawl, Ahrefs, DuckDuckGo, Amazon and a few smaller operators. The lists are fetched when the week is computed, on the Monday after it, so a week is judged with the lists of that Monday.
3. If a list cannot be fetched we use our last copy for up to 14 days and say so; without one the bot is left out of that week and the week is marked partial. A list that comes back empty or much smaller is treated as a failure.
4. We exclude requests from our own tools (the checker and the see-as-bot fetcher use bot user agents) and from our own test networks.
5. Data: server logs of four sites we operate. Requests that name a bot are archived and counted once a week. Only aggregates are published; this page and the downloads contain no IP addresses or network names.

Limits

• Four new and small sites. The numbers describe these sites, not the web, and weekly figures for single bots can be small.
• Published lists can lag behind new crawler addresses. Outside the list means unverified, not proven fake. A single network crawling under many names is a different matter.
• We have no list for every bot. Bots without a machine-readable list, such as MJ12bot or SemrushBot, are not in the table.
• The grouping of addresses by network is a simple rule without a public suffix list, so counts are approximate.
• A request that is verified proves the address, not the intent: it shows the operator's crawler made it.

Download the data

All weeks, free to use under CC BY 4.0 with a link to this page: per bot and week (CSV) · weekly totals and networks (CSV) · everything as JSON. Cite as: Crawlerpeek (2026). Fake bot requests: weekly verification study (Version 2026-W43) [Data set]. https://crawlerpeek.com/data/fake-bots

Check your own logs

Paste a user agent into the user-agent lookup to see which bot it names and how its operator says to verify it. The guide to fake Googlebots and PHP scanners has a script that compares log addresses with the published lists, and each bot page links the operator's own list. When an operator changes its user agent or documentation, the crawler changelog records it.

Related: AI bot traffic on our sites · open AI bot list · crawler changelog