How to detect and verify ClaudeBot, Claude-User and Claude-SearchBot

Which three bots does Anthropic run?

Source: Anthropic's Help Center article above, read 26 October 2026. The article says the bots honor robots.txt directives and will not bypass CAPTCHAs.

How do I find them in my access log?

Search for the three names, case-insensitive. This works on the combined log format used by default in nginx and Apache. Behind a CDN or proxy the first field is the proxy's address unless you log the client IP.

# requests per bot
grep -ioE 'claudebot|claude-searchbot|claude-user' access.log | tr 'A-Z' 'a-z' | sort | uniq -c

# client addresses that used a Claude name
grep -iE 'claudebot|claude-searchbot|claude-user' access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head

Anthropic's article does not publish user-agent strings. In our own logs of four small sites (19 to 25 October 2026), ClaudeBot requests carried strings such as Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com), and Claude-User requests carried ... compatible; Claude-User/1.0; +claude-user@anthropic.com). That is what we observed, not something Anthropic documents, so match on the token and not on the full string. We saw no Claude-SearchBot requests in that week.

How do I check an IP address against Anthropic's list?

Anthropic says that if a crawler's source IP appears in its published list, the crawler is coming from Anthropic. The list is a JSON file with a prefixes array of CIDR ranges (ipv4Prefix entries on 26 October 2026; the script also handles ipv6Prefix in case Anthropic adds them). The article names one list for its crawlers and does not say which range belongs to which bot, so a match shows "from Anthropic", not "this exact bot".

# verify_anthropic.py: is this IP address in Anthropic's published crawler list?
import ipaddress, json, sys, urllib.request

URL = "https://claude.com/crawling/bots.json"
ip = ipaddress.ip_address(sys.argv[1])
req = urllib.request.Request(URL, headers={"User-Agent": "verify-crawler"})
prefixes = json.load(urllib.request.urlopen(req))["prefixes"]
nets = [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix")) for p in prefixes]
print("in list" if any(ip in n for n in nets if n.version == ip.version) else "NOT in list")

Run python3 verify_anthropic.py 203.0.113.7 with an address from your log. Fetch the list again from time to time; it carries a creationTime field (2026-10-20 in the copy we read). Or enter the address in the crawler IP checker, which runs the same comparison.

What does a request outside the list mean?

It did not come from an address Anthropic lists. In our 19 to 25 October 2026 data, 758 requests named ClaudeBot. 10 of them were our own tool or tests and are left out; of the other 748, 719 came from the list and 29 did not. The three Claude-User requests were all in the list (method and limits). The 29 outside requests are not proven fake: a list can lag behind new addresses. A host that sends several different crawler names from addresses that belong to none of the operators is a strong sign of imitation; how that looks in logs. Do not block an address range from Anthropic's list: Anthropic says blocking its IP addresses may not work correctly or persistently guarantee an opt-out, because it stops the bots reading your robots.txt.

Which robots.txt directives do they read?

Anthropic's article documents two: Disallow and the non-standard Crawl-delay. Its own example names only ClaudeBot. For the other two bots the article says its bots honor robots.txt directives, and it names their tokens; it does not discuss them separately, so we use the same syntax.

# Opt out of training (Anthropic's own example)
User-agent: ClaudeBot
Disallow: /

# Or slow ClaudeBot down instead of blocking it (use one group, not both)
# User-agent: ClaudeBot
# Crawl-delay: 1

How groups are matched decides which rule a crawler follows. The three tokens are separate. A group for ClaudeBot does not cover Claude-SearchBot or Claude-User. Anthropic says to place the rule in the robots.txt of the top-level directory of every subdomain you want to opt out. Anthropic's article gives no figure for how long changes take.

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

The example above blocks training and leaves search and user requests open. Blocking Claude-SearchBot or Claude-User can reduce your visibility in Claude's answers, per Anthropic; see which bots to allow. Whether Claude fetches made on a user's request follow robots.txt is something to test on your own site: what each operator documents.

Checked 26 October 2026.