ExaSearchBot (Exa): what it is and how to allow or block it
Updated · checked against the operator's documentation on
ExaSearchBot is Exa's crawler that discovers and indexes public pages for its search and retrieval service. Its robots.txt token is ExaSearchBot.
Facts
- Operator
- Exa
- Category
- AI search crawler
- robots.txt token
ExaSearchBot
- Honors robots.txt?
- Yes, per Exa. Exa says it respects the Robots Exclusion Protocol, does not bypass logins, paywalls or CAPTCHAs, and limits its request rate per site. It states no numeric crawl delay. To remove already indexed pages, Exa points to noindex meta tags or the X-Robots-Tag header, which take effect after a re-fetch.
- User-agent example
Mozilla/5.0 (compatible; ExaSearchBot/1.0; +https://crawler.exa.ai/)
- User agent we logged
Mozilla/5.0 (compatible; ExaSearchBot/1.0; +https://crawler.exa.ai/)Most common variant in our own logs, 172 requests, 19 to 25 Oct 2026. Static snapshot, not the live counter below. A logged string is a claim: anyone can send it.
- Official source
- Exa: ExaSearchBot
- Compared with the source
What it is used for
Exa says it discovers and refreshes pages on the public web for search and retrieval, with the goal of connecting people and applications to your content and sending them back to your site.
Builds the index an AI assistant searches when it answers. Blocking it can keep you out of that assistant's answers.
What happens if you block it
Your pages are left out of Exa's index and are not returned by search and retrieval applications built on it.
Is it really ExaSearchBot?
Anyone can send ExaSearchBot as a user agent, so a log line proves nothing by itself. The way to tell is the requesting IP address.
- Exa signs its requests (Web Bot Auth) and publishes the Exa: signature key directory. A valid signature shows a request is genuine whatever its IP address.
- Paste any user agent from your logs into the user-agent lookup, or follow the scripts in Fake Googlebots and PHP scanners.
How to allow or block ExaSearchBot
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
User-agent: ExaSearchBot Allow: /
User-agent: ExaSearchBot Disallow: /
Does your site allow ExaSearchBot today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on our own sites
Live counter, last 30 days: 8 requests on 2 days identified as ExaSearchBot (latest: ). That is a small sample: read it as an example, not as a measurement of the web.
- Content pages: 6
- Legal pages: 2
Counts come from the websites we operate; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Questions about ExaSearchBot
What is ExaSearchBot?
Exa's crawler that discovers and indexes public pages for its search and retrieval service. Exa says it discovers and refreshes pages on the public web for search and retrieval, with the goal of connecting people and applications to your content and sending them back to your site.
Is ExaSearchBot safe and legitimate?
The real ExaSearchBot is an AI search crawler run by Exa. A user-agent string is only a claim, though, so anyone can send it. Exa signs its requests (Web Bot Auth) and publishes the public keys; a valid signature shows a request is genuine whatever its IP address.
Does ExaSearchBot obey robots.txt?
Yes, per Exa. Exa says it respects the Robots Exclusion Protocol, does not bypass logins, paywalls or CAPTCHAs, and limits its request rate per site. It states no numeric crawl delay. To remove already indexed pages, Exa points to noindex meta tags or the X-Robots-Tag header, which take effect after a re-fetch.
How do I block ExaSearchBot?
Add a group to the robots.txt at the root of your domain: User-agent: ExaSearchBot followed by Disallow: /. A group that names the bot overrides a general User-agent: * group.
Will blocking ExaSearchBot hurt my Google rankings or AI visibility?
It can. This bot builds the index an AI assistant searches when it answers, so blocking it can keep you out of that assistant's answers. Your pages are left out of Exa's index and are not returned by search and retrieval applications built on it.
Related bots
Other AI search crawlers
- OAI-SearchBot OpenAIOpenAI's crawler that surfaces websites in ChatGPT's search features.
- Claude-SearchBot AnthropicAnthropic's crawler that analyzes online content to improve the quality of Claude's search results.
- PerplexityBot PerplexityPerplexity's crawler that surfaces and links websites in Perplexity search results.
- ShapBot Parallel Web SystemsParallel's crawler that discovers and indexes websites for Parallel's web APIs.
All AI Search Crawlers · Bot directory · Which bots to allow or block · robots.txt generator · See a page as a bot