Exa crawlers: what each bot does
Exa runs 2 bots that we document: ExaSearchBot, ExaUser. They have different jobs (AI search crawler, AI user-triggered fetcher), so a robots.txt rule for one does not apply to the others.
| Bot | Category | robots.txt token | Honors robots.txt? | What blocking does |
|---|---|---|---|---|
| ExaSearchBot | AI search crawler | ExaSearchBot | Yes, per Exa | Your pages are left out of Exa's index and are not returned by search and retrieval applications built on it. |
| ExaUser | AI user-triggered fetcher | ExaUser | May not apply, per Exa | Exa says ExaUser does not decide whether your content appears in Exa search, so a rule for it does not remove you from search. To opt out of search and automatic crawling, Exa points to ExaSearchBot. |
Is a request really from Exa?
A user-agent string is only a claim. These are the checks Exa publishes for its bots; where there is none, the user agent alone cannot prove a request is genuine.
- ExaSearchBot: no IP list published that we found (compared 2026-10-25).
- ExaUser: no IP list published that we found (compared 2026-11-09).
Paste an address into the crawler IP checker to test it against the lists we can read.
What Exa says about robots.txt
ExaSearchBot
Exa says it respects the Robots Exclusion Protocol, does not bypass logins, paywalls or CAPTCHAs, and limits its request rate per site. It states no numeric crawl delay. To remove already indexed pages, Exa points to noindex meta tags or the X-Robots-Tag header, which take effect after a re-fetch.
ExaUser
Exa says that because the fetches are user-initiated, robots.txt rules may not apply to them. To opt out of search inclusion and automatic crawling it names the ExaSearchBot token, not ExaUser.
2,568 requests from Exa bots on this site in the last 30 days. Bot traffic data