Amazon crawlers: what each bot does
Amazon runs 3 bots that we document: Amazonbot, Amzn-SearchBot, Amzn-User. They have different jobs (ai training crawler, ai search crawler, ai user-triggered fetcher), so a robots.txt rule for one does not apply to the others.
| Bot | Category | robots.txt token | Honors robots.txt? | What blocking does |
|---|---|---|---|---|
| Amazonbot | AI training crawler | Amazonbot | Yes, per Amazon | Your pages are no longer used to improve Amazon products and services or, possibly, to train Amazon AI models; Amazon says changes can take about 24 hours. |
| Amzn-SearchBot | AI search crawler | Amzn-SearchBot | Yes, per Amazon | Per Amazon, allowing it makes your content eligible to appear in Amazon search experiences such as Alexa, so a block gives that up. Amazonbot, whose data may be used to train Amazon AI models, has its own token. |
| Amzn-User | AI user-triggered fetcher | Amzn-User | May not follow all (user-initiated), per Amazon | Per Amazon, Amzn-User may not follow all robots.txt directives because its actions can be initiated by a user, so a block may not stop every request. Amazon's search crawler is Amzn-SearchBot, with its own token. |
Stay findable, opt out of training
If you want Amazon's search and fetch bots to keep access but its training bot and control tokens to stay away, this is the robots.txt for it. It is one option, not a recommendation.
User-agent: Amzn-SearchBot Allow: / User-agent: Amzn-User Allow: / User-agent: Amazonbot Disallow: /
213 requests from Amazon bots on this site since 19 Oct. Bot traffic data