Ahrefs crawlers: what each bot does
Ahrefs runs 2 bots that we document: AhrefsBot, AhrefsSiteAudit. They have different jobs (seo tool crawler), so a robots.txt rule for one does not apply to the others.
| Bot | Category | robots.txt token | Honors robots.txt? | What blocking does |
|---|---|---|---|---|
| AhrefsBot | SEO tool crawler | AhrefsBot | Yes, per Ahrefs | Ahrefs stops recrawling your pages, so new links to and from your site stop appearing in its link and site data. AhrefsSiteAudit is a separate token and needs its own rule; blocking it stops your own audits in Ahrefs Site Audit. Google rankings are not affected. |
| AhrefsSiteAudit | SEO tool crawler | AhrefsSiteAudit | Yes by default, per Ahrefs | Site Audit crawls by Ahrefs users can no longer read your pages, unless you have verified the site and allowed it to ignore robots.txt. AhrefsBot has its own token. |
330 requests from Ahrefs bots on this site since 19 Oct. Bot traffic data