AhrefsBot (Ahrefs): what it is and how to allow or block it
AhrefsBot is the crawler of Ahrefs, a marketing intelligence platform; its data also powers the Yep search engine. Its robots.txt token is AhrefsBot.
Facts
- Operator
- Ahrefs
- Category
- SEO tool crawler
- robots.txt token
AhrefsBot
- Honors robots.txt?
- Yes, per Ahrefs. Ahrefs says its bots strictly respect robots.txt on both Disallow and Allow rules, and follow Crawl-delay for HTML pages (not when rendering JavaScript, so asset requests can look more frequent).
- User-agent example
Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)
- Official source
- Ahrefs: AhrefsBot
- Published IP ranges
- https://api.ahrefs.com/v3/public/crawler-ip-ranges
- Compared with the source
What it is used for
Powers the database for Ahrefs and Yep, an independent search engine. Crawls the web for a marketing or SEO tool, mostly to build a backlink or site database. It does not decide whether you rank, and blocking it only removes you from that tool's data.
What happens if you block it
Blocking it removes your site from Ahrefs' link and site data going forward. It does not affect Google rankings.
How to allow or block AhrefsBot
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
User-agent: AhrefsBot Allow: /
User-agent: AhrefsBot Disallow: /
Does your site allow AhrefsBot today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on our own sites
No request identifying as AhrefsBot has been recorded in the last 30 days, so there is nothing to show yet. We never fill this in with estimates.
Counts come from the websites we operate; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Related bots
Other SEO tool crawlers
- SemrushBot SemrushSemrush's crawler for data behind tools such as Backlink Analytics and Site Audit.
- MJ12bot MajesticMajestic's crawler that maps link relationships between websites for its search engine and backlink index.
All SEO Tool Crawlers · Bot directory · Which bots to allow or block · robots.txt generator · See a page as a bot