SEO tool crawlers: AhrefsBot, SemrushBot, MJ12bot
SEO tool crawlers collect link and site data for marketing platforms. They do not influence Google rankings, and blocking them only removes your site from those tools' databases.
Crawls the web for a marketing or SEO tool, mostly to build a backlink or site database. It does not decide whether you rank, and blocking it only removes you from that tool's data.
3 bots in this group
| Bot | Owner | robots.txt token | Honors robots.txt? | Requests on our sites (30 days) |
|---|---|---|---|---|
| AhrefsBot | Ahrefs | AhrefsBot | Yes, per Ahrefs | none yet |
| SemrushBot | Semrush | SemrushBot | Yes, per Semrush | none yet |
| MJ12bot | Majestic | MJ12bot | Yes, per Majestic | none yet |
No request from this group has been recorded on our sites in the last 30 days.
Block the whole group
Paste into robots.txt to ask every bot in this group to stay away. Only do this if you mean it: read what blocking changes first.
User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: /