Mediapartners-Google (Google): what it is and how to allow or block it
Updated · checked against the operator's documentation on
Mediapartners-Google is Google's AdSense crawler that matches ads to pages. Verify it by reverse DNS or Google's IP list; it skips User-agent: *, so block it by name. Its robots.txt token is Mediapartners-Google.
Facts
- Operator
- Category
- Special-purpose crawler
- robots.txt token
Mediapartners-Google
- Honors robots.txt?
- Yes, when named (ignores *), per Google. Google says Mediapartners-Google ignores the global User-agent: * group, so only a group that names it applies.
- User-agent example
Mediapartners-Google
- Official source
- Google: List of special-case crawlers
- Published IP ranges
- https://developers.google.com/static/crawling/ipranges/special-crawlers.json
- Compared with the source
What it is used for
Per Google, Mediapartners-Google visits participating sites in order to provide them with relevant ads, and rules addressed to it affect ad-related Google products such as Google AdSense and Google Ad Manager.
Not the crawler behind a search index. What blocking it changes is specific to its tool, so read the next section.
What happens if you block it
Per Google, rules addressed to Mediapartners-Google affect ad-related products such as AdSense and Ad Manager on your pages. Googlebot, which crawls for Search, is a separate crawler.
On desktop the user agent is just Mediapartners-Google. Google documents the mobile variant only by its end, (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html), preceded by various mobile device types.
Is it really Mediapartners-Google?
Anyone can send Mediapartners-Google as a user agent, so a log line proves nothing by itself. The way to tell is the requesting IP address.
- Compare the IP address with the Google: published IP ranges. A request from outside those ranges is not the operator's crawler.
- Google documents reverse DNS names of the form rate-limited-proxy-***-***-***-***.google.com for its special-case crawlers, to be confirmed by a forward lookup.
- Guide: Googlebot and robots.txt: the token, the rules, and how to verify a request.
- Guide: Fake Googlebots and PHP scanners: what hit our new server in 48 hours.
- Guide: robots.txt syntax: groups, wildcards and matching rules.
- Paste any user agent from your logs into the user-agent lookup, or follow the scripts in Fake Googlebots and PHP scanners.
How to allow or block Mediapartners-Google
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
User-agent: Mediapartners-Google Allow: /
User-agent: Mediapartners-Google Disallow: /
Does your site allow Mediapartners-Google today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on this site
No request identifying as Mediapartners-Google has been recorded since 19 Oct, so there is nothing to show yet. We never fill this in with estimates.
Counts come from this site's own server; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Questions about Mediapartners-Google
What is Mediapartners-Google?
Google's crawler that visits sites showing Google ads, to match relevant ads to their pages. Per Google, Mediapartners-Google visits participating sites in order to provide them with relevant ads, and rules addressed to it affect ad-related Google products such as Google AdSense and Google Ad Manager.
Is Mediapartners-Google safe and legitimate?
The real Mediapartners-Google is a special-purpose crawler run by Google. A user-agent string is only a claim, though, so anyone can send it. Compare the requesting IP address with the published IP ranges. Google documents reverse DNS names of the form rate-limited-proxy-***-***-***-***.google.com for its special-case crawlers, to be confirmed by a forward lookup.
Does Mediapartners-Google obey robots.txt?
Yes, when named (ignores *), per Google. Google says Mediapartners-Google ignores the global User-agent: * group, so only a group that names it applies.
How do I block Mediapartners-Google?
Add a group to the robots.txt at the root of your domain: User-agent: Mediapartners-Google followed by Disallow: /. A group that names the bot overrides a general User-agent: * group.
Will blocking Mediapartners-Google hurt my Google rankings or AI visibility?
This is not the crawler behind a search index, so blocking it is not a ranking decision. Per Google, rules addressed to Mediapartners-Google affect ad-related products such as AdSense and Ad Manager on your pages. Googlebot, which crawls for Search, is a separate crawler.
Related bots
More from Google
- Google-Extended AI control tokenA robots.txt token that lets you manage whether Google may use crawled content for Gemini training and grounding.
- Googlebot Search engine crawlerGoogle's main crawler for Google Search, whose index Google's AI features in Search also draw on.
- Googlebot-Image Search engine crawlerGoogle's crawler for the images that appear in Google Search and related surfaces.
- GoogleOther Special-purpose crawlerGoogle's generic crawler that product teams can use for their own purposes.
Other Special-purpose crawlers
- archive.org_bot Internet ArchiveReported to be the crawler the Internet Archive uses for Wayback Machine and Archive-It captures.
- WebMCPIndexBot WebMCP ShieldA survey crawler that measures how widely websites adopt WebMCP.
- AgentTrustBot agenttru.stA crawler that looks for published AI agent discovery documents on newly seen hostnames.
- OAI-AdsBot OpenAIOpenAI's fetcher that checks web pages submitted as ads on ChatGPT.
All Special-Purpose Crawlers · Bot directory · Bots from Google · Which bots to allow or block · robots.txt generator · See a page as a bot