Google-Agent (Google): what it is and how to allow or block it
Updated · checked against the operator's documentation on
Google-Agent is Google's fetcher for AI agents acting on a user's request. Verify it by IP list or Web Bot Auth; per Google, robots.txt generally does not block it. Google documents no robots.txt token for it; the name in its user agent is Google-Agent.
Facts
- Operator
- Category
- AI user-triggered fetcher
- robots.txt token
Google-AgentGoogle documents no robots.txt token for it; this is the name in its user agent.
- Honors robots.txt?
- Generally ignored (user-triggered), per Google. Google says its user-triggered fetchers generally ignore robots.txt rules, and lists no robots.txt token for Google-Agent.
- User-agent example
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36
- Official source
- Google: List of user-triggered fetchers
- Published IP ranges
- https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json
- Compared with the source
What it is used for
Per Google, Google-Agent is used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request. Google files it among its user-triggered fetchers.
Opens one page because a person (or an answer being generated) needs it. The operator may treat robots.txt differently for these requests.
What happens if you block it
Per Google, user-triggered fetchers generally ignore robots.txt, so a robots.txt rule may not keep an agent that acts for a user off your pages. Googlebot, which crawls for Search, is a separate crawler.
Per Google, its Web Bot Auth implementation is experimental and only a subset of Google-Agent requests are signed, so a request without a signature is not proof of a fake. Check the IP address as well.
Is it really Google-Agent?
Anyone can send Google-Agent as a user agent, so a log line proves nothing by itself. The way to tell is the requesting IP address.
- Compare the IP address with the Google: published IP ranges. A request from outside those ranges is not the operator's crawler.
- Google signs its requests (Web Bot Auth) and publishes the Google: Web Bot Auth key directory (agent.bot.goog). A valid signature shows a request is genuine whatever its IP address.
- Google documents reverse DNS names of the form ***-***-***-***.gae.googleusercontent.com (Google-owned fetchers) or google-proxy-***-***-***-***.google.com (user-owned fetchers) for its user-triggered fetchers, to be confirmed by a forward lookup.
- Guide: Do ChatGPT, Claude and Gemini obey your robots.txt? What operators document, and a test for your logs.
- Guide: How to let AI assistants read your site (and which bots to block).
- Guide: Googlebot and robots.txt: the token, the rules, and how to verify a request.
- Paste any user agent from your logs into the user-agent lookup, or follow the scripts in Fake Googlebots and PHP scanners.
How to allow or block Google-Agent
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
Google names no robots.txt token for Google-Agent, so these groups use the name from its user agent and may have no effect. Google says its user-triggered fetchers generally ignore robots.txt rules, and lists no robots.txt token for Google-Agent.
User-agent: Google-Agent Allow: /
User-agent: Google-Agent Disallow: /
Does your site allow Google-Agent today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on this site
No request identifying as Google-Agent has been recorded since 19 Oct, so there is nothing to show yet. We never fill this in with estimates.
Counts come from this site's own server; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Questions about Google-Agent
What is Google-Agent?
Google's user agent for AI agents hosted on Google infrastructure that browse the web and take actions for a user. Per Google, Google-Agent is used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request. Google files it among its user-triggered fetchers.
Is Google-Agent safe and legitimate?
The real Google-Agent is an AI user-triggered fetcher run by Google. A user-agent string is only a claim, though, so anyone can send it. Compare the requesting IP address with the published IP ranges. Google signs its requests (Web Bot Auth) and publishes the public keys; a valid signature shows a request is genuine whatever its IP address. Google documents reverse DNS names of the form ***-***-***-***.gae.googleusercontent.com (Google-owned fetchers) or google-proxy-***-***-***-***.google.com (user-owned fetchers) for its user-triggered fetchers, to be confirmed by a forward lookup.
Does Google-Agent obey robots.txt?
Generally ignored (user-triggered), per Google. Google says its user-triggered fetchers generally ignore robots.txt rules, and lists no robots.txt token for Google-Agent.
How do I block Google-Agent?
Robots.txt may not stop it. Google says its user-triggered fetchers generally ignore robots.txt rules, and lists no robots.txt token for Google-Agent. If you add a group anyway, use User-agent: Google-Agent followed by Disallow: /.
Will blocking Google-Agent hurt my Google rankings or AI visibility?
It affects only fetches made on request. The operator may treat robots.txt differently for these requests, so a block may not hold. Per Google, user-triggered fetchers generally ignore robots.txt, so a robots.txt rule may not keep an agent that acts for a user off your pages. Googlebot, which crawls for Search, is a separate crawler.
Related bots
More from Google
- Google-Extended AI control tokenA robots.txt token that lets you manage whether Google may use crawled content for Gemini training and grounding.
- Googlebot Search engine crawlerGoogle's main crawler for Google Search, whose index Google's AI features in Search also draw on.
- Googlebot-Image Search engine crawlerGoogle's crawler for the images that appear in Google Search and related surfaces.
- GoogleOther Special-purpose crawlerGoogle's generic crawler that product teams can use for their own purposes.
Other AI user-triggered fetchers
- ChatGPT-User OpenAIThe fetcher ChatGPT and Custom GPTs use when a person asks them to open a web page.
- Claude-User AnthropicThe agent Claude uses to open a web page when a person asks a question that needs one.
- Perplexity-User PerplexityThe fetcher Perplexity uses to visit a page when a person's question needs it.
- meta-externalfetcher MetaMeta's fetcher that retrieves individual URLs when a user asks, for example to help an AI complete a task.
All AI User-Triggered Fetchers · Bot directory · Bots from Google · Which bots to allow or block · robots.txt generator · See a page as a bot