Google-Extended (Google): what it is and how to allow or block it
Google-Extended is a robots.txt token that lets you manage whether Google may use crawled content for Gemini training and grounding. Its robots.txt token is Google-Extended.
Facts
- Operator
- Category
- AI control token
- robots.txt token
Google-Extended
- Honors robots.txt?
- Yes (control token). It has no separate HTTP user-agent string, so it never shows up in server logs; crawling still happens as Googlebot.
- User-agent example
- None: this token never appears in server logs
- Official source
- Google: Common crawlers
- Compared with the source
What it is used for
A standalone product token publishers can use to manage whether content Google crawls may be used for future Gemini models and for grounding. Not a crawler you will see in your logs. A token in robots.txt that tells the operator how already collected content may be used.
What happens if you block it
Google states that Google-Extended does not impact a site's inclusion in Google Search.
How to allow or block Google-Extended
Add one of these groups to the robots.txt at the root of your domain. A group that names the bot overrides a general User-agent: * group.
User-agent: Google-Extended Allow: /
User-agent: Google-Extended Disallow: /
Does your site allow Google-Extended today? The checker evaluates your robots.txt for this and every other bot.Check my site
What we see on our own sites
No request identifying as Google-Extended has been recorded in the last 30 days, so there is nothing to show yet. We never fill this in with estimates.
Counts come from the websites we operate; they hold no IP addresses or URLs, and a user agent is only a claim. See the bot traffic data and the methodology.
Related bots
More from Google
Other AI control tokens
All AI Control Tokens · Bot directory · Bots from Google · Which bots to allow or block · robots.txt generator · See a page as a bot