Crawlerpeek
Check a site

Excluded by ‘noindex’ tag (URL marked ‘noindex’): where the noindex comes from and how to remove it

Google saw a noindex rule, in a robots meta tag or an X-Robots-Tag header, and left the page out. Remove the rule if the page should be in Google.

The message in Search Console

URL marked ‘noindex’

Page indexing report, group "Not indexed". Spelling as in Google's help page for the report.

Excluded by ‘noindex’ tag

The same reason as the report table and URL Inspection in Search Console label it.

By the Crawlerpeek editorial team · Sources checked · Updated · Editorial standards

What Google says

When Google tried to index the page it encountered a 'noindex' directive and therefore did not index it. If you do not want this page indexed, congratulations! If you do want this page to be indexed, you should remove the 'noindex' directive.

Source: Google Search Console Help: Page indexing report, read 2026-10-30.

Common causes

  • A robots meta tag with noindex in the page's HTML, set by a template, a page setting or a search-visibility option in the CMS.
  • An X-Robots-Tag: noindex HTTP header, set in the server, framework or CDN configuration. It is not visible in the page source, only in the response headers (Block Search indexing with noindex).
  • A staging setting that went live with the site.
  • A noindex in the original HTML that JavaScript removes later. Google says that when it meets the noindex tag it may skip rendering and JavaScript execution, so changing or removing the tag with JavaScript may not work as expected (JavaScript SEO basics).

How to check

  1. Enter the URL in the AI crawler checker or see a page as a bot: the "Indexing" row reports noindex from a meta tag or a header, in the HTML the server sends before scripts run.
  2. Check the headers directly: curl -sI https://example.com/page | grep -i x-robots-tag.
  3. In URL Inspection, "Indexing allowed?" shows the reason when indexing is disallowed (URL Inspection tool). Use "Test live URL" after your change, then request indexing or click Validate fix.

When to fix it and when to leave it

Leave it for pages you meant to keep out of search, such as thank-you pages, internal search results, account pages or test pages. That is what Google's "congratulations!" is about. Keep those pages crawlable: if robots.txt blocks them, Google never sees the noindex, which is how Indexed, though blocked by robots.txt happens.

AI crawlers and AI search

Google lists noindex among the controls that limit what appears in Search, and a page that is not indexed cannot be a supporting link in AI Overviews or AI Mode (AI features and your website). noindex is an indexing rule, not a crawl block: Google says noindexed pages are still requested (crawl budget guide). To keep a specific AI crawler from fetching pages, robots.txt groups for its token are the documented tool; see block AI crawlers.

Questions

I removed the noindex. Why does Search Console still show it?

The report changes after Google recrawls the page. Google says it can take months for Googlebot to revisit a page, depending on its importance; ask for a recrawl with URL Inspection to speed it up.

Can I use robots.txt instead of noindex?

Not to keep a page out of Google. robots.txt stops crawling, and Google can still index a blocked URL from links. noindex keeps it out, as long as Google may crawl the page and see the rule.

Where can noindex be set?

In a robots meta tag in the HTML head, or in an X-Robots-Tag HTTP response header. The header works for files that are not HTML too, such as PDFs.

Related messages

Sources

Google's documentation, each page read on 2026-10-30. Google updates these pages; if its wording has changed since, Google's page is the reference.

Check the URL: robots.txt per bot, HTTP status, noindex and the HTML a crawler gets.Run the checker