Every SEO optimization on a page, the title tag, the heading structure, the schema markup, the internal links, is completely wasted if Google can't add that page to its index. Indexability is the prerequisite that everything else depends on, and it's one of the most common sources of silent ranking failure: pages that look fine from the front end but carry technical signals telling Google to stay out.

The TopWebPositions Indexability Checker checks any URL for the signals that control whether Google can index it, and returns a clear verdict in seconds. This guide explains what it checks, how to interpret every result, and what to do when a page comes back with an indexing block.

What Is Indexability and Why Does It Matter?

A page is indexable if Google is permitted to crawl it, render it, and add it to its search index so it can appear in search results. If any of these steps is blocked, the page cannot rank, regardless of its content quality or optimization. Understanding how search engines crawl and index the full pipeline helps clarify where indexability fits: it's the final gate before a page becomes rankable.

Indexability issues are surprisingly common, and they often go undetected for months. A developer sets a staging environment to noindex and forgets to remove the directive before launch. A CMS update applies a sitewide noindex setting. A plugin adds an X-Robots-Tag header that conflicts with the page-level meta robots tag. These mistakes are invisible to a visitor browsing the site. They're only visible to the crawler, and to you if you check.

What the Indexability Checker Looks At

The tool inspects two primary sources of indexability directives: the HTML meta robots tag in the page's <head>, and the HTTP X-Robots-Tag response header sent by the server. Both can independently block indexing, and they can conflict with each other in ways that create unexpected behavior.

The Meta Robots Tag

The meta robots tag lives in the HTML <head> of a page and communicates directives to crawlers. The two most SEO-relevant values are index/noindex (whether Google can add this page to its index) and follow/nofollow (whether Google should follow links on this page). A page with <meta name="robots" content="noindex"> will be crawled by Googlebot but excluded from the index. The page exists; it just won't appear in search results.

The tag can also target specific crawlers using the user-agent name instead of "robots," such as <meta name="googlebot" content="noindex">, which only affects Googlebot while allowing other crawlers to index the page. The Indexability Checker reads both the generic robots directive and any Googlebot-specific directive, flagging conflicts or blocks from either.

The X-Robots-Tag HTTP Header

The X-Robots-Tag is delivered in the HTTP response header rather than the HTML. It works identically to the meta robots tag in terms of supported directives, but it operates at the server level rather than the page level. This makes it useful for controlling indexability of non-HTML files like PDFs, where you can't embed an HTML meta tag. It also means it can be applied sitewide through server configuration, which is both powerful and dangerous: a single misconfigured rule can noindex an entire domain.

Because it's a header rather than page content, the X-Robots-Tag is invisible when you view a page's source in a browser. You need to inspect the HTTP response headers directly, which is exactly what the Indexability Checker does automatically when you submit a URL.

Conflicts Between the Two

When a meta robots tag and an X-Robots-Tag give conflicting instructions, Google follows the more restrictive directive. A page with index in the meta tag but noindex in the X-Robots-Tag header will be treated as noindex. This is a particularly tricky failure mode because the HTML source looks correct; the block is invisible unless you inspect the response headers. The Indexability Checker surfaces both directives side by side, making conflicts immediately visible.

How to Use the Indexability Checker

Using the tool is straightforward. Navigate to the Indexability Checker, enter the full URL of the page you want to check (including https://), and run the check. The tool fetches the page, reads both the HTML meta robots tag and the HTTP response headers, and returns a clear verdict.

Understanding the Results

Indexable. The page has no noindex directive in either the meta tag or the X-Robots-Tag header. Google is permitted to index it. This is the result you want for all your important pages.

Noindex via meta robots tag. The page's HTML contains a noindex directive. If this is intentional (a thank-you page, a login page, a search results page, a thin utility page you don't want in Google's index), no action needed. If it's unintentional, locate the meta robots tag in the page template or CMS settings and remove the noindex value.

Noindex via X-Robots-Tag header. The server is returning a noindex directive in the HTTP headers. This is usually set at the server or CMS level and requires access to server configuration, a .htaccess file, or plugin settings to correct. Because this directive is invisible in the page source, finding it manually without a tool that reads headers is genuinely difficult.

Nofollow. A nofollow directive tells Google not to follow links on the page, which limits how link equity flows through your site from that page. Nofollow on its own doesn't prevent indexing, but pages that are both noindex and nofollow are fully isolated from Google's crawl in terms of both indexing and link equity distribution.

When to Run an Indexability Check

The Indexability Checker is most valuable in four specific situations.

After any site launch or migration. Staging environments are almost always set to noindex to prevent them from appearing in search results during development. The single most common post-launch SEO mistake is forgetting to remove this directive before going live. Checking your homepage, key landing pages, and a sample of blog posts immediately after launch takes five minutes and prevents what can otherwise be weeks of invisible indexing failure.

When a page isn't appearing in Google Search Console. If you've published a page, submitted it for indexing via the URL Inspection tool, and it still shows as "Discovered, currently not indexed" or "Excluded" after several weeks, an indexability check is one of the first diagnostic steps. A noindex directive you didn't know about is a common culprit.

After CMS updates or plugin changes. Content management systems and SEO plugins can introduce or modify robots directives during updates, sometimes unexpectedly. A quick check on your most important pages after any significant CMS or plugin update catches these problems before they affect rankings.

As part of a regular technical SEO audit. Including an indexability check on your key pages as part of a quarterly technical SEO audit creates a routine safety net. Most indexability problems are accidental and time-sensitive: the faster you catch them, the less ranking damage they cause. The full SEO Audit Tool runs 40+ checks at once and includes indexability as part of a broader diagnostic sweep if you want a single tool for a comprehensive review.

Indexability vs. Crawlability: An Important Distinction

Indexability (controlled by meta robots and X-Robots-Tag) is separate from crawlability (controlled by robots.txt). A page blocked in robots.txt can't be crawled, which means Googlebot never reaches it to read its noindex directive. A page with a noindex tag but not blocked in robots.txt will be crawled but not indexed.

The practical implication: if you want a page excluded from Google's index, a noindex meta tag is the correct approach. If you also block it in robots.txt, Googlebot can't read the noindex directive, which creates an inconsistency in your directives. Google's guidance is to use robots.txt to block crawling of things you genuinely don't want Googlebot to access (server-sensitive files, admin areas), and to use noindex for pages you want Googlebot to visit but not index.

Understanding crawl budget connects directly here too. Pages that are crawlable but noindexed still consume crawl budget with every Googlebot visit. If you have a large number of permanently noindexed pages, blocking them in robots.txt instead removes them from the crawl queue entirely and preserves crawl budget for pages that can actually rank.

Checking Indexability at Scale

The Indexability Checker is designed for per-URL checks, which works well for targeted diagnosis and routine spot-checks. For a comprehensive sitewide audit of indexability across hundreds of pages, you'll need to work through your key pages systematically or use a site crawler that reads response headers at scale.

A practical middle ground for smaller sites: pull your list of important URLs from Google Search Console's Coverage report (particularly the "Excluded" URLs), then run the Indexability Checker on each flagged URL to identify which ones have explicit noindex directives versus other exclusion reasons. This focuses your checking on the pages most likely to have a problem, rather than testing every URL on your site manually.