Noindex checker

What's the difference between indexable and indexed?

Indexable: The Google bot can crawl a page, this means:

1. it isn't blocked by robots.txt
2. nor by meta tags (it has no <meta name="robots" content="noindex, nofollow"> tag)
3. http headers don't forbid it (it has no X-Robots-Tag: noindex, nofollow tag)
4. url is available (= no 404)
5. google bot is not blocked by any anti-crawling techniques (cloudflare et. al.)

Indexed: Google did index the page. Even if the webpage is indexable it doesn't need automatically that it gets indexed.

Why isn't my indexable page showing up in Google search?

There are many reasons really:

1. No incoming links: You might say "yes but it's in sitemap.xml" - yes but this often is not enough. Google prefers having someone link to it.
2. Limited crawl budget: Google's crawlers have finite resources. For newer or less authoritative sites, they might only index a few hundred pages.
3. Fresh content: If your page is new, Google might not have gotten around to indexing it yet. This process can take anywhere from a few hours to several weeks.

My social media page is not indexable - help!

I found out that it's quite tricky to get LinkedIn, Facebook or Instagram get indexed by Google. I wrote a howto here: Social media backlinks

How can I speed up the indexing?

• If you own the page, then you can use Google Search Console to submit the page for indexing.
• If you don't own the page, all you can do is to link from one of your indexed pages and trigger the indexing in the Google Search Console.

I submitted my product to a directory. How do I check if will be indexed by Google?

When you submit a backlink to a directory, it's crucial to ensure it has internal linking. Here's some steps to check:

1. Look for sections like "Newest Products" or "Recently Added." These often link to new submissions.
2. Try to find your listing by clicking through the directory's categories. If you can reach your page just by clicking, that's a good sign.
3. Check for a sitemap.xml file (usually at domain.com/sitemap.xml). This file lists all pages the site wants search engines to crawl. Check if your page is in it.
4. Consider using tools like Screaming Frog or Sitebulb to crawl the directory to understand the internal linking structure.

There is no internal linking to my submitted backlink - what now?

1. Limited value: Without internal linking, the domain itself doesn't pass on much SEO value to your backlink page. Yes it will transmit the Domain Rank, but not the Page Rank. And this is more important. Usually these backlinks have limited SEO value.
2. Link to the site: You still profit from the Domain Rank of the referring domain. In order to make Google find the backlink, link from your site. Consider linking it from multliple sites (e.g. from your social media profiles).

I reviewed hundreds of directories and found that many at first "good backlinks" are in the end not linked internally. That's why I created a list of impactful backlinks.