Can your website be found in Google?
The best website in the world is invisible if Google may not show it. Four small things on the site decide: noindex, robots.txt, the sitemap and the canonical. The checker reads all four.
How the checker sees whether Google is let in
The page does not say no to Google: no noindex, no block in robots.txt, a sitemap exists and the canonical points at your own domain.
The page may be indexed, but something is missing: no sitemap, or a canonical pointing at another domain. Google finds the page, but more slowly or wrongly.
The page says no to Google: a noindex instruction or a robots.txt that blocks the whole site. Google may not show the page, however good it is.
This check carries 15 of 100 points. The checker sees whether Google is allowed to index the page, not whether it already is. The latter you see in Google Search Console.
A page Google may not show does not exist
Google can only show pages it is allowed to read. A noindex line or a robots.txt that blocks everything is an active no, and Google obeys. I have written more about how Google finds and indexes your site.
The sitemap is the list of your pages. Without it, Google has to feel its way link by link, and a subpage nobody links to can take months to find, or never be found.
The canonical says which address is the original. If it points at another domain, you send Google's trust there, and your own address becomes a copy in Google's eyes.
How to open the door for Google
Open the page source and search for noindex. If you find a meta tag with robots or googlebot and the word noindex, remove it or change it to index. In WordPress the setting is often called Discourage search engines from indexing this site, under Settings and Reading.
Open your-address/robots.txt in the browser. If it says Disallow: / under User-agent: *, the file blocks the whole site. Remove the line or limit it to the folders that should be private.
Create a sitemap if you have none. Most tools and hosts do it automatically, the address is usually your-address/sitemap.xml. Submit it in Google Search Console. Also point to it with the line Sitemap: https://your-address/sitemap.xml in robots.txt, that is the line the checker looks for.
Check the canonical in the page head. It should point at the page's own address on your own domain. Leftovers from an old host or an old domain are one of the things ongoing maintenance catches before they cost you.
Run the checker again. The row should be green.
When I hand over a site, noindex is gone, the sitemap is submitted in Search Console and the canonical is checked, in the Start package as well as in a one-off build. See what it costs.
Noindex and robots.txt in the real world
The most common case is a site built with noindex switched on so it would not show during the build. Then it went live and nobody removed the line. The site works, looks good and is not in Google at all. The owner thinks the competition is just tough.
The checker reads four things. The robots meta tag and the X-Robots-Tag header on the front page, robots.txt at the root of the site, whether a sitemap exists, and where the canonical points. If either of the first two says no, the row turns red; if the sitemap is missing or the canonical points wrong, it turns orange. The check carries 15 of 100 points, the most of any, because without indexing the rest does not matter.
Another case is the move. The site changed host or domain, the new address works, but the canonical in the code still points at the old one. Google then credits an address that may not even exist any more.
Questions about indexing
My site is new, why is it not in Google yet?
Google has to find the site first, and that can take from a few days to a few weeks. The fastest route is to submit the sitemap in Google Search Console. If the row is green in the checker, only time is missing.
Can I keep Google out of a single page without hiding the whole site?
Yes. Put noindex only in that page's head, for example on a thank-you page or an internal page. The checker looks at the address you enter, usually the front page, so a noindex there is always a problem, but on a subpage it can be exactly right.
Does a small site need a sitemap?
It can manage without one, if every page can be reached by links from the front page. But the sitemap costs nothing, most tools make it for you, and it is the simplest way to tell Google that a page is new or changed.
Free site check
Run the check and see whether Google is let in
Enter your web address and I'll check the page right away: secure connection, mobile-friendliness, load time, visibility on Google, and contact paths. You get a score from 0 to 100 and see exactly where it falls short. Free, and no account needed.
✓ Want the full review? I'll go through the site in depth and come back with a plan, free and with no obligation.
✓ Thanks! The full review will land in your inbox shortly.
I personally look at every site that's submitted. Albin
The other checks
The checker looks at ten things. Each has its own page.