Google Search Discovers New Registered Domains For Indexing

Google's Gary Illyes confirmed that Google does look at domain name records for discovery of new domains to include in the Google Search Index. He said this at the 9:40 mark into yesterday's Search Off The Record podcast.

Gary said, "we are looking at domain records for discovery but that also just means that we would discover the domain name." So if you launch a new domain, since 2005, when Google became a registrar (they didn't start selling domains until 2014), Google will know when about new domain records.

Google wouldn't necessarily know about subdomains or subfolders, Gary said "well, depending how the DNS is set up." Meaning, sometimes Google can pick it up from the CNAME records but often not. Gary wrote:

So for example if you are broadcasting in the DNS records the CNAMEs that you set up for... the CNAMEs are a special record for... in DNS which allows you to map custom authority bits to IP addresses. So you could do subdomain.example.com maps to example.com and then the server would know the authority bit is subdomain.example.com, then serve one specific resource on the server and you can set up your DNS to advertise these CNAME records and then we might be able to discover them.

But if the DNS is set up, I would say properly, then these are not advertised. So basically, you just get the A record, which is the main record for a domain name, basically maps the domain name to the server's IP address. And that's how our domain names typically are set up. Basically, just the A record, when you are asking for the domain record, not all the other subdomains as well.

Since 2004, the SEO community thought Google did find new domains to index through this method anyway. So this is not really new, it is about 20 years old. :)

Here is the embed of the podcast, so you can listen to the full podcast:

Gary updated me with this:

Good news! You're in my daily roundup!

Just wanted to reiterate that no one should ever rely on this. We can only ever discover one URL with this method and it's way, way more likely we'd crawl that one URL if you'd seeded it through Search Console for example. so just don't

— Gary 鯨理／경리 Illyes (@methode) April 6, 2023

Forum discussion at Twitter.

Note: This was pre-written and scheduled to be posted today, I am currently offline for Passover.