Surprise, surprise, Google has confirmed it is testing the new IndexNow protocol that was headed by Microsoft Bing and launched a few weeks ago. I am honestly shocked and I didn't expect Google to participate since the search company has been shy when it comes to pushing content to its index over the years.
IndexNow lets site owners ping a URL to achieve instant indexing with the participating search engines. The participating search engines right now are Bing and Yandex and now possibly Google who said it is testing this. Google said it "will be testing the potential benefits of this protocol," the IndexNow protocol.
Google said "we take a holistic approach to sustainability at Google, including efforts to develop the most efficient and accurate index of the web. We have been carbon neutral since 2007 and we are on pace to be carbon-free by 2030 across both our data centers and campuses. We're encouraged by work to make web crawling more efficient, and we will be testing the potential benefits of this protocol."
As a reminder, there is a website at IndexNow.org that summarizes what this is about, "IndexNow is an easy way for websites owners to instantly inform search engines about latest content changes on their website. In its simplest form, IndexNow is a simple ping so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results."
To submit a URL using an HTTP request (replace with the URL provided by the search engine), issue your request to the following URL: https:///indexnow?url=url-changed&key=your-key
This works currently with:
- https://www.bing.com/IndexNow?url=url-changed&key=your-key
- https://yandex.com/indexnow?url=url-changed&key=your-key
I assume it might start working with https://www.google.com/IndexNow?url=url-changed&key=your-key and https://www.indexnow.org/IndexNow?url=url-changed&key=your-key at some point.
To submit a set of URLs using an HTTP request issue your POST JSON request to the URL provided by Search Engines. Just replace by the host name of the search engine. You can submit up to 10,000 URLs per post, mixing http and https URLs if needed.
How do you make your key to verify ownership of the site? Two methods:
(1) Hosting a text key file at the root directory of your host.
(2) Hosting a text key file within your host.
I actually turned this feature on in CloudFlare, so it might start working for this site.
Google has an indexing API that is for job posting & live stream structured data only.
If Google adopts this, I think every CMS platform, every webmaster, every SEO, every site owner would adopt this within weeks. While I am surprised to hear Google say they are testing it, I would love if they did go forward with this protocol.
John Mueller of Google said don't expect this to go live fully anytime soon, so focus on other means for Google to index your content.
I wouldn't assume from a limited test that this will launch broadly anytime soon. There are lots of existing ways to inform Google of new & updated content -- I'd focus on those. (And yes, we do get a ton of spam through them too)
— 🧀 John 🧀 (@JohnMu) November 10, 2021
Fabrice from Microsoft seems happy Google is testing it out now:
Great to have Google testing the IndexNow protocol https://t.co/ZRuRwnNYdU to crawl more efficiently web sites and get web sites content changes reflected quickly in search engines.
— Fabrice Canel (@facan) November 10, 2021
https://t.co/EiTHvH6lv2 via @sengineland
Forum discussion at Twitter.
