Probably the biggest news for the SEO industry out of Google I/O yesterday was that Google said Googlebot is now evergreen, meaning it will stay up-to-date with the latest version of Chromium, their popular Chrome browser. This brings thousands of new features to Googlebot for crawling purposes. I did cover this last night at Search Engine Land and this news shouldn't come as a surprise to anyone - we saw Google testing this just a few weeks ago.
In any event, Google said "Today, we are happy to announce that Googlebot now runs the latest Chromium rendering engine (74 at the time of this post) when rendering pages for Search. Moving forward, Googlebot will regularly update its rendering engine to ensure support for latest web platform features."
Googlebot Supports New Features
Googlebot now supports 1,000+ new features they said, including but not limited to ES6 and newer JavaScript features, IntersectionObserver for lazy-loading and Web Components v1 APIs. But it still can't do everything, Google made a page for JavaScript issues with search that you have to be aware of.
Here is a snapshot of some of those new features:
Our session on #javascript and #seo at 1:30pm tomorrow will be LIT🔥🔥🔥
— Martin Splitt @ 🇺🇲 #io19 (@g33konaut) May 9, 2019
Come to stage 3 to see what it's all about 🙃
Amongst other things, we'll talk:
🏗️Web app architectures
🤖Googlebot
⚡️Lazy loading
🚀Web APIs
🔬Testing
📈Crawl budget & more!https://t.co/6CGsiGgg9w pic.twitter.com/b4MnvpSmWt
It's Still Not Perfect
Also, Google still needs to do two waves of crawling to render the JavaScript. GoogleBot will crawl your page first, and then once again to fully render the JavaScript said Martin Splitt. He said there is still going to be a delay and Google won't be able to crawl and index it all on one pass:
There is still a delay...but more than 1000 new features are supported now!
— Martin Splitt @ 🇺🇲 #io19 (@g33konaut) May 7, 2019
Biggest News At I/O
Ilya Grigorik from Google said this news "takes the cake" from all the other announcements at Google I/O. Why? See what he said on Twitter:
First day of #io19 is a wrap. So many awesome highlights, but if I had to choose just one amongst many..
— Ilya Grigorik (@igrigorik) May 8, 2019
Googlebot running latest Chromium and now also on the evergreen track (blog post @ https://t.co/3sCrOQZpon) takes the cake. Not only is it a huge win for web developers... pic.twitter.com/0CyuVxwTrD
and, a huge kudos to all the folks behind the scenes, both on the Chrome and Search side, that made it all possible. A long and multi-year journey that required deep architectural work in Chromium, layers upon layers of optimizations (scheduling, rendering, ...), and...
— Ilya Grigorik (@igrigorik) May 8, 2019
tl;dr: a huge upgrade and step forward for the web; web developers rejoice; and we all benefit as well.
— Ilya Grigorik (@igrigorik) May 8, 2019
Pretty good start for #io19. :-)
Does Not Seem Live Yet
I went through my log files, I still only see Chrome 41 for GoogleBot across a ton of my different sites and log files. Others say they don't see it as well. I suspect we will see it soon.
Strange - I cannot find any Chrome 74 UA requests in my logs - only coming from Chrome 75.0
— Jan-Peter Ruhso (@JanRuhso) May 8, 2019
Update: Marin Splitt from Google said it is live but the useragent was not updated yet:
Yup. We think it's better to give people who might have a hard-coded the user agent a bit more of a heads up. Expect a blog post on this...
— Martin Splitt @ 🇺🇲 #io19 (@g33konaut) May 8, 2019
GoogleBot Is Shared
GoogleBot is used across many pieces of Google, not just web search but also ads, rendering, and tons of other Google services. Gary wanted to point that out:
I feel the need to point out that "googlebot" is a piece of shared infrastructure and different teams use it at different settings, in particular, some already have the new rendering engine, others don't. It's not like your Roomba that has only an on/off switch! Oh wait... pic.twitter.com/oqZHtoutmd
— Gary "鯨理" Illyes (@methode) May 8, 2019
Forum discussion at Twitter.
Update: I posted a new story with things you may have missed related to this evergreen GoogleBot.
