Just an FYI, Google did not simply flip the switch last week and turn on the new GoogleBot, the evergreen GoogleBot, over night. Google said it was already indexing 10% of the web back in November. It scaled from less than 1%, to 1%, 10%, 20%, 30%, 40%, 50%, 75%, to 100% over several months and just reached 100% last week.
We didn't notice because Google didn't change the useragent of this evergreen GoogleBot, although we did see some evidence of it back in April.
I asked Martin Splitt and John Mueller about this last Friday in a webmaster hangout. I was curious if this was the reason for the chatter around a possible Google update last week. It could have been related to indexing and not an algorithm update but based on Martin's answer, it seems it was not related to the new GoogleBot.
Here is the video embed, it first starts with me asking John Mueller about it, before Martin Splitt joined the hangout at 6:47:
Barry Schwartz: Can we talk about Googlebot a little bit? So it went live technically two days ago, the new Googlebot, the new evergreen Googlebot?
John Mueller: Well we've been testing it for awhile. Hard to say when it went live because there's always this kind of transition period.
Barry Schwartz: So where did it go a hundred percent live or it's still transitioning? I don't know right, I need to have the other Martin here. But my understanding is it's it's a hundred percent live.
Then at 20:45 Martin Splitt joined and John asked him the question for me:
John Mueller: Did we switch to the new version of Chrome completely for crawling and indexing?
Martin Splitt: Yes, yes. So the thing, the thing that we do is as you, know we keep experimenting on new stuff with Googlebot on very very small sets of URLs basically very small sample. Sometimes just like basically a couple of URLs really. But we continuously improve or increase the amount of the sites that we would be crawling with Googlebot over the last couple of months, very carefully monitoring what was happening. The things, the issue that we confirmed that happened a couple months that has nothing to do with rendering. But yeah so we have been increasing continuously increasing the sample size of the URLs that we were crawling with the new Googlebot and half ramped it up to 100% already. I don't know the exact day when we did to be honest. I don't have it on top of my head because it just went smoothly.
Barry Schwartz: Was it from like 10% to like a hundred percent?
Martin Splitt: No, no no. It was less than 1%, 1%, 10%, 20%, 30%, 40%, 50%, 75%, 100%, I think were the steps that I have observed.
Barry Schwartz: You could send me an email with the exact percentages and dates… I am joking.
Martin Splitt: But basically it was a pretty rolling thing. I just sampled it at random points of time and checked in with rendering team. Zoe is in the rendering team, the person that was yesterday on stage for that JavaScript session. And I would I was as excited about it as everyone else. I was like oh joy. But I don't think I know the dates.
Barry Schwartz: But the whole process started a few months ago or a year ago? Or terms of the 1% testing.
Martin Splitt: I don't know. So I remember so I remember I talked to Zoey in December last year and I think that was, oh November. November we were at 10 percent.
So it seems it was pretty close to 80-90% for some time and that algorithm update was likely unrelated to this GoogleBot release.
Forum discussion at YouTube.
