Last April 2024, Gary Illyes from Google said he was on a mission to make web crawling more efficient, he wanted to "figure out how to crawl even less, and have fewer bytes on wire." Gary updated us saying "there's lots more in works" for this but implies some improvements have already been made.
Gary wrote on LinkedIn, "there's lots more in works. but if you already see improvements, I'm satisfied."
He said this in response to David Minchala who asked, "Hey Gary Illyes how's this going? Near as I can tell, looks like you've made good progress. can you confirm?"
Gary wouldn't necessarily confirm, I did ask:
Here are those comments:
Have you noticed any improvements? I mean, we did seem to have a crawling hiccup this month...
Forum discussion at LinkedIn.
Image credit to Lizz
