Google Crawl Rate Drops: Google Responds?

May 2, 2008 - 8:00 am 9 by

Yesterday I reported that GoogleBot is crawling less pages then they once were, based on a large WebmasterWorld thread. Now, I spotted a response from a Googler at a Google Groups thread with similar complaints.

This time, I decided to login to Google Webmaster Tools and check my crawl stats, which can be found under Tools and then at "Set crawl rate." I noticed the same thing for this site, as well as my other sites. Here is a graph from this site:

Google Crawl Rate Dropping

John, the Google representative, blamed the Google Groups thread specific example on a slow site:

Looking at your graphs, it looks like the number of pages that we're crawling has gone down because it's taking so long for us to get each page. In general, there could be any number of reasons why that could be and I think that it may make sense to diagnose it properly, as the others have suggested. Having a slow site is not only bad for search engine crawlers who can't crawl as much as they would like, it's also something your visitors won't be happy about.

But the issue seems way more wide spread then what is in that thread alone and maybe we really do need a response from Google? My traffic referrals from Google seems to be about the same as it was throughout the crawl patterns. But SEOs and webmasters become concern that slow crawl rates and mean slow traffic from that engine in the future.

Forum discussion at WebmasterWorld and Google Groups.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: April 1, 2025

Apr 1, 2025 - 10:00 am
Google Search Engine Optimization

Google Ranking Reddit AI Translated Pages Too Well

Apr 1, 2025 - 7:51 am
Google

Google AI Overviews Tests Check Important Info Disclaimer

Apr 1, 2025 - 7:41 am
Bing Search

Bing Tests New Sitelinks Hover Shadow & Related Search Designs

Apr 1, 2025 - 7:31 am
Bing Search

Bing Copilot Answer Buttons To Image & Video Search

Apr 1, 2025 - 7:21 am
Google Search Engine Optimization

April 2025 Google Webmaster Report

Apr 1, 2025 - 7:11 am
Previous Story: Daily Search Forum Recap: May 1, 2008