A couple weeks ago, Google slightly changed how they show the Google search results, which resulted in several search ranking software packages to not function properly anymore. I reported this under the title of WebPosition Ranking Software Being Blocked by Google? early on and Scott Goodyear from WebPosition commented here and also at WebmasterWorld with his two cents.
Scott took offense to me using the word "blocking" in the title. The truth is, I posed it as a question and Google was not technically blocking them, just yet. But will it happen? It seems like it.
Matt Cutts of Google has commented (once again) at WebmasterWorld on his stance on ranking checking tools, saying:
I wouldn't be surprised if our bot detection continued to get better over time for many types of software that scrape Google. We'd prefer to use that server capacity for real users, not someone who wants to check how they rank for tons of queries going down hundreds of pages.
Does this mean that Google is now going to try really hard to prevent scraper based rank checking software? Seems like it.
Google's JohnMu also commented at our site, explaining some of the history:
I think the information given in the Webmaster Guidelines and the TOS is fairly straight-forward:"Don't use unauthorized computer programs to submit pages, check rankings, etc. Such programs consume computing resources and violate our Terms of Service. Google does not recommend the use of products such as WebPosition Gold™ that send automatic or programmatic queries to Google."
and "5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google."
Matt Cutts also commented in a thread Scott started in the Webmaster Help groups.
I am not going to get into the argument, if this should be allowed or not. I would hate it if people wrote scrapers to constantly scrap my pages, oh wait, they do. It is a waste of my bandwidth and can impact the speed of our page loads. At the same time, there are easier ways to get this data that might be more in accordance with the TOS.
There is also a big debate on why SEOs need to bother with ranking reports.
I'll leave the discussion in the air.
Forum discussion at Google Groups, Sphinn and WebmasterWorld.
