A WebmasterWorld sparked this post from me. At SES Chicago '06, during a session named CSS, AJAX, Web 2.0 & Search Engines the search engine representatives were asked about how they handle CSS.
It is currently easy to hide text using CSS, everyone knows it. But do people do it?
Back to the SES session, on this panel were search engine reps. Many of the search reps were new to conferences and were not necessarily prepared to get certain questions. It all started when a Yahoo representative told the crowd to open up your CSS so Yahoo can peak into it. Then Google said they will also be indexing JavaScript and AJAX and CSS, so don't use it to hack.
Now, if you know Yahoo! and specifically Google, they typically will never say that they will be doing anything in the future. They typically first do and then tell, but not tell and then do.
All the search engines, except for one, I believe (but I forgot if it was Ask.com or MSN) said that you should not block your CSS and JavaScript files from the search engines using your robots.txt, just in case they want to take a peak.
I am honestly still confused by that statement. Well, if we block it, will it raise a red flag? If it raises a red flag, will you manually peak? Are you going to algorithmically crawl those files and look for problems if we keep them accessible to you? If we format something a certain way, but it may appear like spam, but in reality it is not, will an automated ban come on the site?
Personally, I am not worried. But these types of responses, by the search engines, can fuel a lot of questions and unnecessary worries.
As pageoneresults says in the WebmasterWorld thread:
Google has a hard enough time now dealing with html/xhtml. Parsing CSS files and determining whether something is hidden or not is not a solution. Now the bot would need to determine why that CSS exists. There are many valid uses of display:none or display:hidden.For those who may be hiding things through CSS or negatively positioning content off screen to manipulate page content, I surely wouldn't do that with any long term projects. ;)
The penalty for getting busted using this technique I would imagine is a permanent ban. No if's, and's, or but's, you're history. You'll need a pardon from the Governor to be reconsidered for inclusion. ;)
Forum discussion at WebmasterWorld.
