We love it when Google Engineers spill the beans... on just about anything Google especially when it comes to revealing juicy secrets on improving your ranking position in Google. Or better yet ways that they recommend getting your site indexed that are not your typical suggestions.
A recent thread on SEW is talking about a post on High Rankings that covers what Google Engineer, Dan Crow, Director of Crawl Systems at Google has to say about getting your site lovingly indexed by the most popular search engine.
Some juicy tidbits from Dan:
New "unavailable_after" Tags - This little gem will allow webmasters the ability to tell Google when to stop indexing a page at a certain time. For example this might be useful for people with ecommerce sites with lots of coupons that have expired, older news items, and just about anything that is temporary and not permanent in nature on a site. While I really like this planning type tag, I don't see how much better it would be than just disallowing the page, or using a META robots tag to tell Google. Or better yet, using Webmaster Central URL removal feature. I see this tag probably having wide usage on news sites where there is a large number of pages that would need to expire at certain times.
Nosnippet & Noarchive tags - He details that these tags are not generally recommended, because he says "snippets are extremely helpful to visitors, as is showing the cache". Essentially these tags eliminate some problems associated with Google caching and improperly displaying the snippets below the titles in the search results. Google is fine with their use but would rather you not. This we know but its good to hear it again.
Avoid Walled Gardens - Dan used this term from the HR article and I thought it a nice way to explain how a group of pages are linked only to each other and not to anywhere else. He said you could put one of the links from that group in a sitemap and Google would index it and follow the other links. I think pointing an external link to those pages would be a much better idea. He says "those pages would be likely to be indexed via the sitemap...but considered low quality since they wouldn’t have any PageRank. Google is working on a way to change this in the future." Interesting.
For the rest of the discussion about these news see the Search Engine Watch Forums thread and the original High Rankings coverage on this. A thank you to Jill Whalen for covering this.
