Google added a new feature last night to Google Webmaster Tools that really shows you some nice detail on your index status.
The report not only shows the number of pages crawled by Google, but breaks them down on the total number of pages crawled that were indexed, the pages that were crawled that were not indexed and the pages that were attempted to be crawled but were blocked.
Here is what it means:
- Total indexed: The total number of URLs from your site that have been added to Google's index.
- Ever crawled: The cumulative total of URLs from your site that Google has ever accessed.
- Not selected: URLs from your site that redirect to other pages or URLs whose contents are substantially similar to other pages.
- Blocked by robots: URLs Google could not access because they are blocked in your robots.txt file.
Let me show you what various sites look like in this chart view.
A normal looking site's advanced crawl status:
A site that redirected their URLs to a new site (shared by JohnMu):
A new site just starting to get indexed with lots and lots of pages:
One key point as Google said:
Notice that the counts are always totals. So, for example, if on June 17th the count for indexed pages is 92, that means that there are a total of 92 pages indexed at this point in time, not that 92 pages were added to the index on that day only. In particular for sites with a long history, the count of pages crawled may be very big in comparison with the number of pages indexed.
This is an outstanding tool and I think many SEOs and webmasters will benefit from it.
Forum discussion at Google Webmaster Help and Google+.
