MSN Indexing Pages With No Index Commands?

Jan 5, 2007 - 7:07 am 2 by
Filed Under Bing Search

There is a WebmasterWorld thread where folks are reporting that MSN Search is indexing a bit too much. Reports suggest that MSN is crawling and indexing the content found on pages which are excluded via the robots.txt directive.

Normally, if a search engine crawls the page, they may only keep the URL in their index but not the content.

Moderator Receptional adds, "My examples show no snippet. But I can get them by typing unique text into the search query."

So if they are returning results for text found on the page, maybe they are indexing a bit too much?

On the other hand, I have a thread at Search Engine Roundtable Forums suggesting MSN doesn't index enough, specifically new sites...

Forum discussion at WebmasterWorld & Search Engine Roundtable Forums.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 7, 2025

Mar 7, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google AI Mode, Gemini 2.0 Powered AI Overviews, Ranking Volatility & New Search Data

Mar 7, 2025 - 8:01 am
Google Maps

Google Business Profile Now Associated With Local Service Ads

Mar 7, 2025 - 7:51 am
Google Ads

Google Ads Location Targeting New Add Locations In Bulk

Mar 7, 2025 - 7:41 am
Google Ads

Google Ads May Get Channel Reporting For PMax Campaigns

Mar 7, 2025 - 7:31 am
Google

Google Drops The Horrid Page Annotations Feature

Mar 7, 2025 - 7:25 am
Previous Story: Idea To Place Google Image Ads Near Google Text Ads As Alternate