BingBot Hiding User Agent?

Dec 20, 2011 - 8:39 am 1 by
Filed Under Bing SEO

BingBotA WebmasterWorld thread has one webmaster complaining that Microsoft is hiding the user agent for the spider, BingBot.

The webmaster said:

I'm not really sure if this is regular practice with MSN (Bing) or not. Since a few weeks back, they've been spidering my website without a proper User Agent (meaning their UA does NOT say anything about bingbot. It's just a regular browser UA). This happens from three different IP ranges. Yes, I've checked thoroughly and the IP rangers are registered under MSN.

I doubt this is manual review, since it doesn't look at all as normal human behavior.

Did they officially stop declaring their bots in User Agent strings?

So either Microsoft has a weird bug, someone is spoofing the crawler in a weird way or Microsoft is doing this on purpose?

I should note, I did not check my log files to verify this on my sites. So I am going with "zerillos" post and with no back up evidence.

However, Microsoft is known for their crawling issues. Most recently, crawling too hard, updating IPs and many more. BingBot is pretty new, coming to us just over a year ago.

Forum discussion at WebmasterWorld.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 31, 2025

Mar 31, 2025 - 10:00 am
Google Updates

Google March 2025 Core Update Impact Multiple Poll Results

Mar 31, 2025 - 7:51 am
Google News

Google To Pay $100 Million To Settle Old Ad Lawsuit On Clicks Outside Geo Target

Mar 31, 2025 - 7:41 am
Google Ads

Google To Allow Double Serving Ads In Different Ad Locations On Same Page

Mar 31, 2025 - 7:31 am
Google Search Engine Optimization

Google News Fully Transitioned To Automatically Generated Publication Pages

Mar 31, 2025 - 7:21 am
Google Maps

Google Business Profiles Bug Resurfacing Old Fake Removed Reviews

Mar 31, 2025 - 7:11 am
Previous Story: IE Error With Google AdWords Billing Tab