Over the past few weeks, I have been seeing some reports that Microsoft Bing is having their own indexing issues. Meaning, Bing is not indexing pages like it use to for its search engine. In fact, I am told Microsoft has confirmed the issue with CloudFlare engineers and it is on Microsoft's side to fix.
Some thought it had to do with CloudFlare being blocked, but it might be more widespread than just that based on the reports I am receiving both publicly and privately via email.
I asked Fabrice Canel from Microsoft about this and he responded on Twitter with a set of best practices to be indexed by Bing. :-)
Best practices in case a site is not listed at Bing. Investigate it it first via URL Inspection Tool https://t.co/InvjmzlbBB to identify possible crawling issues then read Bing Webmaster Guidelines https://t.co/Gbrm4B2XMX Pay attention to it to ensure your site plays by the rules
— Fabrice Canel (@facan) October 26, 2020
But let me take you through what I have seen. A CloudFlare Community thread has tons of complaints starting the other day. Here is the first complaint:
Hi several People had the same Problem like me also that their Cloudflare hosted Websites had blocked IP addresses from Microsoft from which the WebCrawler tried to access the Webpages.I just checked the email of my Website and got a Notification from Microsoft Bing that their Webcrawler could not access my Cloudflare Website.
Folks at WebmasterWorld have noticed as well, here are some quotes from there:
And now for some really interesting info: I blocked this IP in the Ubuntu UFW firewall on one of my websites and after about a week it was deindexed from BING. It seems that BING is much more strict and if they can not crawl your website they take drastic measures! It might be just a coincidence off course. So it's really not that clear cut if this is a "bad" IP because the reviews from users at "abuse IP" aren't really any proof other than conjecture on their part.
There's a Bing deindexing update/bug going on. A lot of users have reported it on other forums.
Checked on Bing to find out that my site had been completely deindexed like a month ago. That's what I get for never bothering to check Bing.
In September one of my websites was excluded from Bing's SERP. On October, 23 I noticed my three other websites were banned. Even if I type "site:example.com", there are no results. Just the link "Some results have been removed"
CloudFlare confirmed the issue saying "Microsoft has acknowledge this is a legit crawler and this is an issue on their side. They are working to resolve (by setting it to conform to their published expected behavior)."
So it seems like Microsoft Bing is working on a fix.
I am not sure if this is more than just a CloudFlare and Microsoft Bing issue or not, but there are a lot of complaints about this.
The good news is that it looks like Microsoft Bing is working on a fix.
Forum discussion at CloudFlare Community.
Update: More from Fabrice Canel:
For Cloudflare, some sites reported that a small percentage of Bing crawler was not DNS resolving appropriately and we were crawling outside of usual IPs for few days. We fixed that yesterday.
— Fabrice Canel (@facan) October 27, 2020
