The other day I reported on how some are upset that Bing Chat is surfacing content that is behind paywalls, raising suspicions that Bing Chat is not respecting the robots.txt directives. While that is probably not true, as I reported earlier, Fabrice Canel from Bing did hint that Bing will be adding more controls around crawling, indexing, and serving for Bing Search vs. Bing Chat.
Fabrice was asked about this on Twitter, where Joost de Valk, who founded Yoast, asked "is something happening here that should be prevented? The fact bing has access to paywalled content for first click free type stuff means it should probably make specific arrangements for that for Bingchat." Fabrice from Microsoft replied, "Yes, I and we are working to address this."
I asked about this and got a bit of a longer response in the Duda webinar I helped host last week. At the 54:55 mark in the video I asked about maybe Microsoft giving a specific robots.txt directive for Bing Search vs. Bing Chat. I asked, "any thoughts about better controls for Bing Search versus Bing Chat to consume content?"
Fabrice responded, "I discussed on Twitter that I am working on this. We should be able to announce something next month. We truly believe that there is something about publisher and so we should, we will announce, announce something next month."
Here is that tweet by the way:
Yes, I and we are working to address this.
— Fabrice Canel (@facan) March 23, 2023
Here is the video embed:
Forum discussion at Twitter.
