
Google's John Mueller explained how to prevent pop-up dialogs on your website from appearing in Google Search using the robots.txt file. He mentioned how to do this if you are using JavaScript for the pop-up.
The question was posted on Reddit asking how to block these legal pop ups from showing up in Google Search and being indexed by Google. The question was:
Say I have a website websiteA.com - for legal reasons it has an age verification pop up that sits on a separate domain and is inserted on the page via an API. Lets call it verificationpopup.comLets say I wanted to block Google from seeing this - if I added a disallow on verificationpopup.com/robots.txt would that prevent Google from rendering the pop up?
John Mueller replied saying, "That would generally work." "If you're using a specific JS script file to do this, you could also disallow that specific javascript file via robots.txt," he added. He also said that "if you only have one JS file that does a bunch of things, I wouldn't do that."
Here is his more detailed response with these two options:
- The embedded site's content is blocked by robots.txt. If you embed content from verificationpopup-com on your site (say via iframe or by calling a script on verificationpopup-com, etc), and verificationpopup-com disallows all crawling, then crawlers that follow robots.txt (like Googlebot) would not fetch that content, and would not use it in rendering. This is how many ads services work. Googlebot wouldn't crawl these URLs like a separate page, but it would try to fetch them during the course of rendering, and if robots.txt blocks it, then it doesn't fetch the content, and can't use it. You need to make sure that your pages still load when those URLs are not fetched, but for something like this, that's usually not a problem.
- Your site uses a specific JS file to load this, and you block that yourself. For example, perhaps you use a "load-verification-popup.js" file to do this. If you block that page with robots.txt on your site, then the script won't be loaded during rendering (blocked by robots.txt). In my experience, this is less common, but it's an option. If the robots.txt on verificationpopup-com is not something you can influence, you'd need to do this if you wanted to prevent it from being visible/used during rendering.
There are probably other ways to do this but here is the recommended approach from a Google Search representative.
Forum discussion at Reddit.

