Google's John Mueller said in a video hangout from May 1st, that often when he looks into SEOs or publishers complaining about Google de-indexing issues, often the issue is with Google not finding enough quality on the pages it has deindexed. John said in these cases, Google no longer wants to index the pages because they do not meet the quality benchmark.
He said this at the 33:41 mark into that video. Glenn Gabe gets the hat tip, when he posted this on Twitter summarizing this as "Are your articles suddenly getting deindexed? Via @johnmu: This is usually a quality issue & less of a technical one. Our algorithms might be looking at that section of the site, or the site overall, & thinking it doesn't make sense to index pages there."
Here is the video embedded at the start time:
Here was the question:
Our blog section of our website just has a lot of articles, categorized by month they were uploaded. There is no grouping as such, as to putting all related articles within a particular section together, but there is some internal linking between related articles. Could that be the main reason that our articles are not indexed anymore? They used to all be indexed, but then after a particular algorithm update, you de-indexed them. It's the only part of the website that is de-indexed, even though the information and the articles are good quality.
Here is John's response:
I don't think this would be a reason for our systems to de-index a lot of articles on our website.Essentially, we need to be able to discover those pages so that we have the opportunity to go out and index them. But if we can discover those pages, and they're just linked in a way that might not be optimal, then that generally wouldn't be a reason for us to say those pages themselves are not useful enough to actually be indexed.
Usually when I see questions like this where it's like, we have a bunch of articles and suddenly they're not being indexed as much as they used to be, it's generally less of a technical issue, and more of a quality issue.
So it's not so much that we can't find those pages, because probably they're still linked within your website, or at least they were findable in the past. It's probably not the case that we can't index them, because that's probably easy for you to check. And you can double check that there is not a no index, and that they can be crawled properly without crawl errors. That's usually pretty easy to double check. But it's more a matter of just our algorithms looking at that part of the website, or the website overall and saying, we don't know if it actually makes that much sense for us to index so many pages here. Maybe it's OK if we just index a smaller part of the website instead.
He added it is good advice to get third-party advice from someone not you or close to your site:
So that's something where sometimes it's worth taking a hard look at your site, or getting someone more neutral to look at your website. And to give you some advice and say, like, well, it looks like all of these articles are about irrelevant things that happen at some point, and maybe they were relevant at some point in the past, but they're not so important anymore. It might be that someone looks at these articles and says, well, it looks like you've been rewriting existing articles from, maybe, other people's websites. Maybe that's not such a great thing to do. Maybe that's something that Google doesn't really appreciate. Which is definitely true. But all of these things are elements where sometimes when you're creating this website, it's like, your baby, and you're doing the best that you can to make it grow, but having someone neutral look at it can sometimes give you a little bit of a, I don't know, a little bit of a kick to actually be a little bit more critical with your website, with your own content, and to find ways to significantly improve it. And I realize that's always hard to hear, and there is no simple path to significantly improving a website, but sometimes that's really the direction that you need to head if you care about this website, and the way that it's shown in search.
More: Take a hard look at your site & gain objective feedback from 3rd-parties to determine quality issues. Is it a relevance problem, are you rewriting content from other sites, etc.? There's not single path to significantly improving a website, but that's what you need to do: https://t.co/87PFVUKmjN
— Glenn Gabe (@glenngabe) May 4, 2020
Forum discussion at Twitter.
Update: Gary Illyes from Google added a bit more on Twitter after this post went up:
Of course, this is oversimplified, but often when you see some stuff disappeared from our index, it might just be that it was replaced by higher quality stuff...s. Stuffs? Stuff? Can't England.
— Gary 鯨理／경리 Illyes (@methode) May 15, 2020
