Often we will see Google issue out mass manual actions for link schemes or other webmaster guideline violations. We see it often when Google slaps a big link network or something of that nature. They are considered "manual actions" but some of it has to be automated in terms of figuring out who should be hit by the penalty.
As you know from yesterday, I interviewed former Google spam fighter, Murat Yatagan (@muratyatagan), on Google+. I asked him about this at the 6:24 minute mark. In short, it seems like Google does use automated methods to locate a lot of this but it is reviewed manually by many Googlers before the manual actions go out. There is a lot of manual work done by many Googlers to analyze the data to ensure there are very few, to zero, false positives here.
Let me share part of the transcript because typing it all up is making my fingers fall off:
Barry: So you talked a little bit about how you were part of the team that did manual actions but yet you wrote a lot of code or Python scripts to create these dashboards for directors, managers and engineers to see what's going on. This code that you wrote, did it help, was it specifically around reporting around the manual actions or more about you figuring out; these are the types of of manual actions were issuing, I wrote code hopefully to help you find this and more of an automated fashion? And maybe send out automated manual actions, if that makes sense?
Murat: Well that's a great question but it is a tricky one as well. So I guess I shouldn't be going into to much in detail in this manner. But still I'll try to give you a hint so that they can make a more educated guess of what people are doing.
As I mentioned people collaborating on team member… So what we're doing is my first try to identify what are the patterns of spam and let's say as an example having too much repeated keywords in URLS and those keywords at the same time in descriptions, in the snippets, in other parts of above fold of a page. And that gives you a hint that there's something going on here with the site, it is some shady usage of the pages, it shouldn't be like this. So and then you start thinking about what we are going to serve to user and you need to be in the shoes of Google so I was trained very hard to understand, to create unbiased decision for a page for a web site. And after a while you need to gain some trust from the other team members and after you get trust, your actions are automatically applied to manual penalties. So of course we're taking one-by-one penalties. Scaling up part was the most significant part of it. Otherwise mentioned it will be a one by one drop so you're always going to be much slower than the spammers. And in the end, if I create some guides… okay these are the patterns of spam. So spam can have this element, this element and everybody implying those elements to the big data… And trying to be integrated into the search algorithms.
Barry: So I'm just trying to clarify. Often we see manual actions some of them will be like one off, like this person. But often we see like we'll see like a big paid links manual actions go across thousands of websites or whatever the number is. That seems like there is some type of automation terms of data collection in terms of which sites are doing that. So someone wrote let’s say a Python scripts maybe his name is Murat and basically what he what happened was it collected all these sites are participating in this link scheme, you fed into this manual actions database somebody review it at a higher level, maybe manager, and then said this is legit let's go ahead and issues all these manual actions.
Murat: Well that basically generally applies to be did. Like it's just the part of what we did. The most of the bigger part is actually including more clever solutions. Although this sounds like it's kind of an automatic thing, it was a lot of manual analysis and developed a manual process. But after a while you start seeing some patterns. Like it's you know it's like the Matrix movie, when Neo…
You can watch the whole video here:
Forum discussion at Google+.
