Google has released a short video on how Search Quality Evaluators help Google make Google Search better. I assume this is part of Google trying to debunk some myths and get the word out that these Search Quality Evaluators do not manually impact the search results, at least not directly.
Here is the short video:
Here is the transcript:
Every year Google runs hundreds of thousands of tests to see if a change we're considering to search will actually make it better for the people who use it. One of the essential ways we test for that is by turning to real people. We call them Search Quality Raters. There are over 10,000 of them around the world and though the job title sounds simple what they do is actually quite rigorous work.It starts with our Search Quality Evaluators Guidelines, a publicly available manual every reader studies and is tested on before providing ratings. This gives readers clear guidance on how to evaluate things like relevance, trustworthiness and expertise. Then when we're considering a change to search, we ask a large number of readers to each review hundreds to thousands of searches. For each search readers see results from two versions of google and then open every link listed to provide a quality rating for each. Because raiders are required to give ratings based on common guidelines, free of personal opinions and beliefs, the rating process demands a lot of additional research. Like who an author is and if other reputable sources view them as authorities. Though a reader's task is to give a rating to each page they see. That one rating doesn't directly affect how that one page will rank in real world searches. Instead we look at all the ratings from all the readers across a wide range of searches to see which version of Google consistently delivered the highest quality and most helpful results. Last year alone we ran over 380,000 search quality tests, plus nearly 63,000 with our Search Quality Raters, resulting in more than 3,600 improvements to our search algorithms. And while our search results can always improve, this process has been very effective over the past 20 years and helping us ensure the changes we make meet the quality standards that people everywhere have come to expect from Google.
Forum discussion at Twitter.
