I often run reader polls on this site asking specific SEO and SEM related questions.
Often I get a negative response that some of the polls are not scientific enough to be trusted.
So let me be clear - they are not scientific polls, they are reader polls. I poll our readers, not the community at large. And those who participate in the polls are more likely to be biased in one direction. I almost always say, when posting the poll results, that of the people who participated in the poll, the responses were X.
Plus, you have to take into account the poll bias, not just the bias of those who are likely to see the poll and those who are likely to participate in the poll. Then you must take into account the authors bias in interpreting the poll.
Let me give you some examples...
(1) I posted a poll that said 65% of SEOs were hurt by the Penguin update. Really? Do I think of all the SEOs out there, 65% had been negatively impacted? Probably not. I'd assume it is a lot less. But those who are more likely to respond to the poll are those who are mad and upset and who have been hit by Penguin. That is just plain logic. We did have well over a 1,000 responses but still, the poll is skewed by those who read my site and are eager to reply to the poll.
(2) Another recent poll I skewed the response of the poll by giving it a sensational title. I titled it Only 14% Of SEOs Trust Google. Only 14%, no way. I skewed my title to get clicks. Only 14% of SEOs FULLY trust Google, whereas 60% trust Google always or sometimes, which makes Google look much better. Only 35% of SEOs would never trust Google. Again, the title that is more in your face is 14% of SEOs trust Google but I skewed the title to attract more people to look at the poll results. I then published the full results so you can see. But clearly, again, apply the same bais I listed in example one and maybe those results should be more like only 20% of SEOs don't trust Google or maybe not?
So how skewed are my polls here? 87.3%. Okay, I am kidding but you get the point. I think I might use this post in linking to future poll results.
Forum discussion at Google+ & WebmasterWorld.
Image credit to ShutterStock for skewed graph
