Thursday, 27 September 2012

The misleading average of the opinion polls


When faced with a myriad of opinion polls all showing different results it is natural, I suppose, that people turn to an average of the findings to get a shorthand look at what is actually happening. So it is that the Real Clear Politics site with its RCP Average has gained such a following. But really, does this daily figure help or hinder an understanding of the likely voting intention of Americans at the forthcoming presidential election? On my, admittedly cursory examination, there are grounds for thinking that the RCP average is quite misleading.
The reason is simple. Not all pollsters use the same methods. Some are thorough and others are slipshod. They use a range of different measures to weight the findings from their actual interviews to try and make their published figure representative of people likely to vote. As the reliable British site UK Polling Report stresses, "by averaging quality polls will ropey ones you don’t get better figures, you just make the better ones worse."
And the evidence suggests that there are some "ropey" polls in the Real Clear Politics sample that distort the result.
Take Rasmussen for a start. Surely its results should not be taken seriously by anyone.
Some evidence. 
Since the beginning of April (I have chosen that starting date because it is when Gallup began running an Obama versus Romney question) the average result of all polls in the RCP sample puts Obama at 47.2% and Romney at 44.6% - a difference of 2.6 percentage points. Yet the average of the Rasmussen figures produces figures of Obama 45.6% and Romney 45.8%. An Obama win of 2.6 has become an Obama loss by 0.2!
Gallup also has tended to give an Obama figure lower than the total RCP average: Obama 46.4% to Romney's 45.7% - a difference of just 0.7 points..
Yet Gallup is one of the most respected pollsters in the country. Is this just one of those statistical aberrations that can happen or is there a reason? I have noticed on one blog site  the suggestion that Gallup assumes low African American turnout based on 2010 when others base on 2008 or allow the sample to be the sample without weighting.
The RCP Polling Summary
The difference between Obama and Romney since April
All polls - Obama +2.6
All polls without Rasmussen and Gallup - Obama +3.4
Rasmussen polls - Obama -0.2
Gallup polls - Obama +0.7
Post a Comment