Nate Silver and his team at FiveThirtyEight are still at it. And I sincerely suggest that journalists interested in giving accurate information in elections spend some time reading his wonky post-mortems.
As most of you probably know, Silver’s method of aggregating polling data was vindicated on Tuesday night, when his model had Obama the favorite in every state that he in fact won. In FiveThirtyEight’s projection of the popular vote, Obama was a 2.5 point favorite. He won the popular vote by 2.6 points.
In a post last night, Silver deals with the basic issue: Which Polls Fared Best (and Worst) in the 2012 Presidential Race.
A snippet:
As Americans’ modes of communication change, the techniques that produce the most accurate polls seems to be changing as well. In last Tuesday’s presidential election, a number of polling firms that conduct their surveys online had strong results. Some telephone polls also performed well. But others, especially those that called only landlines or took other methodological shortcuts, performed poorly and showed a more Republican-leaning electorate than the one that actually turned out.
Twenty three polling firms conducted 5 or more polls in the final three weeks of the election. Of those, TIPP had an average error of .9 percent and the tiniest of partisan leans with a .1 percent bias toward Romney. In other words, simply averaging the TIPP polls of the presidential race would have led someone to a prediction as good as Silver’s. But in terms of average error for individual polls, the second best was Google Consumer Surveys at 1.6, with RAND Corporation at 1.8.
But it’s even more interesting to look at the terrible misses of the 60 final Rasmussen polls, which included the tracking polls, and Gallups abysmal performance in its tracking poll. The Washington Post/ABC News poll had a bad year too:
Back on October 19th, I wrote a lengthy post about why Gallup’s numbers didn’t look trustworthy this year.
Time for some soul-searching at Gallup and at Rasmussen too — 60 polls with an average miss of 4.2 percent! With a Republican bias of 3.7 percent! One could argue in Rasmussen’s defense that the race moved toward Obama at the end. But if it did so, it couldn’t have been more than a point in the final week or so.
Also an interesting note: of the 23 polling firms that were most active in gauging the presidential race, 19 of them had a statistical bias toward Romney — the polls that were supposedly biased against him actually turned out to be biased against Obama.