Politics

Gallup defends (?) its erratic presidential polling results; Paul Ryan says results were “a shock” on election night

by  • 

I’ve obviously been following the ongoing questions about the failures of two of America’s best known polling firms — Gallup and Rasmussen — and the stunning (to me) revelations that the Romney/Ryan team really did believe all the right-wing spin that they were destined to win last week’s election.

From Talking Points Memo, Ryan: It ‘Looked Like We Stood A Pretty Good Chance Of Winning’ On Election Day:

“The polling we had. The numbers we were looking at looked like we stood a pretty good chance of winning,” Ryan said. “So, when the numbers came in, going the other direction. When we saw the turnout that was occurring in urban areas which were really fairly unprecedented, it did come as a bit of a shock. So, those are the toughest losses to have — the ones that catch you by surprise.”

Click here for the entire interview video and read a bit more:

As they saw the states fall one-by-one, reality began to settle in, and by the time Ohio was called for President Barack Obama, they knew the race was over.

“Once we realized that we probably weren’t going to win Ohio, that’s when we realized that it probably wasn’t going to turn out for us,” Ryan said.

Really? Even on election night, with North Carolina too close to call deep into the night, with Obama performing well in Virginia and Florida, with states like Iowa and Colorado showing good returns for Obama — even with all that going on, Ryan thought they were in a position to win until Ohio was called?

Wow, talk about insularity.

Anyway, there’s a bit of a bizarre memo posted on Gallup’s website today. It’s dated November 9th, but no one in the media covering such things seems to have seen it until today. From Polling, Likely Voters, and the Law of the Commons by Gallup Editor-in-Chief Frank Newport:

We will continue to examine our likely voter procedures, a real key to understanding the final popular vote. Our final estimate of registered voters was an unallocated 49% for Obama, 46% for Romney. The transition to likely voters moved that to the unallocated 49% Romney, 48% Obama.

We have modified our likely voter procedures in a number of ways over the years since they were first developed by George Gallup and Paul Perry decades ago. But I think it is clear that voting today is subject to new pushes and pulls, including, in particular, the highly sophisticated ground games employed by the Obama (and, to a lesser degree, the Romney) campaign this year. These methods may in the end affect voters who were not certain about voting at the time of a poll interview, but who were brought into the voting pool at the last minute by aggressive get-out-the-vote and late registration methods. Our traditional “bootstrap” method of identifying likely voters is self-weighting — letting voters’ responses to questions determine their probability of voting. This bears investigation. We will use the government’s post-election data, along with internal evidence, to see if further assumptions, investigations, or changes might be necessary.

We do believe that the presidential campaign underwent significant changes as it progressed this year. Romney clearly gained as a result of the first debate in Denver, and he held onto at least a marginal lead position in our polling until the week before the election, when Superstorm Sandy hit. Obama gained five points on the gap between our last pre-storm polling and the final poll. It may be that he continued to gain on into Election Day.

So Gallup thinks they were right all along? Romney was 6 points up a week before the election, even though the vast weight of the polling gave the edge to Obama? Then Hurricane Sandy swung the election 5 or more points toward Obama, even though its main impact was in states where Obama was already destined to prevail? And that there their miss of a single point (“a statistical tie”) was acceptable given that Obama’s win will be more than 2.5 points once all the votes in California are counted?

Sitting here in Savannah, I was convinced that Gallup had dramatic problems with its likely voter model, which I wrote about in October and then referred to briefly in my electoral prediction on Nov. 5th:

Newport also takes a jab at Silver and other aggregators, while raising a legitimate concern:

It’s not easy nor cheap to conduct traditional random sample polls. It’s much easier, cheaper, and mostly less risky to focus on aggregating and analyzing others’ polls. Organizations that traditionally go to the expense and effort to conduct individual polls could, in theory, decide to put their efforts into aggregation and statistical analyses of other people’s polls in the next election cycle and cut out their own polling.

Silver has already tweeted a couple of reasonable and pithy responses:

Sort of a disappointing response from Gallup to all of this.