From The Washington Post’s Gallup and USA Today part ways:
Gallup will no longer be conducting polls for USA Today, the two organizations announced Friday.
Both said the split, after 20 years of collaboration, was a mutual decision based on the changing media and polling landscape.
“Given these shifts, Gallup and USA Today have made a mutual decision to move in independent directions beginning in 2013, and Gallup will evolve the polling it conducted in partnership with USA Today in some different and new strategic directions,†Gallup said. “As it has been, Gallup.com will remain the primary source for Gallup polls conducted in the U.S. and around the world.â€
USA Today said in its own statement that it is already in negotiations with another pollster.
Well let’s hope that whatever pollster USA Today ends up partnering with in upcoming elections, the newspaper has at least learned some key lessons from 2012. It doesn’t look like Gallup has learned any.
I wrote a number of posts critical of Gallup both before and after the general election in November. In mid-October, reading the work of Nate Silver and others, I even asked: What’s up with Gallup tracking poll giving Romney a big lead?
The serious problems with Gallup’s application of its likely voter model seemed pretty obvious to me in real time — so they must have seemed obvious to many others. The problems should have seemed obvious to USA Today too, but the paper reported on individual Gallup polls as if they were gospel — and in the process ignored dozens of state-specific polls that gave a much clearer picture of the complexion of the race.
How bad did USA Today end up looking?
Check out the lengthy piece on October 15, 2012 — three weeks before the election: Swing States poll: Women push Romney into lead.
From that breathless article:
Mitt Romney leads President Obama by four percentage points among likely voters in the nation’s top battlegrounds, a USA TODAY/Gallup Poll finds, and he has growing enthusiasm among women to thank.
As the presidential campaign heads into its final weeks, the survey of voters in 12 crucial swing states finds female voters much more engaged in the election and increasingly concerned about the deficit and debt issues that favor Romney. The Republican nominee has pulled within one point of the president among women who are likely voters, 48%-49%, and leads by 8 points among men.
Ha.
Ironically, if Gallup had just headlined its poll of registered voters — a 2 point edge for Obama in both the swing states and nationwide — they would have ended up looking pretty good.
Consider those 12 “swing” states: Colo., Fla., Iowa, Mich., Nev., N.H., N.M., N.C., Ohio, Pa., Va. and Wis.
Three weeks after that poll, Romney won North Carolina by almost exactly 2 points. But he lost all other 11 states. Obama won 8 of those 11 states by 5 points or more, including Michigan with a 9.5 point margin.
In all, Obama ended up winning those 12 states by 4.09 percent, slightly better than his 3.85 percent win nationwide.
By the way, for a great spreadsheet of the final popular vote tallies, check out this spreadsheet by David Wasserman of the Cook Political Report.
There are some obvious lessons here for reporters.
As Nate Silver might say, every poll has the potential to tell you something, but it’s easy to overplay the importance of any single poll.
A poll of 1,023 registered voters across 12 different states — like the one referenced in this piece — is especially problematic because it necessarily means very small samples in some of those states. Indeed, Gallup’s margin of error for the swing state poll reported on so definitively — legible in the fine print — was +/- 4 to 6 points.
It’s simply irresponsible journalism to report so heavily on a single poll when there are dozens of others being done each week.
It’s even more irresponsible when the poll in question so obviously contradicts state level polling with much larger sample sizes in each individual state.
And it’s still more irresponsible to report so heavily on a single poll when numbers crunchers like Nate Silver at FiveThirtyEight and Sam Wang of the Princeton Election Consortium are arriving at such wildly different conclusions based on all the numbers in the public record.
I don’t read USA Today enough to know if they published any sort of mea culpa for this type of coverage, but it’s worth noting that the Washington Post, The New York Times, and other major outlets similarly gave heavy coverage to polls they had commissioned.
Gallup responded to its big miss by missing the point — see Sam Wang here or my response here.
Let’s hope USA Today and other news outlets that commission polls are more measured in their reporting in 2014, 2016, and beyond.