“The world may have a polling problem,” declared Nate Silver, the former New York Times statistical wunderkind who’s since launched FiveThirtyEight under the ESPN banner, in an article last year.
Silver made this claim following the failure of UK polls to predict results of the country’s general election last May. In the US, he noted the Democratic surge in 2012 and Republican congressional takeover in 2014 were both largely unforeseen.
What’s behind the declining accuracy of political polling? And, what can be done about it? A handful of tech firms – including Rocket Fuel, Semcasting and Centro – are trying to help by using data collection through digital channels to supplement wobbly results of telephone polls.
Hamstrung By Mobile
As with traditional cookie-based audience measurement, political polling has been crippled by the migration of users to mobile devices. US adults with cell-only service (meaning they have no landline) went from less than 5% to almost 50% over the past decade, according to the Centers for Disease Control and Prevention.
“We are less sure how to conduct good survey research now than we were four years ago, and much less than eight years ago,” wrote Cliff Zukin, president of the American Association for Public Opinion Research, in a NYT op-ed last summer arguing that the political polling paradigm is broken, with no fix in sight.
The Federal Communications Commission makes it cost prohibitive to poll mobile phone users by banning robocalls, meaning real human beings must dial the tens of thousands of numbers required to complete a standard 1,000-person survey. (For landlines, the standard practice is to auto-dial and then pass the call to a live pollster when a person picks up.)
Considering that it’s also standard to pay poll respondents a small fee for their time, even a small mobile survey can turn into a campaign money pit. Bear in mind too that polling is most useful when it can be replicated over and over to demonstrate trends.
There are also mobile red herrings that have never affected pollsters before, said Scott Keeter, the director of survey research at the Pew Research Center. For instance, consider how easy it is to be misled by area codes when dialing mobile users.
Keeter said this is a particularly painful issue because of the importance of exact regional data for political polling.
The Tech Opportunity
The explosion of Internet and cell phone use in the past decade has caught political pollsters flat-footed, and digital marketing specialists believe they are poised to win budgets by insinuating themselves into that knowledge gap.
“We have been working with pollsters who are experimenting with blending online and offline polling,” said JC Medici, Rocket Fuel’s national director of politics and advocacy. Medici said that as other channels, like offline or mobile outreach, become more expensive, Rocket Fuel is increasingly using online polls as a value-add for political clients.
Five years ago, digital specialists may never have interacted with a campaign pollster, according to Peter Pasi, a Collective VP and GOP media consultant. Nowadays, he said, those roles are coming together under a campaign director.
In off-the-record conversations, 2016 presidential campaign directors and pollsters agreed, saying digital tech experts, once siloed within a campaign just like a direct-mail vendor or TV media-buyer, are being deployed more cohesively with polling operations.
Semcasting CEO Ray Kingman said his company has begun offering “flash polls,” which survey voters over social media, email and display. The results lack the academic rigor of traditional poll sampling, but will be familiar to marketers that regularly run product or brand lift surveys.
“Even with the shift to more digital ads in political campaigning, traditional polling data remains one of the most important KPIs for a campaign,” said Grace Briscoe, Centro’s VP of candidates and causes.
But ad tech-driven polling solutions come with a few major shortcomings.
If mobile polling is insightful but massively expensive, Internet surveys have the opposite problem: They’re cheap, but unreliable. Campaigns don’t want to see poll results where margins of error can’t be calculated, as is the case with almost all online surveys.
Many people interviewed said much of the “polling” conducted by digital specialists is really just an ROI tool for a campaign’s digital team to justify increased spend (like the “States of Waste” research project from Google and Targeted Victory, a Republican ad tech firm, that highlights wasted TV spending), as opposed to a strategic tool to predict voter behavior.
Pasi said ad tech is well suited to extend the capabilities of political polling. If you wanted to gauge a subset of, say, suburban women, a campaign could commission an expensive phone and door-to-door survey or just track engagement with content (whether an ad or a survey) delivered to an online audience segment.
There are also drawbacks and concerns about the reliability of evidence.
Keeter said the professional polling industry is struggling to model around survey respondents who are just in it for free airline miles or (unlike direct mail and phones) could easily be using throwaway personal details. Online polls and surveys still provide value, and at a low cost, but he cautioned against trying to extend their utility into budget decisions.
“Online surveys have been integral in a market research context for years,” said Keeter. “They’re accustomed to making business decisions based on them, but there’s no analogy for the consequences when you’re making those decisions in a presidential election and end up with egg on your face.”
“We can do brand lift studies on whether someone’s a popular candidate,” said Pasi, “but that doesn’t say how digital efforts affect the outcome.”
The opportunity and danger is large. As a result of the shortcomings of traditional polling, two of the most respected polling firms, Pew Research and Gallup, have abandoned presidential primary polling, which is a lucrative traffic driver but also a potential embarrassment when predictions go badly awry.
But one business’s embarrassment is another industry’s treasure. For ad tech companies that have long since come to terms with mobile measurement failures and are eager for any purchase whatsoever in Washington, DC, digital polling doesn’t have to be perfect to be profitable.
