The accuracy of Facebook’s reporting is under the microscope again courtesy of Pivotal analyst Brian Wieser and an Australian trade pub called AdNews.
In a note to investors on Tuesday, Wieser called attention to Facebook’s claims that it reaches more people than exist within certain age groups, based on US Census data.
Wieser was tipped off by an AdNews story last week that noted a 1.7 million person discrepancy between the number of 15- to 40-year-olds Facebook says it can reach in Australia and the official population of the country, as specified by the Australian Bureau of Statistics.
Wieser recreated the experiment in the US using official government numbers with similar results.
Through its Ads Manager, which serves as the control center for advertisers creating and placing ads on Facebook, Instagram and through the Audience Network, Facebook professes to have a potential reach within the US of 41 million people between 18 and 24 years old; 60 million people between 25 and 34; and 61 million people between 35 and 49.
US Census data, by contrast, pegs the number of 18- to 24-year-olds in the US last year at 31 million and the number of 24- to 34-year-olds at 45 million.
Although Facebook got the total number of 35- to 49-year-olds correct, the overestimation of the other age groups rings in at 25 million people.
The dotted line between miscalculated metrics and advertiser trust is clear, said Jason Kint, CEO of Digital Content Next. But advertisers also need to ask the right questions.
“Not to state the obvious, but any metrics discrepancy with Facebook adds to the distrust in their incredibly powerful half of the duopoly,” Kint said. “The industry has failed to probe deep enough on the reliability of data, and I expect this is changing quickly as there have been more and more calls for third-party verification of Facebook’s data.”
At around this time last year, Facebook informed advertisers of multiple miscalculated metrics, including an overstatement of video ad view time and an erroneous inflation in the number of people visiting Pages. Although outrage was rampant at first, the misreported metrics weren’t buying metrics, and advertisers didn’t pull spend in any significant way.
What they did start to do, however, was push for more third-party measurement. That, Wieser told AdExchanger, is the real story here.
“People are focusing too much on Facebook and not enough on the Nielsen and comScore angle, and that’s the biggest takeaway here: a reinforcement of the value of third-party measurement in all forms,” Wieser said. “The fact that you can find errors by just challenging a piece of data and trying to independently verify it and then find that, in fact, you can’t, highlights the need to always have a little skepticism. Facebook needs to be mindful about how its numbers are interpreted.”
According to a statement from Facebook, reach estimations in Ads Manager aren’t “designed to match population or census estimates,” but rather “to estimate how many people in a given area are eligible to see an ad a business might run” based on a number of factors, including Facebook user behaviors, user demographics and location data from devices.
“This is just an estimator and campaign planning tool,” Facebook stated. “It’s not a business’s actual reach or campaign reporting, and is not billable.”
It’s also true that Facebook’s age data is self-reported and that Facebook’s default targeting is built from everyone in a specific location, which can include people who aren’t counted in the census, such as nonresidents and tourists.
And in a tweetstorm on Tuesday, former Facebook executive and “Chaos Monkeys” author Antonio Garcia Martinez, hardly a Facebook apologist, pointed out that the discrepancy is almost meaningless at Facebook’s scale:
It's meant to give a general idea of audience size, for not-very-specific segments, no more.
— Antonio García Mtez. (@antoniogm) September 6, 2017
Although Wieser is dubious of that logic – “You can put any caveat you want, but the numbers provided by the basic interface manager need to be plausible,” he said – he doesn’t see this episode having any tangible impact on Facebook’s bottom line.
“If you’re an advertiser, you have to spend on Facebook. That doesn’t mean you’re happy about it, but you are going to keep doing it,” Wieser said. “But it does arm TV sellers and make their argument stronger. And as Facebook tries to invest more heavily against Watch and becomes more credibly able to capture TV budget, they’re going to face more scrutiny than they previously faced.”
But Facebook, seemingly as ever, will be just fine, said Brian Baumgart, CEO and co-founder of Conversion Logic.
“The issue may, in fact, be data-related and rooted in how users report their age – it’s an inherent challenge across all platforms – and Facebook is the leader in using intelligence and other technologies to verify stated attributes through machine learning,” Baumgart said. “Facebook has spent the last several months developing tools aimed at more transparency for marketers and their campaigns, [and] they’ve been open to feedback and responded to marketers’ needs. This shouldn’t bring Facebook’s reporting into question.”
