Is BT Just A Sales Tool?

Andy Atherton is COO of Brand.net, an online advertising network.

Networking - Andy Atherton, Brand.netA senior agency executive who manages the digital account for an Ad Age 50 CPG manufacturer recently delivered the best line I have heard in a long time.  We were talking about Behavioral Targeting (BT) and he said, “In my experience BT is a much better sales tool than a success tool.”   He then enumerated, through specific, practical examples, the many weaknesses of BT.   Since I have been discussing this in private for years, this pithy formulation, delivered unprompted from an industry leader, convinced me that the time was right to do a little public debunking on BT.

I’ll frame the discussion using an example from another meeting I had the same week with a digital buyer for a major national peanut butter brand.  A bewitching sales person had convinced this fellow that through the miracle of BT, based on Nielsen data, he would be able to run a mass reach campaign against customers of a named competitive peanut butter brand who were going to use the product for baking rather than spreading.  In short, he could target, at scale, moms who are baking peanut butter treats for the holidays, but whose kids won’t eat PB&J sandwiches.  I have to say that sounds amazing from a sales perspective.  I mean, how could you not want to buy that?  Unfortunately, the expectations the seller had set in the buyer’s mind were far from the reality of what’s possible.

Before I go any further, I want to make it crystal clear that I in no way mean to suggest that what Nielsen is doing isn’t great – we’ve done a lot of work with Nielsen and they do a fantastic job.  I am simply using the downstream sales claims that are often made invoking Nielsen data as an example of the shenanigans that happens routinely today in BT sales pitches.

With that said, let’s begin to dissect our example.

The origin of the underlying purchase data for the peanut butter targeting was Nielsen’s HomeScan panel.  The HomeScan panel consists of approximately 95,000 households that have agreed to give Nielsen rich demographic data and to scan every barcoded item they bring into their homes.  Of those, there are about 75,000 web-enabled households that have agreed to allow Nielsen to monitor some or all of their web behavior.  Within this panel, Nielsen absolutely can identify all households that have purchased a named product and absolutely can enable media partners to indirectly target like households with online advertising.  Unfortunately, even for a high penetration product like a major national peanut butter brand, a reasonable expectation would be that less than 25,000 of the 75,000 households made purchases of the named product.   Furthermore, the user overlap between the Nielsen HomeScan panel and even a large media partner is unlikely to be above 50%.  Let’s be generous and, for the sake of argument, say that this online media provider can identify 15,000 households that purchased the competitive peanut butter brand based on the Nielsen data.  Let’s call these Competitive Peanut Butter buyers “CPB”s.

It is a fact that, based on Homescan data alone, Nielsen has absolutely no idea what these 15,000 CPBs intend to do with the peanut butter after they have scanned it--spread it, bake with it, feed it to pets-- they don’t know.  It is flat out misleading for a salesperson to suggest otherwise.  Nielsen themselves certainly would never say that nor would they ever allow direct targeting to homescan panelists.  But again, for the sake of argument, let’s just assume that magic occurs, and the 15,000 total sample is reduced to the 3,000 users who actually meet the profile that the buyer thinks he’s getting:  CPB bakers only.  Let’s call these users “CPB Bakers” and let’s say the buyer was willing to pay a $50 CPM to reach that incredibly tight target.  Do the straightforward math:  he could only spend about $1500 to reach those CPB Bakers, unless campaign frequency was permitted to go above 10 - already a frequency more than 50% higher than our CPG customers are targeting.  If that campaign drove $3 in sales for every $1 in media – a fantastic ROI – only several dozen more cases of peanut butter would be sold.  Nationwide.

You get the picture.  To run a campaign reaching millions of users, as CPG advertisers need to do, the actual purchase data for thousands of users must be grossed up by a factor of 1000 or more and anonymized - thousands of specific users become millions of anonymous users - using Look-Alike Modeling.

Let’s keep digging.

Look-Alike Modeling is an attempt to identify similarities in browsing or clickstream patterns (“Look-Alike Patterns”), among the small group of users that have actually exhibited the target behavior (our CPB Bakers in this case).  It takes that set of Look-Alike Patterns and looks for similar Look-Alike Patterns among other users for whom there is no data about the target behavior.  This involves a huge assumption:  that similarity in Look-Alike Patterns is a reliable predictor of purchase behavior.    Ironically, Nielsen’s own data (as well as work by others) clearly demonstrates that at least clickstream data is not a predictor of offline purchases.  But, again for the sake of argument, let’s assume the exact opposite:  that Look-Alike Patterns are indeed predictors of offline purchases.

Even if we assume that Look-Alike Patterns are predictors of offline sales, with 3,000 users’ data driving a 3,000,000 user campaign, why would we assume that Look-Alike Patterns would be better predictors than the demographic, contextual and geographic variables that marketers have used for decades to segment their audiences?   Continuing our specific example, why would we assume that a campaign intended to drive offline sales of peanut butter would perform better if it was based on the Look-Alike Patterns of 3,000 CPB Bakers than if it targeted, say, moms aged 25-54 in high-quality food, health and parenting content?

I am aware of no evidence that BT performs better than other approaches in driving offline sales.  Indeed, what data I have seen suggests that it does not.  It’s one thing if BT can be tested and tuned with CPA data in a “closed-loop” lead-gen or ecommerce setting, but it’s an entirely different thing to assume that BT will be more effective than other methods in driving offline sales without the measurements to prove it.

What do you think?

10 Comments

  1. Andy, In its early incarnation, BT was absolutely a publisher's sales tool. Designed to help monetize hard to sell ads in general content, BT enabled news publishers, for example, to sell more space in news and sports to auto dealers who had already bought out the slim volume of auto ads available. As networks and data moved BT closer to center stage, it began to chunk off 15%-20%-25% of a given campaign budget.

    BT was the "it" solution for a while, but over time it's become a tool of choice only for pure remarketing and special situations such as auto and travel. You're right that no one has produced credible evidence that BT has done a whole lot for most categories, least of all CPG, which requires far more predictable reach and frequency than targeting, by definition, can deliver.

    I applaud your effort to shed some light here, especially given your experience at Yahoo, which is generally regarded as among the best practitioners of the art and science of targeting.

    I do think that the recent trend to multivariate targeting of audiences is a step in the right direction, but reaching television-like scale online is always going to be a big challenge for many ad categories.

    Reply
  2. The PB example used in the article indicts sales people that do not base their BT pitches and proposals in reality, not the idea of BT as a marketing practice. There are two issues here: 1) Is BT actually being delivered as pitched, and 2) If so, does it work and/or is it worth the premium? Number one might vary by sales organization. Number 2 needs to be evaluated by much more than CTR or "Actions." There are many, many reasons why a "Widget" advertiser needs to pay the premium to reach "Widget Enthusiasts" independent of performance across that segment.

    Reply
  3. Jeff Braddock

    Andy misses the mark here significantly on what the CPG advertiser was telling him. The issue with ad networks and the perceived value of BT is that ad networks aren't truly doing behavioral targeting. These organizations are setup to arbitrage and optimize ROS media, not find target audiences. His CPG contact's goal is to find audiences and nobody is fitting his bill.

    What ad networks are selling is site retargeting, not behavioral targeted audiences. To sell audiences, it requires companies like Andy's and his competitors to start investing in data. That means not taking shortcuts using cheap data resources like Quova and Digital Element to get the incorrect ZIP+4 information and applying the wrong demographic and psychographic data points.

    In Andy's example, the point he misses is that Nielsen data will only work well if it can be matched properly. Look-a-like models work in the offline, so there is not reason they won't work in the online world. When the ad network refuses to invest in data that improves the matching, the campaign fails before it starts.

    Bennett Zucker is correct that progress is being made in multi-variate targeting, and the limitations start and end with Ad Networks that simply display no interest in finding audiences. CBS Interactive's recent decision to ban ad networks evidence enough that ad networks need to move towards audience targeting if they plan to survive past 2010.

    Reply
  4. Alex English

    The fact that a sales person is out talking about targeting capabilities that don't exist (i.e. Peanut Butter Bakers) is pretty sketchy. Having said that it's no surprise and isn't it par for the course. Isn't ad sales a process of spinning what you have into gold?

    Having said that I do this this example is unfair. Let's say that the look-a-like models aren't delivering 3X ROI but the real question that isn't addressed in the example is, does the look-a-like modeled campaign deliver more ROI than a non-targeted campaign? That's the unanswered question. Nielsen needs to release some normative data so we can really understand the value of their targets.

    Reply
  5. Great dialog folks!

    A couple comments/clarifications to add to the thread:

    @ Jeff Braddock: The approach is described is absolutely not retargeting. As I explicitly pointed out, individual users are not retargeted. A behavioral profile is constructed based on a small core of purchase data and this profile is used to identify target users. My point is not that look-alike modeling isn't a valid approach. My point is that BT isn't the only way to "deliver target audiences". "moms aged 25-54" is a target audience, it's just not a precise a target as BT sales forces like to claim. But as the example shows, the claims are far beyond the reality and based on the data that I have seen BT doesn't outperform other ways of delivering target audiences if the goal is driving offline sales.

    @ Alex English: Good question. Fortunately, Nielsen did release normative data on a 10/6/09 Citi Research conference call. John Burbank, CEO of Nielsen Online, quoted the average ROI across the >200 offline impact studies they have run at 157%. That's a useful benchmark. I do want to reiterate though that this article is in no way intended as an attack on Nielsen. As I say in the article itself, I think the work they are doing with their major media partners is in many ways the gold standard of BT. That's why it provides such a useful case study in the limitations of the approach.

    Keep 'em coming!

    Reply
  6. So apparently if you write something negative it gets taken down. So having used this tool and saw penetration actually affect penetration and buy rate actually drive buy rate something must be working. It has delivered the best ROI we've seen and it WORKS!!! So, Andy why are you out here selling against it. Had your sales reps tell me that what you offer is exactly like Yahoo Consumer Direct. If it is so bad why does your sales rep keep pushing the match against this. Hypocrit.

    Reply

Add a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>