The Rise Of 'Header Bidding' And The End Of The Publisher Waterfall

waterfall.illustration

 

The traditional publisher waterfall, where impressions are exposed to sales channels in descending order of the perceived value of each channel, has always stuck in the craw of yield-obsessed media sellers.

At its worst, the model ignores pockets of high-value inventory and reduces competition, and thus artificially caps a publisher's revenue.

Publishers have by and large been locked into this arrangement since programmatic selling became a thing, but that is changing.

In recent years, sellers have begun experimenting with a new technology approach that allows exchanges to bring in demand before the ad server call. The technology goes by different names: header bidding, pre-bidding, advance bidding, holistic yield management and tagless integration (even though it requires tags).

To enable it, publishers put a piece of code in the header of their pages, allowing demand sources to submit bids before the ad server callout. This process brings the digital media industry closer to the programmatic idealist's dream. That is, a single unified auction where demand sources compete side by side rather than sequentially, ultimately yielding higher CPMs and fill rates.

“Philosophically, we believe that all the demand should occur in one place,” said Jason Fairchild, CRO of OpenX, one of the sell-side platforms enabling header bidding.

SSPs and buying platforms are the primary vendors supporting pre-bidding, and their self-reported data suggests publishers are enthusiastic about these offerings. Rubicon said upwards of 50 partners are using it, Index Exchange claimed to have brought on 150 and OpenX said “dozens” of partners use its tool.

Programmatic consultant Matt Prohaska, who estimated header bidding has quintupled over the past year, noted that ad servers were designed to make sure a publisher’s direct-sold campaigns delivered. They were not designed for the daisy-chaining of ad networks or exchanges.

“The ad server has fundamentally not changed much in 18 years, but it is now the most critical piece of the ad tech stack because it’s where the decisioning takes place,” Prohaska said.

In a normal ad server waterfall setup, direct buys go first. Then, publishers allocate inventory to one or more exchanges, using estimates and averages of what each exchange can bring in – not by gauging demand in real time. As the dominant publisher ad server, Google's DoubleClick for Publishers is often "driving the bus" throughout this process.

Historically, a publisher’s ad server has to pick just one pool of demand each time, which lowers competition and CPMs.

“Managing programmatic partners is not optimum, no matter how you do it,” said Jonathon Padron, the business operations manager and programmatic lead at ThinkProgress.org. “As a publisher, you want a completely consolidated marketplace, and that’s not really how it is. You have to daisy-chain through.”

Header bidding levels the waterfall such that buyers and exchanges first submit real-time bids, before direct buys are fed to the ad server. Publishers have the option to let any exchange beat a direct-sold impression. And pitting exchange bids against each other drives up the second-price auction.

“This kills the waterfall,” said Andrew Casale, CEO of Index Exchange.

The Google Exception

One reason programmatic buyers like header bidding is that it allows them to bypass the favorable relationship Google has set up between its ad server, DoubleClick for Publishers, and its exchange, AdX.

Among the boosters of header bidding, this close tie-in between two Google platforms is sometimes viewed as an obstruction to progress. Once they integrate with AdX, publishers can get better yield by enabling the "dynamic allocation" and "enhanced dynamic allocation" features, which give AdX a chance to beat other demand in the ad server, even if it’s not its turn.

Many publishers love how dynamic allocation increases yield – StudyBreak Media Director Emry Downinghall called it “literally transformative for our business” – but those same publishers acknowledge this setup privileges Google’s demand sources, since no other exchange is given the ability to compete for an already allocated impression in the ad server.

That creates a waterfall that many insiders feel unfairly advantages Google and privileges AdX demand.

As one publisher bluntly put it, “Dynamic allocation screws the other exchanges because it allows AdX to cherry-pick inventory.”

As header bidding becomes more popular, DFP might be pressured to change to better incorporate programmatic demand from non-AdX sources. But Google declined to comment about any plans for DFP or how header bidding affects its ad server.

Many believe Google’s desire to keep a tight integration between DFP and AdX, and the financial benefits to itself of doing so, will keep this from happening.

“I’m not sure if that would happen since the dominant ad server is owned by Google,” Downinghall said.

“What they have right now is very good for them,” said Steve Hulkower, director of product management at Rubicon Project. “The way AdX plugs into DFP is powerful for AdX and Google, so I don’t know that they would break that.”

Meanwhile competitors are rushing in with their own solutions. Prohaska said four other companies are trying to build a complete publisher stack to rival Google’s: OpenX, AppNexus, Rubicon and AdForm.

AppNexus’ acquisitions indicate it’s executing on this strategy. It bought Xaxis’ ad server, OAS, and publisher forecasting tool Yieldex, which could improve ad server decisioning. In Europe, where anti-Google sentiment is stronger, AdForm is building a stack from scratch.

“I think it would be great if the ad server could change and do all these things for us,” said Paul Bannister, EVP of CaféMom. “Because it hasn’t, we don’t want to leave all the money on the table.” 

So, You Want To Set Up Header Bidding

While header bidding answers many concerns about the yield inefficiencies of the waterfall, it’s still technical and complex to implement. Interested publishers need to gear up for the technical challenge and develop trusting relationships with partners.

Header bidding requires putting third-party code on a website so buyers can view impressions before the ad server gets involved. These tags require ad ops expertise, and since they can give buyers a god’s-eye view of available inventory, publishers risk exposing valuable data.

Generally, header bidding is set up through a relationship with two types of partners: an SSP/exchange (Rubicon, OpenX, IndexExchange, PubMatic, InterMarkets and Sonobi can all do it) or a buying platform (like Criteo, Amazon’s A9, Yieldbot and Conversant).

On rare occasions, an advertiser itself can get its tags placed on publisher pages to access inventory before the ad server. P&G, for instance, has the clout to do this. But most advertisers will need to work through an exchange or buying platform.

Some publishers prefer to work with one or two partners. Others – like StudyBreak Media, whose sites reach college-age students – implement multiple partners, a mix of exchanges and buying platforms.

StudyBreak Media's Downinghall recommended adding each partner slowly, to allow time for both QA fixes and to see the performance impact of each partner – made via year-to-year comparisons.

“Honestly, we’re not able to run an A/B test,” Downinghall acknowledged. “I believe this tech is beneficial for the publisher but it depends on how you’re set up and how tolerant you are to commit for testing.”

StudyBreak Media added its first pre-bid tag last fall and is finalizing its fourth partner now. Thus far, 25% to 30% of impressions offered by StudyBreak Media are picked up by header-bidding partners.

Tribune Publishing, on the other hand, only works with Rubicon. Lori Tavoularis, Tribune’s VP of programmatic and platforms, said her company gets more value by minimizing tech partners because it wants to maximize yield without detracting from its premium offering.

“What’s good for a digital-only publisher might not be good for a multiplatform media group,” Tavoularis said. ”Everyone has their own priorities internally.”

That said, she’s continuing to evaluate additional partners, and is contemplating bringing on a mobile-specific header-bidding partner.

One problem: Using more than one partner makes it possible for an advertiser to bid twice on the same impression, and it’s not clear how to tell when or how often that happens.

“We think there will be backlash or resistance on the buy side,” said Kaylie Smith, head of Rubicon Project’s Seller Cloud. “If you keep it to one to two partners, you’re negating the issue.”

There’s a resource consideration as well. Header bidding requires time and attention from the publisher’s ad operations team. After implementing a new tag, ad ops staff must monitor for latency. Header tags load before content, so a breakage in that chain means audiences won’t be able to read anything.

CaféMom and StudyBreak Media said that while these breakages happen only a few times a year, it’s important to have protocol in place that will pause the errant tag quickly.

Negotiating Header Bidding Setups

When bringing in header-bidding partners, publishers should know they are the ones with leverage, according to Prohaska. Because header bidding exposes buyers to 100% of inventory, publishers should expect to be compensated for that access. Both sides of the transaction need to get equal value.

Many buyers offer to pay premium prices to participate in header bidding, said Erik Requidan, VP of sales and programmatic strategy for Intermarkets, which supports header bidding as part of its offering to publishers.

At the same time, publishers should exercise particular caution when working with buying platform (as opposed to an exchange). Buyers tend to pursue sellers, instead of the other way around, an indication of the value they perceive in these arrangements.

If a header-bidding partner picks up impressions for a fixed price, it could cherry-pick the publisher’s best inventory. One way to ensure a fair value exchange is to require a buyer’s bid to compete in an auction with the rest of the buyers once it reaches the ad server. Then publishers don’t end up selling their most valuable inventory for what amounts to a discount.

StudyBreak Media is wary of allowing advance bidding buyers to buy at a fixed price and opt out of the auction.

“If a buyer wants something at a $5 CPM, it’s a valuable impression,” Downinghall said. “Who’s to say someone doesn’t want it at $25? This is an auction, and that’s the whole point of an auction.”

Header bidding also might give buyers and exchanges too much (from the publisher’s perspective) understanding of site traffic.

Fear of data leakage has especially slowed down commerce-based publishers, Rubicon noted.

“The biggest thing we’ve been worried about is URL-level access to the site, which we restrict,” explained CaféMom's Bannister. CaféMom works with multiple header-bidding partners. As Bannister noted, gaining visibility into URLs can let advertisers buy valuable sections of a site without paying a premium.

Header-bidding setups help publishers fulfill private marketplace campaigns. CaféMom runs its private marketplaces through Index Exchange so those buyers can look at impressions before they’re winnowed away by direct-sold campaigns.

Dentsu Aegis' trading desk, Amnet, pays attention to header bidding when working with private marketplaces.

“It helps when you have very strict impression goals to deliver, and you want to make sure the private marketplace is going to scale up,” said Vincent Bareges, director of publisher development.

Header bidding solves a buyer problem: the need for access to a publisher’s best inventory. And it solves a publisher problem: the loss of programmatic revenue because of ad server setups that reduce competition.

That mutual benefit is the surest sign that more publishers and buyers will seek access to header tags in addition to the hundreds of publishers that have already implemented it.

8 Comments

  1. Rob Byrnes

    No matter what happens with "header bidding" the decision still lies with the ad server that uses hard rules of priority settings or price to decide. The ad server still does not know when a $5 audience based bid is more perishable than a $10 contextual direct sold ad. I'd love to see ad server logic based on perishability - that'll significantly raise yield and win rates.

    Reply
  2. Happy to see AdExchanger writing on this topic!

    This is a hugely important shift in how publishers optimize revenue between channels and the best article on this topic to date, especially in covering some of the potential considerations as publishers integrate these new solutions.

    It's been awhile since I did much writing on my blog, but I blew the dust off to write some tactical posts around how header-bidding works, and how publishers are operationalizing with this concept. Hopefully the ad ops people in the room find them useful -

    http://www.adopsinsider.com/ad-exchanges/diagramming-the-pre-bid-redirect-path/
    http://www.adopsinsider.com/ad-exchanges/pre-bid-holistic-ad-serving/

    Reply
  3. You can't write about header bidding, or prebid, without writing about where to slot these pre-bid partners in the stack. Most people assume that with a pre-bid set up, you're still going to stick them in the standard slot in the ad server, just above open marketplace. However pre-bid gives a publisher full visibility into their overall demand, allowing a publisher to identify demand opportunities that can safely be placed above direct. To benefit marketers, and still maintain high yield and fill, you can use fewer partners and still achieve excellent results.

    Reply
  4. Thanks for a great post and bringing up such a burning sell side problem.

    Header bids are important but they only solve or add a portion of publishers yield value. The optimization of those slots across multiple partners and exchanges in real time is where we see most of the additional value being created.

    In today's complex adtech eco system, machines not humans, can cope with the vast amount of data and optimal yield combinations to succeed.

    Reply
  5. Advance bidding has been around for a while but is just starting to gain momentum as publishers recognize the value of a unified auction in their adserver from the diverse set of partners they can be working with. We've built a tool to make advance bidding/header bidding implementations themselves easier to integrate/manage and less risky for publishers, as page latency, data leakage, accountability, transparency are real issues for those hosting pre-bids in this fashion.

    Reply
  6. Vitor Nagata

    Really enjoyed the article and how it covered the pros and cons of Header Bidding.

    The only thing I don't agree with is that it's mutually beneficial.
    It might solve for ONE publisher problem which is loss of programmatic revenue because of waterfall impressions, but in the long term it pushes all the risk to publishers because SSPs/Exchanges/AdNetworks that are cherry picking impressions are using the most valuable inventory a publisher has, at a set price (specially adnetworks like Criteo), and leaving the less premium inventory to be delivered to the publishers direct sold campaigns.

    Data Leakage and Audience Mapping is also a big big issue. One of the most valuable assets for a publisher is its own audience and data and by allowing a buyer to track all its users they are just giving it away for free.

    In an ideal world all demand sources would be bidding in real time against each other through a unified platforms (using a net CPM bid instead of a gross CPM bid as some do today) but that would require SSPs/Exchanges to integrate with each other and with all major ad servers and it doesn't seem that any of this is coming any time soon.

    However, ad networks/direct buyers already have access to a publisher inventory available on SSPs through programmatic direct or RTB, so I don't believe publishers should let them do Header Bidding unless there's a SUBSTANTIAL guarantee for them. Otherwise, they'd trading a much bigger decline of long term revenue for a small increase in the short term.

    The last but not least problem is user experience, with all the added latency publishers will get from multiple tags being fired in the header, there's a bid change users will have a suboptimal experience and there's also a big impact on publisher's viewability rates which is one of the hottest topics today.

    At the end of the day, I believe publishers should chose a preferred SSP and try to concentrate as much as they can by pushing all their premium programmatic deals to a single platform. This way all their most valuable (higher CPMs) deals are managed centrally and the rest shouldn't represent that much money.

    Reply
  7. I'm confused by the suggestion that exchanges offer differentiated demand, and therefore if you aren't trying some supped up (and inefficient) waterfall to get all of the demand in the ecosystem competing you are missing out as a publisher. Aside from one notable exception (AdX with Adwords), every exchange largely has all the same demand (at least in desktop and mobile web display world). So a publisher integrates with one exchange, has it compete with all of their direct sold they are willing to have it compete I'm confused by the suggestion that exchanges offer differentiated demand, and therefore if you aren't trying some supped up (and inefficient) waterfall to get all of the demand in the ecosystem competing, you are missing out as a publisher. Aside from one notable exception (AdX with Adwords), every exchange largely has all the same demand (at least in desktop and mobile web display world). So a publisher integrates with one exchange, has it compete with all of their direct sold they are willing to have it compete with, and you are maximizing competition for every single impression.

    I am ignoring the other elephant here about whether a first or second price based exchange might yield more for publishers (and the resulting trend among publishers to work with multiple exchanges to help set up a waterfall of floors, effectively) but pubs aren't doing this because different exchanges offer different demand (again, aside from the aforementioned (and significant) exception).

    I would argue, and was considered here, that "header buyers" see advantages in getting preferred access to inventory and getting rich data profiles by having their tags on every page for every user, and only for that reason do they spend so much effort convincing pubs to do it, despite that it is actually a less efficient way for publishers to maximize yield.

    Reply
    • This should read:

      I'm confused by the suggestion that exchanges offer differentiated demand, and therefore if you aren't trying some supped up (and inefficient) waterfall to get all of the demand in the ecosystem competing, you are missing out as a publisher. Aside from one notable exception (AdX with Adwords), every exchange largely has all the same demand (at least in desktop and mobile web display world). So a publisher integrates with one exchange, has it compete with all of their direct sold they are willing to have it compete with, and you are maximizing competition for every single impression.

      I am ignoring the other elephant here about whether a first or second price based exchange might yield more for publishers (and the resulting trend among publishers to work with multiple exchanges to help set up a waterfall of floors, effectively) but pubs aren't doing this because different exchanges offer different demand (again, aside from the aforementioned (and significant) exception).

      I would argue, and was considered here, that "header buyers" see advantages in getting preferred access to inventory and getting rich data profiles by having their tags on every page for every user, and only for that reason do they spend so much effort convincing pubs to do it, despite that it is actually a less efficient way for publishers to maximize yield.

      Reply

Add a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>