The Real Costs of Real-Time Bidding (RTB)

"Data Driven Thinking" is a column written by members of the media community and contains fresh ideas on the digital revolution in media.

Data-Driven Thinking - Rob LeathernToday's column is written by Rob Leathern, CEO of CPM Advisors, an online advertising optimization company.

In late 2009, the story of Real-time bidding (RTB) in the display ad space appears to be conforming to a pattern familiar to those in the software and media industries: the hype cycle (http://en.wikipedia.org/wiki/Hype_cycle). We have reached the "peak of inflated expectations" and some might say we are now on our way to the "trough of disillusionment". Companies on the sell side have announced real-time bidding systems before they existed, or that are actually something quite different (not real-time, not bidding, or neither). Some on the buy side publicly stated they were doing RTB with certain partners before it was even possible, others have publicly and privately said different things. For the record, for our CPMatic buying platform we have been building and testing real-time bidding and related cookie management infrastructure, but it is not (yet) bidding on
real-time inventory as of the writing of this. We will, but will probably NOT put out a press release when we do, though.

There is no doubt that the real story of the value behind RTB is centralized price discovery and streamlined inventory access (and the ability for this to happen at scale), and this will take some time to be realized in a meaningful way and will take widespread publisher adoption of new selling platforms. In the near term until publisher adoption increases though we’ll probably have a bunch of companies bidding in real time against each other for 15% of the ad inventory out there! In some ways RTB may be a red herring that might distract us from the underlying goal of creating a centralized market mechanism that provides quick access to inventory that gets the owner of the inventory the best price for it – RTB could cynically be seen as a way for ad exchanges/hubs outsourcing their decisioning and infrastructure costs to others. More about this later.

One thing I haven’t seen or heard a lot about is the costs of RTB and the implications for ad seller and exchange business models. Greg Yardley, CEO of Pinch Media and a former Right Media product manager (and a former colleague of mine and disclosure an advisor to CPMa) blogged it in September 2009 which I will reproduce here verbatim as a nice summary of some of the issues:

This sounds pretty expensive to implement vs. auctioning based on predefined rules. First, you’ve got to make a decision about every impression. If you’re using real-time bidding, and you’re buying one impression out of a hundred, it’s got to pay the cost of the ninety-nine other decisions not to buy.

Next – someone correct me if I’m wrong, but I think you’ve got to have as close to real-time reporting as possible. There’s not much point making decisions in real time if the info you base them on only gets updated every few hours. Real-time reporting at scale is neither easy nor cheap.

Finally, your models have got to account for the data that can’t arrive in real-time, or you’re going to put your efficiency gains at risk. For instance, click and impression fraud is generally filtered out by the exchange after the fact, but before your reporting’s updated. CPA offer conversions take time to trickle in- forms can’t be filled out instantaneously.

Add all of the above up, and you may be wishing you stuck with your predefined rule sets.

Real-time bidding promises the marvelous ability to jump ahead of the problematic daisy-chain and buy exactly the user and ad space you want, at the price you want. But in so doing it will completely change the way different companies buy and charge advertisers for media. Why? Because a buyer’s infrastructure costs will potentially become decoupled from the costs of actually delivering an ad to a user. Advertising agencies have passed on a lot of their costs to clients, (sometimes with hefty markups e.g. 15c CPM adserving anyone?) but they’ve always been able to point to some kind of media delivery (and in some cases creative creation) as the underlying deliverable to which they attach costs.

Pay for my infrastructure please

Right Media’s (RMX) infrastructure has buckled under the strain of over 9 billion daily impressions. Latency in adserving and pixel serving abounds, and reporting outages have been common. Strange and shadowy undocumented limits on various things like targetable number of line items, confusing decisioning inconsistencies etc. And yet it still functions fairly well at huge scale, for all that, when you consider the immense work it is doing. What is clear though, is that it has been hard to make this kind of system scale and so instead, a different approach might be to have a much simpler dumber center with the complexity in decisioning pushed out to the buyers. The exchange pushes some of its costs outward to bidders, but in so doing instead of one entity having to manage the hardware for bidding, buying and adserving, everyone would to have their own version. This is where the math starts to get both interesting, and confusing.

Lets look at some hypothesized costs of bidding. Based on some rough estimates, assume it costs 50% of the cost of serving an impression to send a bid (fully burdened with maintenance costs and so on it may actually be higher, but let’s just keep it simple), so if you spent $0.02 CPM on adserving costs at scale, you might expect to pay a $0.01 CPM to bid, per thousand bids. So now, the biggest question becomes what percentage of the bids you make do you actually win?

The RTB systems let (or will let) you "pre-bid" – create a filter bucket that narrows the range of inventory you will be bidding on. This could be broad, like US only, or very narrow for example only targeted to a specific user cookie or segment created by you or by a data provider. Presumably you would want to bid on a fairly wide range of traffic, looking to find your ideal audience and take advantage of the wonders of RTB. Because if your bid range was really narrow then it begs the question – why bother to do real-time bidding at all?

Hypothetically let’s assume you bid on a broad range of traffic and win 5% of your bids. In this case, if you end up buying 500mm impressions your costs look like this:

pic1

Your serving costs at 2c end up being $10k but your bidding costs eat up $100k! If you are an ad network with margins (like the public comps) of say 40-50%, in this example say you are bidding on normally priced (not super-cheap) traffic at a $2.00 CPM cost and charging clients a nice markup to sell it at $3.50;

pic2

Assuming RTB gives you more/better access and campaigns perform better, you should see better client retention and performance but at the cost of lower margins. Here the margin for this hypothetical ad network went down from 42% to 37%. But if the traffic they were bidding on was cheaper, say they were looking for a 50c CPM campaign and paying 88c (same margin pre-serving or bidding costs), then their margin is dramatically impacted by these costs:

pic3

So in this case real-time bidding becomes less about finding the cheap inventory, and more about bidding on slightly more expensive inventory so that the bidding costs don’t depress margins too much. This is the case for an ad network, who has already historically had to bake in underlying technology costs into a single (usually fixed) CPM price. For an agency-type business model, the agency would presumably need to pass along the bidding costs to the client… and this is just the costs of bidding. Other costs for maintaining a real-time infrastructure are not accounted for by this simple set of calculations.

This is an admittedly simplistic analysis, not taking into account the uptick in results from getting more specific inventory. But it also doesn’t take into account the additional infrastructure investments needed to get those benefits. Also, having centralized price discovery and streamlined inventory access would appear to provide many of those benefits even in a static pre-bidded environment instead of in a real-time bid environment.

In summary, RTB is still a work in progress both in terms of business model AND technology. Therefore clients who are working with vendors who are doing real-time bidding on their behalf (or plan to) will need to ask questions not only about how the tech works, but how it gets paid for (out of whose pocket) as well.

Follow Rob Leathern (@robleathern), CPM Advisors (@cpmadvisors) and AdExchanger.com (@adexchanger) on Twitter.

12 Comments

  1. Nice article, Rob. Interesting to see a breakdown of the costs, however I disagree with the a few assumptions you make.

    1 - "There is no doubt that the real story of the value behind RTB is centralized price discovery and streamlined inventory access" - to me the value is around making bid decisions based on proprietary data and models. RTB itself doesn't centralize anything. There's nothing stopping premium networks from sending their own data stream of biddable impressions to RTB platforms - could be an interesting model.

    2 - Exchange filters aren't important because they reduce the number of impressions seen. Between exchange-side geo and context filtering, cookie mapping and 3rd party data, a DSP may not need to see billions of impressions that they know nothing about.

    3 - Bidding is expensive. Very interesting that RTB / spot markets for individual impressions are coming to fruition at the same time that enterprises are getting comfortable with cloud computing. EC2 instances cost $.08 - $2.40 / hour. Cycles cost much less at scale. Google (and the other markets that support RTB) are incentivized to make bidding quick and accessible to as many parties as possible.. they also operate one of the largest cloud infrastructures. I'll be surprised if Google doesn't tie App Engine into AdX.

    Your comment that centralized price discovery and inventory access can achieve the same benefits is interesting... but how sophisticated do the pre-bids need to be, and how much horsepower is needed to generate them, to be as efficient as RTB? Are those efficiencies enough to bring publishers to a centralized pricing and discovery system?

    Reply
    • Some good points certainly, Marc. On this topic I wish I had the luxuries I used to have as an analyst (though unfortunately writing overly long 40-page reports was something I got stick for back then :-)) to write much much more. A few quick thoughts though:

      The tradeoff between "pre-bidding" (static statement/filter of what you want) and actually bidding for something is very important, crucial. If you win few bids by bidding across a lot of inventory, your costs are high. If you pre-filter aggressively, then one may ask what the point of bidding at all is - except in cases where people are bidding against each other for data provided by specific data providers (less so retargeting, if that is mostly where an advertiser hands that to a single vendor) - so a bunch of folks will bid aggressively on the auto-bmw intender data etc.

      The other thing to keep in perspective that I mention with my comment about 15% of inventory, is that not only is the % of inventory going to RTB going to be small for a while, it's also going to be non-premium. Today that's typically all about the 6 to the nth impressions on a site. (That fact itself is not well priced in current systems BTW)

      Getting premium inventory out of daisy chains and to be universally addressable and biddable is really the goal (for me and our advertisers), whether that requires RTB, or something else entirely.

      Reply
  2. Rob I think you are correct that the major aspect to be excited about is not the real time aspect of the exchange but the benefits that the centralization of vast amounts of inventory into a liquid market everyone can access will bring. Google's search auction proves the benefits of having an easy to access marketplace with enough parameters for marketers to segment from and drive their business.

    Real time bidding matters when you have individual cookie level data and want to buy on that unique impression event. But I can think of many cases where you would be willing to reserve impressions in blocks for common data such as DMA/location, URL, site, etc. Perhaps a hybrid approach where most of the auction functions in set prices for clear segments (like keywords are to search) with special over-rides to bid for that exact impression itself based on unique data (re-targeting, 3rd party sources) would reduce the technological implementation cost and bring us closer to the end goal.

    Reply
  3. Rob you definitely have a good perspective here but a few things crossed my mind while reading your article.

    - Hardware and software costs go down over time. Yes ad serving was 15 cents at one point but today it's almost free. RTB allows for a lot of efficiencies on the human side where costs only go up. There's no need for ad tags to be emailed as attachments and phone calls to be made to adjust CPMs. The ability to dial up and down spend on the fly and on a per impression basis can save a lot of money elsewhere. These costs are harder to quantify but they are material.

    - The model of assuming serving costs for impressions that are unfilled is not new. Any tag that uses passbacks follows the same cost structure. With some major ad networks filling at 20% on average, these costs have existed for a long time and are not unique to RTB.

    - Another misconception about RTB is that the way it's implemented is by the buyer plugging in to a huge pipe and seeing all impressions. I don't know of a supply platform that is working in this way. The reality is that there is pre-filtering with RTB too.

    - The largest contributor to RTB hype is the assumption that it's a silver bullet. The reality is that we're all solving an enormous problem and RTB is another arrow in the quiver and we need all the arrows we can get. It is going to fill a large need for both buyers and sellers but will not be the entire solution. For publishers, mixing RTB with tag buys and audience data is going to be key for monetization in the coming years.

    Looking forward to seeing the RTB solution from CPM Advisors come to life!

    Reply
  4. I second what Brian says. We have been bidding in real time for close to a year now on multiple exchanges and have not see the costs bear out the way Rob describes them. If you build your system correctly the costs of bidding are vary outweighed by the money saved in efficiency and effectiveness.

    Reply
  5. Very good look at RTB but I thought one of the primary reasons for real-time bidding systems was to create new efficiencies at scale yet these new efficiencies are not accounted for in your calculations. Factor them in and this is a very different story. I have to second Zach's experience, we have also not seen the economics of RTB play out this way, quite the contrary in fact.

    Reply
  6. Yes efficiencies are not calculated here as I mentioned. My point is to be simplistic to get people thinking about it. I'd love to see one of you guys take a cut at the estimated savings for us all to see.

    Reply
  7. A few thoughts:

    -For most large scale ad servers, variable serve cost is less then $.01/CPM.

    -The largest variable in this cost is normally bandwidth (of which there is essentially none for bidded impressions not purchased).

    -The second largest variable cost is normally related to reporting (of which there is none or essentially none for bidded impressions not purchased).

    -I believe pre-serve decisioning / bidding is a minority component of the sub-$.01/CPM variable serve cost for most ad servers (so think $.001/CPM range as a ballpark for a RTB function - and that's without optimizing for a RTB environment, simply using more legacy ad server processes and their associated costs).

    -This cost will be included into the smarter players algorithms, so that when it becomes cost-prohibitive, these undifferentiated / low success rate / less valuable impressions are filtered out beforehand instead of real-time decisioned on. A smart algorithm could handle automatically bucketing impressions into larger groups, and understanding which of these it is cost-effective to RTB on, and which are not cost-effective to RTB on and therefore should be excluded or fall under more broad non-real-time bid strategies.

    Reply
  8. Rob,

    excellent article. I'm curious where you see the Open X spot market on the continuum between RTB and presold campaigns.

    Also, you write "Because if your bid range was really narrow then it begs the question – why bother to do real-time bidding at all?"

    I can see 2 reasons. One is, as you mention - price discovery : you must probe the spot market to know if you are paying a reasonable amount for presold campaigns, and the other is geotargeting - if I want mobile browser impressions within 100m of a Starbucks location, is a presold campaign really going to work for me right now ?

    Reply
  9. I agree with the real time bidding statement being 'puffery' and more costly.

    As an advertiser first, transparency is king. My buying decisions are made in a group at a table in the marketing conference room. Before I even make a proposal to the group I need to know as much as possible - targeting, sites, prices, etc...

    Even getting content out of daisy chains I don't think is an advertisers concern. Its these daisy chains that create a innovate market with new ways to reach customers. Its its the different targeting technologies, grooming algorithms, etc that are birthed from the way this market is tailored.

    Reply

Add a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>