As marketers seek more transparency into their programmatic buys, they’re bypassing their demand-side platforms and going straight to exchanges and publishers to ask for log-level data about the bids they’ve won and lost.
More than a dozen of the most hands-on, sophisticated marketers – including P&G, L’Oréal, T-Mobile and Bayer – are leading the charge. Select agencies, including Havas and IPG’s Magna Global, have also requested log-level data from exchanges.
Initially, buyers wanted to audit the buying path to ensure exchanges weren’t doing anything untoward. But now, buyers can use this data to prune how they access digital inventory. Log-level information helps them do supply-path optimization and whittle down who they work with based on performance and fees.
Advertisers are planning to use this data to power their buying strategies. Buyers could find pockets of undervalued inventory though log-level analysis and make more intelligent decisions than competitors who don’t have access to this data. Granular pricing information could inform negotiations in direct deals with publishers.
Agencies could create planning and forecasting tools around programmatic inventory with better visibility into the supply side.
But these potential applications are still nascent, according to more than a dozen sources AdExchanger spoke to for this story. Gaining access to this data and analyzing it is a long-term, expensive, multi-year project.
And it’s a project that’s ruffled feathers even as it lays the groundwork for a more mature phase of programmatic.
Why is log-level data so important?
The first marketers embarking on this project started making formal requests for information (RFI) last year to exchanges like Index Exchange, OpenX, PubMatic, Rubicon Project, Sovrn, Telaria and Xandr. These exchanges started sending regular data feeds over to buyers to audit and analyze this year.
Google’s exchange is the only exception, refusing to send over log-level data to buyers in favor of aggregated information.
The log-level data shared by these exchanges includes impression-level details like ad size, geo, session depth, URL, placement, browser, and the “bid landscape,” which describes the number of auction participants and the price they bid. Exchanges have also been willing to share this level of detail for impressions that buyers lost, a previously inaccessible data point.
But identifiers are not included. Every exchange and publisher who sends this data to buyers attested that they stripped out uniquely identifiable information from the log files, citing privacy concerns.
Buyers also want impression-by-impression fee information. Fees can differ from one impression to the next, even on the same publisher, because of different contracts and costs associated with different programmatic deal types.
Some of this log-level data is available through the DSP – but not all of it. DSPs often don’t ingest every field the exchange fills out. And they usually don’t know why a buyer lost a bid or what fees are taken out.
So getting data directly from an exchange, which sees more than a DSP, gives marketers a glimpse at the bigger picture. They can audit the supply chain by monitoring any discrepancies. And they might understand an exchange’s access to publisher inventory better than if they purely looked at the exchange through the lens of their DSP.
How advertisers obtain log-level data
Analyzing log-level bid requests requires setting up data feeds between an exchange and marketer, two groups that until recently rarely worked directly with each other. Establishing a business relationship poses both legal and technical challenges.
From a technical standpoint, an exchange’s log-level data far exceeds what can fit in an Excel spreadsheet.
To transport such large amounts of data, exchanges often set up cloud-to-cloud connections.
Two exchanges productized these connections. Early this year, Rubicon Project rolled out a product to share feeds of log-level data with buyers, said Adam Soroca, Rubicon Project’s head of buyer team. Index Exchange publicized its log-level data sharing tool in September.
Once they have the data, buyers pay to store this data, and find analysts to process it.
Accessing log-level data also requires legal work to set up.
Brands and exchanges hold contracts with their DSPs, not with each other. Some buyers have been told that they don’t have the right to access any of this data because there’s no direct contract between the brand and the exchange.
“There has been some pushback from SSPs about using publisher contracts and relationships in order to obfuscate what the publisher fees are, which in today’s market I find unacceptable,” said Havas EVP and head of programmatic Andrew Goode.
To establish a relationship, marketers typically sign an NDA with the exchange and set up terms for data sharing. Buyers need to make sure they aren’t violating their DSP agreement by getting data from the exchange. And exchanges need to make sure they’re not violating contracts with their publishers, which often mandate the exchanges keep fee data private.
Exchanges are often eager to share data. AppNexus for example could share fee data with buyers.
Exchanges who receive these RFIs see an opportunity to cozy up to brands that are making decisions about what exchanges to work with – and which ones to cut.
But with that carrot comes a stick: Exchanges that don’t share data fear they’ll end up on the chopping block as buyers winnow down who they work with to include only transparent exchanges. The largest exchange of them all, operated by Google, likely doesn’t feel the threat of being sidelined.
Marketers and agencies that have asked Google for log-level data have been told that since they don’t hold a contract with Google’s exchange, they can’t access the data. On occasion, Google has shared aggregated fee information. Though Google’s decision is hardly a surprise given its tough stance on data sharing, it nevertheless has exasperated some buyers.
The pickle for publishers
Publishers are also fielding RFIs from brands who want access to log-level data, and hearing from buyers that are processing exchange data. They see this trend as either a threat or an opportunity.
Some publishers want buyers to understand the path to their inventory better and believe both sides will benefit from greater transparency.
Plus, when buyers request log-level data specifically from publishers, they can tie access to media commitments. Because of the complexity of data sharing, publishers won’t provide log-level information to anyone but their top spenders.
But even those publishers have limits to what they’ll share. One brand asked a publisher to share the direct-sold CPM price for individual impressions. But the publisher refused to share that proprietary pricing data.
Other publishers fear data sharing could put them at a disadvantage. If buyers see impressions clearing at lower prices than what they’ve been paying, they can use that leverage to negotiate lower prices for private marketplaces or direct deals.
Yet, some buyers do this already, using DSP data to compare bids on the open marketplace to direct deals.
The difference is that exchanges share data about auctions a marketer lost, not just ones they won, which could enable brands to do pricing analysis on a much larger scale and with greater precision.
But as much information as exchanges have about pricing, there are certain details that buyers won’t see in the logs.
For instance, publisher inventory in logs might show two seemingly identical 300x250s placements. But one might be historically a much better place for an advertiser to buy – and only accessible via a higher price in a separate marketplace. These nuances in inventory differences require working directly with a publisher, not just directly with an exchange.
What buyers will do next
Brands undertaking the large task of analyzing log-level data shows that programmatic is entering a new era of maturity.
Marketers don’t want to just use RTB, they want to know what’s inside an RTB bid request.
Marketers with in-house programmatic teams are propelling this trend because they have more budget flexibility to hire data analysts or pay to store data. Many also see the strategic importance of auditing their supply chain. These projects could create a competitive edge for marketers who use this data.
Agencies see log-level data access as a way to deliver value to their clients. They can increase their negotiation clout and ensure transparency for clients, especially for buyers that don’t have the resources to do this type of analysis themselves.
The big downside to processing log-level data is the cost and resources involved, which put it out of reach to all but the most sophisticated buyers. Fortunately, working with this data does not necessarily need to be an ongoing project.
Havas, for example, pared down the exchanges it worked with this year and is now establishing a legal code of conduct it wants all of its partners to sign – all big transparency projects aided with the transparency it received from log-level data. But now that it’s processed log-level data enough to gain confidence in how auctions work, it’s asking exchanges for more aggregated information that it can internally process more quickly.
“Log-level data has been held up as the panacea, the gold standard. That might be a false economy,” Goode said.
As buyers continue to request data from exchanges, the process may become even more turnkey than standardized data sharing and legal agreements. Another option is for DSPs to ingest more information from exchanges in the future, making it more accessible to anyone using a buying platform.
But first, marketers must establish trust directly with exchanges, and getting log-level data is the most bulletproof way to do it.