Bhutani also has to consider how a given test will impact his team’s resources. In most cases, LendingTree will ask prospective vendors to run the test as a managed service, at least at the beginning, to see how the results look before deciding to invest the time to learn how to use the tool internally.
“We’re always in discussion with vendors for new solutions, but only a small number of companies pass through our filters,” Bhutani said.
Cost is also an issue. Although a portion of LendingTree’s marketing budget is allocated to testing with the assumption that some percentage of the experiments won’t pan out, running tests too often could negatively impact the bottom line.
It’s a bit of a balancing act, said Rob Deichert, managing director for North American operations at Criteo, which has served as LendingTree’s main retargeting partner since going through the testing process several years ago. (LendingTree also works with DoubleClick Bid Manager, the Google Display Network and Yahoo Gemini.)
“How frequently you test is something that every advertiser needs to think about,” Deichert said. “Testing a lot is great, but there’s a real cost to that. You also need to consider what the testing does to your customer experience. If the vendor you’re testing doesn’t perform as well as what’s already in place, that means lost sales.”
The testing methodology is generally fairly straightforward – head-to-head bouts in which one vendor is pitted against another on a level playing field. The advertiser does a hard cookie split of its users and gives each vendor the same amount of budget. Whichever one generates the most leads wins.
“We gravitate to where we’re getting the best return for the least effort,” said Bhutani. “It all depends on the ROI.”
Although Criteo doesn’t often find itself retesting with existing clients, if a vendor wants to win news business, participating in tests is de rigueur, especially in a sphere of the ad tech space as well-established as retargeting.
“Most potential clients already have a retargeting solution in place – I’d find it shocking if there was any digital marketer out there not doing retargeting in at least some form,” Deichert said. “And for those, we generally have to go into some sort of head-to-head test against the incumbent.”
For its part, LendingTree isn’t looking for a piddling improvement. A potential vendor has to make it worth his while, Bhutani said.
“I wouldn’t put a hard, fast number on it, but I would not consider a vendor that was talking about bringing a 5% return, let’s put it that way,” Bhutani said. “It has to be a potential game changer for us to do it.”
LendingTree is in the process of testing native solutions and dynamic creative providers to help improve cross-sell on its growing product line, which in 2015 expanded from mortgages to incorporate offerings like personal loans, student loans, business loans and free credit scores. Mobile and video are also on LendingTree’s ad tech testing docket.
“Given how much the industry is changing and how fast trends are shifting, we need to be ahead of or at least in step with what’s happening in the market,” Bhutani said. “We’re always keeping our ear to the ground.”