Blocking bots might feel like the only leverage publishers have left.
But keeping them out by default is a bit like unplugging the phone when your customers are trying to reach you, says Tim Frank, corporate VP for monetization, commerce and the AI economy at Microsoft, on this week’s episode of AdExchanger Talks.
“A lot of businesses right now are intentionally blocking these types of agents and systems, and it’s important to know that this is a very fast-growing channel,” Frank says. “If you’re not available, then you’re never going to be found.”
Bot traffic surpassed human traffic on the web for the first time in June, according to Cloudflare, which maintains a live tracker of automated versus human HTTP requests distributed to HTML content. Between then and now, the share of search requests initiated by bots increased from 57.4% to 58.6%.
But letting the crawlers in isn’t a guarantee that they’ll recommend you. Frank has a phrase for it: “Information is the new bid.”
In the AI era, he says, the brands that get recommended won’t necessarily be the ones with the biggest ad budgets, but rather those with the most clear, complete and transparent product data.
“Information quality matters a ton,” Frank says. “But I want to broaden the term information, [because] it’s not just what’s on a website.”
Agents get their information from multiple sources. They weigh a brand’s own claims against whatever they can get their hands on (if they had hands) – and they never get bored halfway through.
“For brands,” Frank says, “it’s not just their own narrative, it’s what do the reviews say? What are people saying in different forums? Agents triangulate this and it’s not only the brand’s message anymore.”
Which is why we’re starting to see more GEO and AEO startups come out of the woodwork. Some of them are slinging snake oil while others, to be fair, are just trying to roll with a rulebook that gets rewritten every few weeks.
But the cheapest and easiest way for brands to find out where they stand in AI search is almost anticlimactic in its simplicity: Just ask. Interview the models the way you’d interview humans in a focus group.
“Go ask each of the models why they’re doing what they’re doing,” Frank says. “It’s free research.”
Also in this episode: Why Frank is bullish on the Universal Commerce Protocol, tips for navigating the three eras of the internet (human, LLM and agentic) and Frank’s fascination with black holes and space, which led him to an early job building satellites.
For more articles featuring Tim Frank, click here.
