Home Data-Driven Thinking Marketers, The Free Token Lunch Is Over. Is Your Stack Ready?

Marketers, The Free Token Lunch Is Over. Is Your Stack Ready?

SHARE:
Jon Morra, chief AI officer, Zefr

Anthropic just dropped Claude Sonnet 5, reigniting a broader debate about how AI should be priced. Some argue that published token rates no longer tell the full economic story because changes in token consumption and model behavior, such as the use of reasoning tokens, can materially affect real-world costs.

Meanwhile, other alternatives like Fable 5 are priced at twice Anthropic’s prior top-tier model and no longer treated as unlimited plan features. Regardless of where you land in that debate, it’s becoming clear that the economics of AI are changing. Not to mention the actual, ever-changing pricing structures.

Most companies running AI-powered marketing tools are not ready for what this means. They built their systems assuming both tokens would stay cheap forever and the number of tokens needed to achieve an outcome would stay flat.

For years, AI builders and buyers have been rewarded for a pretty simple strategy: use the biggest model you can get your hands on. That worked while the cost of the hardware needed to run the models stayed within budget. It was easy to assume that more intelligence was always the right answer, but when you have to consider the cost of that intelligence, that is not always the correct, economical choice.

Now marketing needs to pay attention because this changes the conversation and the questions that marketers should be asking about their AI tools. 

Matching model to task

Think about the kinds of decisions AI is making across advertising today. Is this video suitable for a family-friendly campaign? Does this creator align with a brand’s values? Is this creative likely to violate a platform policy? Important questions, but they aren’t the same as asking an AI model to solve a complex legal problem or prove cutting-edge math problems.

Ramp, which tracks AI spending across tens of thousands of businesses, found that although the average cost per million tokens has fallen dramatically over the past year, AI spending continues to climb for two reasons: breadth and depth. On the breadth side, companies are using AI to solve far more problems than they were a year ago.

On the depth side, most modern AI models now ship with reasoning enabled by default. Reasoning uses many times more tokens to “think” about answers to problems before generating the actual output tokens that respond to the prompt. Cheaper tokens didn’t lower AI budgets; they encouraged organizations to consume many more of them, a classic case of Jevons paradox.

We are shifting back to unit economics, and marketers need to start looking at right-sizing models to the problems they actually need to solve.

Researchers are also beginning to evaluate AI in a different way. Instead of asking how much a token costs, they’re asking how much it costs to get the right answer. Researchers recently proposed a framework they call “cost-of-pass,” which measures the expected cost of producing a correct result. They concluded that different kinds of models are economically optimal for different kinds of work. Lightweight models excel on simpler tasks, larger models on knowledge-intensive ones and reasoning models earn their keep only when the problem is complex enough to justify the additional cost. My colleagues and I also studied this concept under the brand-safety paradigm, asking when to trust model outputs or escalate decisions to more expensive human review.

That’s exactly where I think enterprise AI is heading. We’re relearning that solving billions of repetitive decisions efficiently is a very different engineering challenge from solving one extraordinarily difficult problem. For engineers like myself, it can be challenging to overcome our shiny object system. But when a technology needs to work at scale, we must.

Per-token price for frontier models may go up or down, but the cost to solve complex problems is going to increase. Anthropic won’t be the last lab to make that move. The question every marketer should be asking their AI vendors this quarter isn’t “Which model do you use?” It’s “Why does every task get the same model?” If the answer is habit rather than architecture, that’s where the next invoice surprise is coming from.

“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.

Follow Zefr and AdExchanger on LinkedIn.

Must Read

Los Angeles, California - 26 February 2023: Reddit social media platform displayed on smart device

Reddit Is Going Login Only. Blame Google

Starting in Q4, Reddit will test logged-in-only mode to protect its user engagement, improve ad targeting and keep AI from scraping its content.

DoorDash Acquires Instacart As Consolidation Comes For Mobility Media

DoorDash on Thursday confirmed the rumors and announced its intention to acquire Instacart at a $17 billion valuation.

Meta’s $25 Billion Privacy Fine In New Mexico Is A Big Number. But Is It A Big Deal For Ad Tech?

Last week, a district court judge in New Mexico ordered Meta to pay $25 billion for misleading Facebook users about how it handled their data. Does it matter?

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters
Comic: Gamechanger (Google lost the DOJ's search antitrust case)

Google Earned More Than $100 Billion In Ad Revenue Last Quarter, But Is It Even An Ad Business?

Google earned $127.6 billion in Q3, up 25% year over year, practically all of which came from ads. Yet advertising hardly came up during its earnings call on Wednesday evening.

Flat Hand Holding a magnifying glass, find the person you want from many people. Find the right thing for you.

Meta Has A New Ad Ranking System Built On LLMs, Plus Upgrades To Andromeda And GEM

Meta now scores every eligible ad for every impression at more than a billion ads per second, one of three ad ranking upgrades Meta announced on Wednesday.

Marketbridge Acquires The B2B Advertising And Analytics Startup Channel99

The agency services group Marketbridge added to its portfolio on Tuesday by announcing its acquisition of Channel99, a B2B advertising and attribution tech company.