Google gave the world an unintended taste of an ad-free internet this week. And publishers who rely on the tech giant’s ad serving software were reminded just how much power Google has over their ad revenue.
Google Ad Manager (GAM) was out of commission for about three hours Thursday evening across web, app and video inventory. Any ad placement served through GAM, even ads monetized by a non-Google SSP, went dark, leading to lost revenue for publishers of all sizes at a time when they can least afford to lose out on revenue.
GAM service has since been restored, and Google has identified and corrected the problem, according to a Google employee who worked to restore its service.
But the outage left publishers “shooting blanks across their pages,” said Matt Barash, SVP of Americas and global publishing at Index Exchange. It was “the longest [GAM outage] in memory and an eternity by Q4 standards,” he said.
The worldwide GAM outage began at about 8:06 p.m. ET and persisted until 10:40 p.m. ET, according to a Google Ads Status Dashboard update. However, the company is still investigating the full scope of the outage and will be sharing an update soon, said Erica Walsh, communication manager for Google Ads.
Some publishers said they began seeing indications of an outage starting around 7:04 p.m. ET.
Publishers saw their served impressions and revenue through GAM drop to zero. One publisher, who asked to share data anonymously, shared a chart showing how its revenue was affected by the outage.
“Given the number of publishers GAM serves globally, this time of year being peak for digital ad spend, and macroeconomic factors that have caused ad spend to be less than we’ve seen in previous years, it’s safe to say last night’s outage will have a significant impact on publisher revenue,” said Paul Bannister, chief strategy officer at CafeMedia. “Every dollar counts right now.”
But as bad as the timing was, it could have been worse. “Luckily this was late at night relatively for US customers, so the traffic should have been a little lower and the big campaigns should have run for the day,” said Scott Messer, an ad tech expert and industry consultant.
Assuming a consistent number of impressions per hour, the three-hour downtime cost publishers about 12.5% of their daily GAM revenue, or about 0.4% of their monthly GAM revenue.
However, 0.4% of a large number is another large number, so Google is unlikely to offer publishers any make-goods for the loss in revenue “given the scale and spread of this,” Messer said.
According to GAM’s standard Service Level Agreement (SLA), Google guarantees 99% uptime for “Traditional Ad Serving” as calculated on a monthly basis. If a media owner’s downtime exceeds 1% for the calendar month, Google would offer a reduced service fee for the following month. Since this event’s downtime was only 0.4%, Google is not bound by GAM’s SLA to indemnify affected media owners.
Google’s uptime SLA of 99% is on the low end compared to IT industry standards. It is common for web hosting companies to offer uptime guarantees of at least 99.9%, said Krzysztof Franaszek, ad-tech researcher and founder of Adalytics.
While the episode illustrates just how much of the open web’s programmatic infrastructure relies on Google’s ad tech, publishers do have alternative ways to rewire their inventory in the event of an issue.
“I’d imagine publishers have a renewed focus on a backup plan in case of outages,” Barash said. “It’s important to hedge in an industry where exclusivity can often amplify risk and exposure.”
One example is Salon, which lost about $500 in revenue to the GAM outage before its ad tech partner Sovrn was able to stop the bleeding. “Ads were down for about two hours, but we were able to resume ad serving [on Salon] around 9 p.m. ET by bypassing ad requests to GAM and serving programmatic ads from our SSP partners for the remainder of the outage,” said Carson McGrath, director of ad management at Sovrn, which works with Salon to monetize its ad revenue.
In lieu of an ad server rendering ads, ads could also be rendered directly from Prebid or other header bidding solutions, Messer said. “So, while you won’t need a backup ad server, publishers may want to explore directly connecting to demand sources,” he said. “This may inspire some programmatic-only publishers to rethink their entire stack. Although it would be costly to skip the Google demand, it would be possible to monetize your site entirely without an ad server by rendering header bidding ads directly onto your site.”
Another approach would be to have a system in place that serves fallback ads from a direct deal that are embedded into a website’s code in case programmatic ads are not loading, Franaszek said. Safeguards like these are used by some publishers like the Washington Post, he added.
“Hard-coded AdSense and any video ad serving done outside GAM were still running,” Messer said. “Let’s not forget that the code-on-page juggernauts like Taboola and Outbrain were also still generating revenue for publishers.”
All told, the GAM outage is more fodder for antitrust arguments against Google’s stranglehold on digital advertising. “This event serves as a stark reminder of how dependent web publishers are on Google and the tight grip the company has on the ad-serving market,” Barash said. “It’s also a reminder of the critical importance of independent ad tech, the need for optionality and greater overall provider choice in our industry.”
But while Google may be one of the biggest targets for scrutiny in the ad industry, how it handled the incident speaks well for the company, Messer said. “This could have happened to anyone, and it was still a very fast recovery for a global outage,” he said. “It’s actually great to have such a solid technology partner operating the backbone of the internet’s revenue.”
And the incident provided some food for thought as to just how much the open web’s ad infrastructure slows down the internet. “This was an interesting experiment for how publishers can work toward making their websites and apps load much faster by optimizing parts of their digital advertising experience,” Franaszek said.
Publishers now have a unique opportunity on their hands to measure how their ad loads affect time on site and pages per session. “Google created an internet-wide no ads test,” tweeted Emry Downinghall, SVP of programmatic revenue and strategy at Unwind Media. “The Q4 timing totally sucks, but [it’s] an opportunity for pubs to look at what UX metrics were impacted during the outage.”
Update 12/10/22: A previous version of this article said Google Ad Manager has no uptime Service Level Agreement. GAM does have a 99% SLA for Traditional Ad Serving.
In addition, Google contacted AdExchanger to clarify that the Google Ads Status Dashboard update on the GAM outage is not the company’s official statement on the scope of the outage, and that Google is still investigating the incident.
