The dominance of Google's advertising markets

Google, dominant player in adtech

The subject of this post will be a little different from what I usually cover on this blog. Instead of observing personal data leaks on websites and apps, I will try to document, non-exhaustively because the abuses are numerous, how Google established itself across advertising markets. If you want to learn more, here are some recently published studies:

• The study by Dina Srinivasan: Why Google Dominates Advertising Markets.
• Studies by Damien Geradin and Dimitrios Katsifis: Google’s (Forgotten) Monopoly – Ad Technology Services on the Open Web, 'Trust me, I'm Fair': Analyzing Google's Latest Practices in Ad Tech From the Perspective of EU Competition Law and Competition in Ad Tech: A Response to Google.
• The study by Fiona M. Scott Morton and David C. Dinielli: Roadmap for a Digital Advertising Monopolization Case Against Google.
• The “OECD secretariat preparation notes”: Competition in Digital Advertising Markets.
• The study by the UK Competition and Markets Authority (CMA): Online platforms and digital advertising market study.

The days when advertising campaigns were created and managed "manually" (contracts by fax, sending ad banners and tracking elements by email, etc.) between agencies and publisher sales teams are over:

• Like financial markets, this management is now automated (“programmatic” or “RTB” for “Real-Time Bidding”).
• The technology that enables these advertising markets is called an SSP (Supply-Side Platform) or an Ad Exchange.
• Ad space is now sold one impression at a time: the publisher offers an advertising opportunity (displaying an ad on your device), and the advertiser is free to bid on that opportunity or not.
• The advertiser often has personal data about you when it receives this opportunity, allowing it to bid what it believes to be the "fair price".
• It competes with other advertisers, each going through buying bots called DSPs (Demand-Side Platforms).
• The auction happens very quickly, often in less than 100 milliseconds.
• Much less publicized, preferential agreements between advertisers and publishers (called "deals" in the adtech world) have not disappeared, but they too now mostly run through "programmatic" pipes.

These automated advertising markets are the source of massive personal data leaks; read, for example, Brave's GDPR complaint against Google and the IAB (the adtech lobby) on RTB. And they are dominated by Google. Here, for example, are the figures for the UK (the figures are similar in the other main Western markets):

Online Platforms and Digital Advertising Market Study (CMA July 1st 2020)

As a result, Google's chosen business model (surveillance capitalism) and its operational decisions (supporting web advertising monetization through these automated marketplaces, trampling the notion of consent, etc.) have a disproportionate influence on other players in the advertising web, who often have no choice but to enter the surveillance capitalism game as well.

What is the link between Google's domination and the health of the web in general? If we look at Google's annual reports, we can see that Google itself captures (Google Search, YouTube) an increasingly large share of the ad spend made through Google's buying tools, which are themselves already dominant.

In 2007, Google captured 64% of its advertising revenue. In the first quarter of 2020, Google captured 85%.

Consequence: publishers capture an ever smaller share of this funding (the share on which Google takes its tax) and find themselves deprived of significant revenue. We can cite the critical economic situation of the news sector, which leads to:

• Great precariousness among many journalists.
• A lack of resources to handle in-depth investigations, which are so critical for our democracies.
• Low media autonomy in the face of financial power. It is difficult to create a large profitable media outlet today; French media are mostly owned by a few wealthy industrialists, see the French media map produced by Acrimed and Le Monde Diplomatique.

Dina Srinivasan's thesis is that Google could never have dominated advertising markets to this extent if those markets had been properly regulated. She shows that many practices used by Google to favor its own business have been prohibited in financial markets. Google was therefore able to extend its web thanks to public inaction. The studies by Damien Geradin and Dimitrios Katsifis, and then by Fiona M. Scott Morton and David C. Dinielli, also detail Google's multiple conflicts of interest and lack of transparency.

Before 2007, Google was already an advertising giant

In 2000, Google launched Google AdWords (today Google Ads) to better monetize its search engine. It quickly became a success. Why?

• First, obviously, its highly effective search engine, appreciated by users (even if since 2000 Google has multiplied abuses of its dominant position in this area; take a look at the evolution of search results, it is quite striking).
• A “self-service” system where each advertiser (from a small local craftsman to a multinational) can create their advertising campaign quickly as long as they have a credit card.
• Performance-based payment: the advertiser buys the keywords it wants and only pays when the user clicks on the ad.
• More relevant advertising: in order to maximize its revenue, Google has an interest in favoring cheaper ads with better click-through rates.

In 2003, Google launched Google AdSense in order to extend its advertising network to third-party websites. What levers does Google rely on?

• First, on the fact that it already had a unique captive market: countless advertisers already on AdWords, small and large, with their credit cards. Elsewhere on the web, launching an ad campaign meant paying an agency.
• Second, on the fact that it already crawled the web, and was therefore able to provide relevant contextual advertising: making sure that ads from an advertiser operating in a specific sector could be shown on sites related to that sector (contextual advertising).
• Third, on the fact that for a “small” website without the means to create an ad sales team to approach agencies, there was little competition.

In short, it is also a success.

In 2007, the acquisition of Doubleclick allowed Google to obtain a key position among publishers

But large publishers' ad sales teams did not only use Google AdSense to monetize their ad inventory. They were wary of the quality of ads served by Google AdSense (the variety of advertisers also meant unwanted ads could appear), and did not necessarily like the ad formats (originally text only, no banners or videos). They also worked with other ad networks and, above all, had strong relationships with media agencies, allowing them to sell most of their ad inventory directly, at more attractive prices.

To manage all their ad campaigns (direct sales, sales through ad networks such as Google AdSense, or sales through new marketplaces), they used specialized tools called ad servers.

Likewise, the main media agencies did not only have Google search results to distribute ads for their advertiser clients. They needed a tool to properly manage and measure all their ad campaigns. This tool was also an ad server; note that its features are significantly different from those of a publisher ad server, even if they share a common foundation.

To get closer to large publishers and advertisers, Google bought Doubleclick for $3.1 billion. At the time, Doubleclick already had a very strong position with its publisher ad server (DFP for Doubleclick for Publishers), but also with its advertiser ad server (DFA for Doubleclick for Advertisers). Above all, most competing ad servers were positioned on only one side (publisher or advertiser).

In 2009, the launch of the Doubleclick Ad Exchange

By 2009, Google had long since automated the “Search” market through its sponsored links. It had extended its AdWords ad network to YouTube and its Google AdSense partner network; its performance advertising marketplace was particularly effective. By contrast, “Display” ad campaigns (ad banners and videos on the web) were still mostly managed manually, “the old-fashioned way”:

• Media agencies and publisher sales teams would agree on ad campaigns (X impressions, delivery period, possible targeting, price).
• The agencies would then configure the ad campaigns in their ad servers (DFA or others), then send the campaign tags to the publisher sales teams.
• The publisher sales teams would then configure those ad tags in their ad servers (DFP or others).

This double setup is very laborious, especially when you want to run your ad campaign across many sites. How does ad serving work? It is already not that simple:

Via Ad Ops Insider, How does ad serving work?

To fill this gap, intermediaries called ad networks already occupied an important place: they offered advertisers the ability to reach many publishers without having to contact them one by one (Google AdSense was one of them). Publisher sales teams would then configure the ad network tag in their ad servers.

But managing multiple ad networks remained complex and inefficient: ad networks could not buy in real time because there was not yet an auction system (compensation was fixed in advance, regardless of the user), and they reserved the right to refuse an advertising opportunity. Publisher sales teams had to manage the different ad networks in a system called "Waterfall": a cascade of ad networks called one after another (those assumed to be the most profitable first), a system intended to maximize revenue while limiting damage to the user experience.

Via Ad Waterfalls explained, "passbacks" happen because the ad network has to call back the publisher's ad server if it has nothing to serve, so that the publisher ad server can then call the next ad network... The user experience is catastrophic.

Inspired by AdWords' success and building on its recent purchase of Doubleclick, Google launched its own marketplace: the Doubleclick Ad Exchange (read the interview from the time, in 2 parts). What impact does programmatic have on ad delivery? The mechanisms become more complex:

Via Ad Ops Insider, How RTB ad serving works

Google was slightly behind other players such as Right Media, AdECN, AdBrite and ADSDAQ. But programmatic was only just getting started and the new marketplace ticked all the right boxes:

• Real-time auctions, where the buyer can use the personal data it holds about the user.
• Payments secured by Google (trusted intermediary).
• For buyers on the new marketplace, simplified access to AdSense publishers but also to large publishers already using Doubleclick DFP and having activated Doubleclick Ad Exchange.
• For Adwords buyers, new access to major DFP publishers who have activated Doubleclick Ad Exchange.
• For large publishers, simplified access to AdWords advertisers, but also to the various ad networks that had already integrated with the marketplace (from launch, Google announced that it had integrated most of the large American ad networks).
• For large publishers, dynamic allocation in DFP between their direct sales and sales via the Doubleclick Ad Exchange, allowing them to maximize revenue (competition between directly sold campaigns and Doubleclick Ad Exchange auctions).
• For AdSense publishers, additional demand via Doubleclick Ad Exchange buyers.

How programmatic works with Google, thanks to the acquisition of Doubleclick and the launch of the Doubleclick Ad Exchange:

Do you see conflicts of interest emerging?

In 2010 and 2011, other acquisitions allowed Google to expand its weight in Display

In 2010, Google bought AdMob for $750 million. AdMob offered monetization solutions for mobile sites and apps. Google was already very well placed with Google AdSense on the web and decided to pay a high price to avoid missing the mobile advertising wave. It rewrote the software to include it in the Doubleclick suite in 2013.

Still in 2010, Google made one of its best moves by buying Invite Media for $70 million. Invite Media was a leading DSP and a missing piece at Google. It allowed an advertiser or agency to buy ad inventory on Ad Exchanges. Google was thus completing its tools for advertisers: just as it had been able to add the Doubleclick Ad Exchange component to its DFP publisher ad server, it added the Invite Media buying component (later renamed Doubleclick Bid Manager or DBM during the software rewrite) to its DFA advertiser ad server.

These acquisitions of advertiser-side tools allowed it to offer two buying platforms:

• Google Ads: the buying platform allowing all advertisers, small or large, to buy advertising on Google search results. It also allows small advertisers to run ad campaigns on YouTube, AdSense and Google's Ad Exchange.
• Google Display & Video 360 or DV 360: the “Display & Video” buying platform for large advertisers and agencies, integrating the old DBM but also DFA (renamed Doubleclick Campaign Manager or DCM). They can buy inventory on YouTube, on AdSense, on Google's exchange, but also on the main third-party Ad Exchanges.

In 2011, Google strengthened its dominance on the publisher side by buying AdMeld for $400 million. AdMeld was a competitor to Doubleclick Ad Exchange, and Google won on several levels:

• AdMeld had a lot of quality publishers in its clients.
• If the Ad Exchange part was already well handled by Google, AdMeld complemented DFP and Doubleclick Ad Exchange with better ad network management (programmatic was not yet dominant).

The programmatic ecosystem. In 2011, through Doubleclick, Google held the main publisher ad server (as well as the main advertiser ad server, absent from this graph), but also a new major exchange (Doubleclick Ad Exchange, improved thanks to AdMeld) and one of the main buying platforms (Invite Media). Google had other pieces that it integrated into its “stack”: AdMob on mobile, AdWords and AdSense, Google Analytics, Google Tag Manager, etc.

In 2013, Google's exchange (Doubleclick Ad Exchange, renamed AdX, and now Authorized Buyers) became the main programmatic marketplace. Google's buying platform (DBM, now DV 360) also became dominant.

Mechanism for delivering advertising, entirely within the Google ecosystem. Check out the Wall Street Journal's excellent infographic, How Google Edged Out Rivals and Built the World’s Dominant Ad Machine: A Visual Guide

Beyond the quality of Google products (which should not be underestimated), what are the key points of this rapid rise?

Google has an information advantage over its competitors

Thanks to its acquisition of Doubleclick in 2007, Google acquired a huge footprint on the web. Many large publishers use Doubleclick as an ad server (DFP), which allows Google to be first in line to identify users on the web and recognize most of them via Doubleclick cookies. With most large agencies also using Doubleclick on the advertiser side (DFA), Google further increased its footprint (even if a site does not use DFP, most of its ad campaigns will be served with a DFA advertiser tag).

At this point, you could say: Doubleclick is a tool purchased by publishers and advertisers; the user identity in the tool (a digital identifier) is not Google's property, but theirs. And indeed, that is what Google told the US Congress during the investigation into the Doubleclick acquisition: "That data is owned by the customers, publishers and advertisers, and DoubleClick or Google cannot do anything with it".

Without access to a user identifier, it is much more difficult to monetize a user on the programmatic market. A Google study indicates, for example, a 52% drop in revenue from users who do not have third-party cookies, necessary for storing user identifiers for programmatic purposes on the web. This figure is also found in other independent studies.

And Google is gradually using the Doubleclick user ID for its own interests:

• From 2009 for its Google Ads buying platform, it places Doubleclick cookies on its own sites and with its AdSense partners: "Google uses the DoubleClick advertising cookie on AdSense partner sites and certain Google services to help advertisers and publishers serve and manage ads across the web".
• Still in 2009, Google uses the Doubleclick identifier on its Exchange, the Doubleclick Ad Exchange, allowing Google Ads buyers to recognize users browsing sites from new Exchange customers. By contrast, other buyers (ad networks and other DSPs, used by large advertisers) do not have access to this identifier: "For buyers, Google identifies users using a buyer-specific Google User ID consisting of an encrypted version of the doubleclick.net cookie, derived from but not equal to that cookie. Google passes the user ID to the buyer (raw cookies are never sent)."
• Still in 2009, Google prevented its publisher ad server clients (here: "The DoubleClick cookie ID associated with the user, encrypted.") and advertisers (here: "You will never be able to decrypt user IDs, and Google will not disclose the encryption method. No encryption keys will ever be provided to any Campaign Manager customer or any third-party partner.") from accessing Doubleclick user identifiers, contradicting its previous statement: "That data is owned by the customers, publishers and advertisers, and DoubleClick or Google cannot do anything with it".
• In 2012, when Google rewrote Invite Media to launch DBM, its own buying platform for large advertisers, it had access to the Doubleclick identifier: "For itself, Google identifies users with cookies that belong to the doubleclick.net domain under which Google serves ads."
• In 2013, Google only provides its DFP publishers with logs, without the Doubleclick identifier: "The DoubleClick cookie ID associated with the user, encrypted."
• In 2018, Google removed the Doubleclick identifier from logs from its DCM server: "You will never be able to decrypt user IDs, and Google will not disclose the encryption method. No encryption keys will ever be provided to any Campaign Manager customer or any third-party partner". Reminder: when buying DoubleClick, Google declared: "That data is owned by the customers, publishers and advertisers, and DoubleClick or Google cannot do anything with it".

Checkmate! Here are the consequences:

• Advertisers and publishers used to work directly together for targeted advertising campaigns, via the Doubleclick identifier. They must now use Google, DBM and AdX tools (or Google Ads and AdSense for small advertisers and small publishers).
• DBM and Google Ads have an advantage over other buying platforms: those competitors must synchronize their identifiers with the Doubleclick identifier (cf. Cookie Matching) before they can recognize users when Google sends them advertising opportunities and therefore bid effectively. At best, buying platforms can hope for user non-recognition rates around 20%; at worst, more than 50%.
• Competing buying platforms do not have the same footprint on the web while the Doubleclick cookie is omnipresent: among AdSense and DFP publishers, on Google search, on Gmail, on YouTube, etc.

How does ID synchronization work? It is shown here between a DSP and several SSPs, but the mechanism is the same when an SSP has to synchronize identifiers with several DSPs:

Via Ad Ops Insider, SSP to DSP Cookie Syncing Explained

Google explains the benefits of its “superior cookie reading” very clearly to its customers:

Google Ads and Display & Video 360 drive optimal performance when purchasing inventory on Ad Exchange because these buying platforms share the same infrastructure as the marketplace. This means that potential cookie read loss when Google Ads and Display & Video 360 users purchase on other marketplaces is reduced when purchasing on Ad Exchange. So when purchases are made through Google Ads and Display & Video 360 on Ad Exchange, it's more likely that these solutions will detect impressions that meet their targeting criteria, leading to increased bidding pressure and increased demand for the publisher's inventory.

Protecting your privacy?! Google's fine hypocrisy

Google does not detail the reasons that led it to remove third-party access to the Doubleclick identifier. It simply says it acted for privacy reasons:

• To prevent different players from combining their information (each player has its own version of the Doubleclick identifier, and therefore cannot share user information with a neighbor).
• To prevent players from combining identifiers with personally identifiable information.

If the objective of protecting privacy is laudable, Google does not apply it to itself: it does combine the information it has about you via its various services (Search, YouTube, Gmail, Maps, etc.), via its various Doubleclick clients (advertisers or publishers), it also combines DoubleClick information with the personal information from your Google account if you have not correctly adjusted your Google settings. Here are 2 key decisions:

• In 2012, Google offered 70 different services (and 70 different privacy policies), and the personal data from those services was properly compartmentalized. For example, YouTube did not know your Google search history and therefore could not target you with ads related to that history. Google then decided to offer a single privacy policy and combine your personal data across all these services. Reactions were unanimously negative and investigations multiplied, but nothing happened: Google did not back down.
• In 2016, Google quietly changed its privacy policy and removed the ban on combining information linked to the Doubleclick cookie with personal information from your Google account unless Google had your explicit opt-in. The association became automatic for new users. For existing users, Google disguised this dangerous association by asking whether they wanted access to additional features, under the title: “Some new features for your Google account”.

The Googolopoly game, created in 2008 (since then, the number of companies acquired by Google has exploded). Google's original mission was to organize information on a global scale. Today, it is more about taking over ever-larger parts of your activities, to influence you more effectively.

These decisions are accompanied by many insidious changes to Google products. In 2018, for example, Chrome automatically signs you in to your Google account when you sign in to a Google site (Gmail, for example), without asking for your opinion. It also nudges you into sending your entire browsing history to Google through a Dark Pattern; read “Why I'm done with Chrome”.

If we focus on Doubleclick data, remember Google's statement in 2007: "That data is owned by the customers, publishers and advertisers, and DoubleClick or Google cannot do anything with it". Today, for example, Google collects personal data via its Google Ad Manager publisher clients (example: users browsing a financial information site), personal data that it can then use for its own benefit. How? By selling advertising campaigns via Google Ads to banks, targeted at users of the first site, but on other websites (which have themselves integrated Google AdSense or Google AdX).

If we look carefully at the Google privacy policy, hidden under the section "Protecting Google, our users and the public" (and not under the section "Offer personalized services, particularly in terms of content and advertisements"), we can read:

Depending on your account settings, your activity on other sites and in other applications may be associated with your personal information in order to improve Google services and the ads served by Google.

This is how Google discreetly appropriates data from its publisher and advertiser clients for its own benefit. But careful: prohibiting third parties from accessing the Doubleclick identifier was designed to protect your privacy!

Google has a speed advantage over its competitors

When a marketplace (SSP or Ad Exchange) offers an advertising opportunity to a buyer bot (DSP), it has a limited time to respond, generally between 100 ms and 200 ms. If it often fails to respond on time, the SSP will reduce the number of opportunities sent. Google's buying platforms (Google Ads, DV 360) therefore have a huge advantage: they are located on the same servers as the Google marketplace (AdX):

Google's speed advantage, reduced latency.

Google also highlights this “reduced latency” in its documentation, with other players being able to “lose” up to 25% of auction requests:

When purchases are made through Google Ads or Display & Video 360 on other marketplaces, the users in question experience the same network latency as the rest of the buyers. We've found that in some cases, latency issues can prevent buyers from submitting a bid for up to 25% of auction requests, limiting their participation in auctions taking place on other marketplaces.

This time advantage allows Google's buying platforms to be more "intelligent": to take the time needed to retrieve user data and run the right algorithms in order to choose the ad that will have the strongest impact on the user, at the optimal price (and that will maximize Google's profits).

Note that Google offers buying platforms a colocation option (via Google Cloud Platform), but only the OpenX SSP chose it. It is difficult to know whether other buying platforms refuse to use Google Cloud for cost reasons, obvious conflict-of-interest concerns, or simply because they judge that this advantage is not so decisive. The figure of 25% of auction requests lost due to latency problems cannot be audited because Google does not provide its publishers with information about auctions excluded because they arrived too late (cf. Bids data in Ad Manager Data Transfer).

Google imposes its own buying platforms

To purchase inventory on YouTube, an essential site, advertisers have to go through Google's buying platforms because in 2015 Google decided to cut off access to YouTube inventory for third-party buying platforms. Google was also challenged by US authorities in 2013 for limiting Google Ads API functionality, making it difficult to purchase Google sponsored links via third-party tools.

As we saw previously, using Google DV 360 and another buying platform in parallel runs the risk of bidding twice for the same user (because the other buying platform does not have access to Doubleclick cookies and therefore cannot recognize that it is the same user) and thus artificially increasing prices.

Google's buying platforms are therefore essential for advertisers, with only Facebook able to compete, but it has no longer had a footprint on the web since the shutdown of its Audience Network (now only available in the app ecosystem) and can thus "only" sell access to its own properties on the web (Facebook, Instagram).

Google directs purchases to its own platforms and sites

When a small advertiser creates an account on Google Ads, they must first start a "Search" campaign (on Google search results) before they can purchase inventory on third-party sites. A first method for directing purchases to Google's own sites.

Google can also direct purchases from its buying platforms to sites using its monetization solutions:

• Until 2016, Google only directed Google Ads purchases to the Google Ad Exchange (cf. the presentation of the Doubleclick Ad Exchange at the time: "Ad Exchange is the only exchange offering access to the full demand of Google AdWords"), preventing other Ad Exchanges from taking advantage of this potential advertising windfall. This has since changed, but in a limited way: only for remarketing campaigns, cf. Google Ads online help.
• While it has not blocked access to other Exchanges from its buying platform for large advertisers, DBM (now DV 360), it is frequently accused of directing purchases primarily to its own Ad Exchange: "Most DBM buyers we talk to say more than half of their spend goes to AdX when they use DBM", the CEO of rival AppNexus declared at the time.

Google directs sales to its own marketplace

On the publisher side, the dominant server is Google DFP. This was recently merged with Google AdX and renamed Google Ad Manager. Following the acquisition of Doubleclick in 2007, Google took advantage of DFP's dominant position to impose AdX. Here's how it did it:

• Directly in DFP, the publisher can configure its guaranteed volume advertising campaigns (pre-sold, for which the buyer undertakes to purchase a certain volume of advertising opportunities, based on precise distribution criteria) and its non-guaranteed volume advertising campaigns (the publisher offers advertising opportunities, the potential buyer has no obligation to purchase).
• Advertising campaigns with non-guaranteed volume may come from ad-networks having a direct agreement with the publisher. But as mentioned previously, the publisher must then configure a system of calls to ad-networks in cascade (one after the other), leading to loss of revenue (fixed price for each ad-network, no direct competition) and poor user experience (slowness).
• Advertising campaigns with non-guaranteed volume can also come from Google AdX, integrated “natively” into DFP since 2010. This allows real competition between buyers (real-time bidding). It is also connected to Google's two buying platforms: DV 360 (formerly DBM) and Google Ads (formerly AdWords).
• A competition system called “Dynamic allocation" arbitrates between advertising campaigns with guaranteed volume and those with non-guaranteed volume. Its objective is to maximize the publisher's revenue while ensuring that guaranteed-volume campaigns respect the contract with the buyer and serve the right impression volume, on the right criteria (inventory, targeting, distribution dates). How? By dynamically setting a “shadow price” for guaranteed-volume campaigns (high enough to allow them to serve) and checking whether non-guaranteed campaigns offer a better price.
• If AdX is natively integrated into DFP, other marketplaces are not as lucky. Until 2017 and the announcement of Exchange Bidding, DFP required publishers to configure other marketplaces as simple ad networks (and therefore call them one after the other, after AdX if AdX had not already "preempted" the opportunity). They also had to enter a fixed selling price for each marketplace, even though the point of an auction is not knowing the selling price in advance.

Consequence: at the end of 2014, publishers wishing to work with other marketplaces found themselves “hacking” DFP via a system called “header bidding”.

Header bidding, an imperfect hack to put Google AdX in competition

Header Bidding step by step, detailed on the Ad Ops Insider website:

Very briefly, because DFP does not allow real competition between AdX and other marketplaces, header bidding allows marketplaces (excluding AdX) to compete outside the ad server (via the page header), before the call to DFP. The winner of the auction is then put into competition with the other DFP campaigns, guaranteed volume or not. The publisher has to create as many “line items” (campaigns on DFP) as price brackets, for each marketplace!

Via the article What are the consequences of header bidding on programmatic buying?, header bidding allows real competition among third-party marketplaces.

This implementation allows publishers to loosen Google's grip a little through competition with other marketplaces. It also allows them to decide on the maximum response time granted to different marketplaces, letting them factor in bids from slower buyers. The downside: the tags from the different marketplaces, all called directly via the page header, cause delays and degrade the user experience.

Finally fair competition between the different marketplaces? Not really...

The “last-look”, or how Google gains a new advantage over its competitors

The devil is in the details and third-party marketplaces are still at a disadvantage compared to Google AdX. Here's why:

• RTB auctions then operate almost exclusively in "2nd price auction" mode: the buyer only pays the price of the second bid. Example: on the AppNexus marketplace, A bids €5, B bids €1 ==> A wins and should pay €1.
• This €1 price also allows AppNexus to win the header bidding competition (against other third-party marketplaces).
• AppNexus then submits this €1 price to DFP. Via dynamic allocation, AdX buyers know that they must bid more to hope to win the competition.
• C bids €2, D bids €0.5, the floor is €1 ==> C wins and pays €1.

This advantage that Google grants itself is called “last look” in the adtech world.

In this example, buyer A was ready to spend €5, but could not spend that bid. The third-party marketplace had a potential buyer at €5 versus a potential buyer at €2 on Google AdX, and still unfairly lost the competition. The publisher could have earned €5 (minus commissions), but ultimately earns €1 (minus commissions). Only one winner: Google, which satisfied its buyers and took its commission via Google AdX.

In order to maximize their chances of winning in the competition with Google AdX, third-party marketplaces gradually (and belatedly, from 2018) decided to switch to auctions in "1st price auction" mode: the buyer then pays the price of their bid. A first part of the problem is solved: in the previous example, buyer A wins and pays €5. But Google continues to pass this floor price to Google AdX buyers (including Google Ads).

They therefore maintain an information advantage and can decide to bid 1 cent more to win the auction ("last look"). Third-party marketplaces remain at a disadvantage.

Google Ads can arbitrage without anyone realizing it

While an advertiser will be entitled to transparent information on their DSP (bid price and purchase price) when they buy on AdX, this is not the case if they use Google Ads: it does not provide this level of transparency in its reports. Also, the purchasing method on Google Ads is per click (CPC or cost per click), whether the advertising is broadcast on Google search results or on Google AdX. But programmatic sales are made by impression (CPM or cost per 1000 views), making comparisons more difficult.

Also, when it relies on Google AdX, Google Ads has a huge information advantage:

• It does not need to synchronize user identifiers.
• It benefits from the omnipresence of Doubleclick trackers (all Doubleclick publisher and advertiser clients, Google Analytics clients having activated advertising features, AdSense and YouTube), information that it authorizes itself to combine for its own benefit (remember, the merger of the privacy policies of Google's 70 different services in 2012).
• If the user has not opted out, the Doubleclick information is also combined with the personal information associated with the user's Google account (remember, Google's terms of service change in 2016).
• Finally, it represents a considerable number of advertisers, all interested in displaying their advertisements on Google search results.

This information allows Google Ads to predict the click-through rate with great precision, and therefore to bid at the best price to optimize its own income (while guaranteeing acceptable performance for the advertiser). An example below, from the paper “‘Trust Me, I’m Fair’: Analyzing Google’s Latest Practices in Ad Tech From the Perspective of EU Competition Law”:

Here, Google Ads receives €6 but only pays €3 to win the auction, it can keep the difference

Note that Google Ads takes advantage of several factors to arbitrage:

• Just like the other participants in the AdX auction, it only pays the price of the second bidder ("2nd price auction"). But as it operates in a black box, it can afford to collect the difference between its bid and the actual sale price, without any accountability.
• The CPC purchasing method makes the audit task even more complicated: only Google has the information on the expected click-through rate.
• Even if the AdX auction changes to a "1st price auction" (we'll talk about this later), the "last-look" allows it to bid less than it could, and collect the difference.

Open Bidding, Google's answer to header bidding

Header bidding remains a hack that Google doesn't like:

• Google did not originate it and, above all, does not take commission on sales that go through other marketplaces.
• As a result, Google has no interest in making operations easier for the publisher: setting up header bidding in DFP is complex, reporting is also very complicated (new adtech players are taking over to help with these complexities).
• Google does not allow Google AdX to be integrated into header bidding solutions. The publisher must run a first auction on the header bidding side, then a 2nd auction in DFP, degrading the user experience. Multiplying partners in the header also runs the risk of degrading the user experience.

In 2017, Google proposed moving the competition to the server side (on its own servers):

The "Server-Side Header Bidding", explained on the Ad Ops Insider blog

The feature is initially called EBDA (Exchange Bidding in Dynamic Allocation), now renamed Open Bidding. Google Ad Manager puts marketplaces that are willing to integrate with it into direct competition (not a given; note for example the absence of AppNexus, one of the largest marketplaces, and the main initiator of header bidding) with Google AdX.

Marketplaces gain easier access to the many publishers who use Google Ad Manager. Google also provides marketplaces (and other buyers) with the effective selling price of each auction, which it does not do during header bidding integrations.

The advantages for the publisher?

• Even if the publisher still has to sign a contract with the different marketplaces, the system is “plug & play”, with no operational headaches.
• Payment is made directly by Google, which makes accounting easier (and publishers know that Google pays quickly, which is less true of smaller SSPs, who must be paid by DSPs, who themselves must be paid by agencies).
• The publisher has no limits on the number of third parties to work with, because there is no impact on the user experience (no additional tags on site, calls take place on the server side).

But, nothing is free and Google takes a commission of 5% or 10% depending on the advertisements (standard, video or app). A third-party marketplace is at a terrible disadvantage:

• It must first take its commission, then submit the bid to Google, which also takes its commission before comparing the reduced price to other bids; see Google online help: "Highest net bid (which takes Ad Manager's revenue share into account) wins the impression".
• Just like third-party buying platforms on the Google Ad Exchange, it does not have direct access to the user (Google calls this server-side) and must therefore synchronize its user identifiers with Google's before an auction. This leads to a lower user recognition rate, and therefore poorer monetization (especially since it must also synchronize its user identifiers with those of the buying platforms).
• It is also at a disadvantage in terms of speed: it must respond to Google in less than 160 ms, knowing that it must run its own auction itself. As seen previously with third-party buying platforms buying on Google AdX, being located on the same servers (Google Ads and Google DV 360) allows buyers to bid more often and better. The logic applies here between Google AdX, advantaged because it is located on the same servers as Google Ad Manager, and other marketplaces.
• Smart buyers look for the cheapest opportunities, and therefore with the fewest intermediaries. They will favor a direct purchase via Google AdX rather than an Open Bidding partner. This optimization mechanism is called "Supply Path Optimization" or SPO, it is applied by all DSPs worthy of the name. If you are interested in the topic, you can read this article by AppNexus co-founder Brian O'Kelley.
• Obviously, the publisher can disable a third-party marketplace, but it is impossible to disable Google AdX.

Google doesn't let publishers understand how bidding works

Understanding how these auction mechanisms work is crucial for publishers: which buyers are interested in my inventory? What price are they willing to pay? How do they behave if I change the floor price? Should I contact them directly if they buy a lot of inventory, to secure preferential agreements? Transparency is essential, namely access to all the information on these auctions, only Google restricts the information:

• Access to information costs money!
• Auction data (how much each buyer is willing to pay) and impressions (who won and what the final sale price is) are accessible via 2 different files, one for "impression data" and another for "auction data".
• Auction data contains the buyer's name, the bid value, and whether the auction won. Missing from the file: auctions from other marketplaces, as well as the effective purchase price.
• The impression data contains the winner, the effective purchase price as well as the bids from other marketplaces.
• Google Ads advertisers are always hidden behind the Google Ads buyer: it is impossible to know them, nor to know how much they paid (Google's margin is unknown).
• A publisher could link these 2 files, but since Google's announcement in September 2019, this is no longer possible.
• Result: impossible to verify the impartiality of Google's unified pricing, impossible to verify whether Google is arbitraging, impossible to verify whether Google runs the AdX auction after receiving bids from other marketplaces, etc.
• Also, Google gains valuable information on the behavior of rival marketplaces. It would not be difficult to use this information to predict their future bids, and thus benefit Google AdX.

The end of the “last-look”, really?!

Google decided in March 2019 to follow other marketplaces and switch Google AdX to “1st price auction” mode, first communicating vaguely about the end of the “last-look”:

An advertising buyer’s bid will not be shared with another buyer before the auction or be able to set the price for another buyer

Google's explanatory diagram is also very vague (and it is impossible to know if the "1st price auction" applies when Open Bidding is not activated):

In May 2019, it gave a little more detail:

Going forward, no price from any of a publisher’s non-guaranteed advertising sources, including non-guaranteed line item prices, will be shared with another buyer before they bid in the auction

Why did I put "including non-guaranteed line item prices" in bold? Because these "line items" (campaign elements entered on Google Ad Manager) are used in particular by third-party marketplaces for header bidding. And if we look at the history of the Google article via the Wayback Machine, we realize that these "line items" were not included in the initial version of the article, here is the complete passage from the time:

After the transition, Ad Manager will have a single auction that compares the prices from a publisher’s guaranteed campaigns with all of a publisher’s non-guaranteed advertising sources — including real-time bidding partners, such as Authorized Buyers and Exchange Bidding partners — and prices from non-guaranteed line items, like those from a publisher’s header bidding implementation. Going forward, no price from any of a publisher’s non-guaranteed advertising sources will be shared with another buyer before they bid in the auction. As has always been the case, all real-time bidding partners integrated with Ad Manager — including Google Ads and Display & Video 360 — will be notified of an auction at the same time.

If we read the initial excerpt carefully, as Damien Geradin and Dimitrios Katsifis, the authors of "'Trust me, I'm Fair': Analyzing Google's Latest Practices in Ad Tech From the Perspective of EU Competition Law" did in October 2019, Google does not mention the non-guaranteed line items when it says it no longer shares price information before the auction, only the non-guaranteed advertising sources, namely AdX buyers (Authorized Buyers) and Exchange Bidding (Open Bidding).

It turns out that the specialized press had already announced the end of the "last-look" in 2017, which seemed at the time to concern only Open Bidding (was that really the case, if Google only communicated in 2019?). Same story in March 2019: the specialized press rushed to announce the end of the “last-look”. One argument put forward: the "last-look" would be incompatible with 1st price auctions. This is not the case: Google Ads (or another AdX buyer) can absolutely decide to bid 1 cent more than the winner of the header bidding auction if it has the information, making it possible to give much less money to the publisher and potentially keep the margin for itself.

So what happened with the article? According to the Wayback Machine again, the non-guaranteed line items were added to the article between November 15, 2019 and August 15, 2020. Was this following the paper by Damien Geradin and Dimitrios Katsifis? Did Google really remove the “last-look” (depriving itself of an arbitrage option via Google Ads) in May 2019? Did it really do it between November 15, 2019 and August 15, 2020, and if so, why did it not communicate about this development? It is impossible to verify because Google does not provide sufficient transparency via auction and impression data, so there is reason to doubt it.

The “1st price auction”, an excuse to block the monetization of Google Ads

With the launch in 2019 of 1st price auctions, Google has shaken up the rules for creating floor prices for publishers. Previously, they could decide to set floor prices higher for certain buyers. A frequently used rule: increase the floor price of Google Ads (to prevent Google Ads from finding itself paying much less than it could, thanks to the "2nd price auction" and the "last-look").

Also, these floor price rules only applied to AdX buyers. Publishers could manage header bidding or Open Bidding partners separately, and apply a lower floor price to them if they wanted. The new "unified rules" remove this option, and obviously Google presents this as an advantage:

Our new unified pricing rules will help publishers more easily manage floor prices across all non-guaranteed partners. For example, instead of setting up the same floor prices in multiple places — in the auction in Ad Manager, and with their Exchange Bidding and other non-guaranteed advertising sources — which can take a lot of time and can lead to errors, a publisher can set up a single unified pricing rule to control pricing from one place.

Google explicitly prohibits creating different floor price rules depending on the buying platforms. It presents this abusive maneuver as a fairness measure:

To maintain a fair and transparent auction, these rules will be applied to all partners equally, and cannot be set for individual buying platforms.

Above all, this is a massive loss of control for the publisher over the sale of its inventory, and a golden opportunity to accelerate arbitrage via Google Ads.

The AMP weapon to eliminate header bidding

The AMP format is a web page format launched by Google in 2016. Its objective is to speed up the loading of pages on mobile, but it is seen by many as a threat to the open web: through its dominant position in online search, Google forces publishers to develop a specific and very limited version of their site, cached on Google's servers. Without an AMP version of their site, publishers do not have access to carousel. The lightning bolt representing AMP in search results also encourages users to click on those results.

Also, Google does not say that AMP is a ranking factor (better ranking in search results), but it emphasizes the fact that loading speed is very important: "Even though AMP itself does not directly impact your position in search results, speed is a determining ranking factor for Google Search". In theory publishers have the choice, in reality and if they do not want to lose their ranking in Google search results, they are forced to provide an AMP version of their pages.

And what about advertising? AMP prevents publishers from using client-side header bidding (a javascript library), so they are encouraged to use Open Bidding if they want to work with third-party marketplaces, or simply to use the Google marketplace exclusively. Since AMP pages are cached by Google, the entire advertising chain can run on the same servers: once again, the speed advantage (but for monetization, apart from the speed benefit demanded by Google, it is far from won).

Since AMP is widely criticized, Google's new strategy consists of pushing Core Web Vitals, new metrics intended to better represent the overall user experience. These metrics were scheduled to be taken into account the following year in rankings: "Today, we’re building on this work and providing an early look at an upcoming Search ranking change that incorporates these page experience metrics". Publishers will not need to have AMP versions of their pages to be featured in the carousel, but rather to have a good score on Core Web Vitals.

Core Web Vitals, a measurement of user experience on different axes. Today, Google already measures these indicators via Chrome users who have synchronized their browsing history and have the sending of statistics activated (it makes aggregated statistics available via the Chrome User Experience Report).

Although this initiative is laudable, there are fears that AMP will be replaced by an even more problematic version, Web Bundles: a new format allowing a site to group all the resources of a web page in the same file, and potentially have it cached by third parties such as Google. A key point for us: these Web Bundles will greatly complicate the work of ad blockers. Read Peter Snyder's article on this subject, WebBundles Harmful to Content Blocking, Security Tools, and the Open Web.

Programmatic direct, an additional weapon to tax the advertising ecosystem

If “Display” advertising now mainly runs through programmatic channels, direct relationships between buyers and publishers have not disappeared. In particular:

• Publishers with quality inventory and/or a specific audience and/or "innovative" advertising formats can still offer preferential deals to buyers (inform them of these special opportunities, or even give them priority over other buyers). Obviously, buyers will push to be entitled to these preferential agreements.
• Buyers wishing to secure their purchases will always need publishers to guarantee a certain impression volume. This need is particularly important when inventory is scarce: if I want to promote a new film, I might for example want to serve 10 million impressions on YouTube between November 5 and November 8.

Before programmatic became more widespread, these transactions involved only advertisers and publishers. As a result, they did not pay an RTB tax (no commission on the media price to be paid on the DSP side, no commission to be paid on the SSP side, only the ad server delivery cost, which is fixed and therefore much cheaper). Google, like its DSP and SSP competitors, saw a significant missed opportunity here: it therefore pushed the advertising ecosystem to bring these transactions into the programmatic pipeline (via a system called “Deals”):

• First with "Preferred Deals": agreements between one buyer and a publisher, at a fixed price, with priority over other non-guaranteed campaigns (AdX, Open Bidding, header bidding and ad networks), but with non-guaranteed volume (the buyer does not commit to purchase; they can choose the opportunities).
• Then with "Programmatic Guaranteed": agreements where the buyer agrees to purchase a guaranteed volume of impressions.

Here are the arguments that can justify paying for a service on these advertising transactions:

• In the case of "Preferred Deals", the buyer chooses the opportunities (in the old world, when called, the advertiser's server has no choice).
• The DSP manages all of the buyer's campaigns, which allows this tool to offer unified reporting.
• Payments to publishers are now simplified and secure, the SSP takes care of it.
• If setting up direct campaigns is complex, Google allows buyers and publishers to negotiate agreements directly in their respective interfaces (DV 360 and Google Ad Manager). Note that this option could have been offered without the RTB pipes, but Google had nothing to gain from it.

And if we compare the share of “Deals” with open auctions, we realize that these direct relationships represent a considerable part of advertising investment. They were less visible before because advertising campaigns did not pass through programmatic channels:

eMarketer figures and forecasts for the US market: “Deals” were expected to exceed open auctions in 2020. France generally follows the same trend, 1 or 2 years later. The potential for programmatic players is therefore enormous.

But it is difficult to justify continuing to take a commission on media amounts spent (X% on the DSP side, Y% on the SSP side): this is a direct agreement between a buyer and a publisher; the intermediary is not generating the business. We would rather expect a fixed CPM, similar to ad server pricing. However, this is Google's choice, and the programmatic ecosystem has aligned itself with it.

Direct programmatic, an additional advantage for AdX

These direct transactions are a good way to discriminate against other marketplaces:

• Google offers a Marketplace in DV 360: this allows buyers to discover publishers' offers, but also to negotiate with them directly in the interface. If the negotiation is successful, campaign creation is automatic for both buyer and publisher. Problem: this Marketplace is almost exclusive to AdX publishers (except Rubicon).
• “Programmatic Guaranteed” involves being able to guarantee a volume of impressions. Only ad servers are capable of managing impression volume. As a result, marketplaces that are not coupled with an ad server cannot sell Programmatic Guaranteed (to Google DV 360 or another buying platform). Publishers using Google Ad Manager (DFP and AdX) are of course already equipped.
• Within Google Ad Manager, "Preferred Deals" are only available on AdX: Open Bidding partners and marketplaces using header bidding cannot be prioritized over other non-guaranteed campaigns.

So, if you are a third-party marketplace, it is almost impossible to compete with AdX among Google Ad Manager publishers; you are left collecting crumbs.

Replace Google Ad Manager? Mission (almost) impossible

We have seen it many times, the first tool called on the publisher page is strategic: it is the publisher ad server (DFP in the case of Google, which was recently "merged" with AdX to be renamed Google Ad Manager). Google promotes its own demand via multiple processes: "last-look", complexity in managing header bidding, tax on Open Bidding, black box on auction and impression data, restrictions on floor price rules, better user recognition, better connection with its own buying platforms, merging of your Doubleclick and Google personal data, facilitation of direct programmatic, etc.

What happens if a publisher decides not to work with Google on the ad server side? The difficulties are immense:

• Adding a new marketplace on Open Bidding is very easy, adding a new marketplace in header bidding is relatively easy, but migrating an ad server is a much more complex operational task, which can take several months.
• Google AdX is the only marketplace with the full Google Ads demand (other marketplaces only have remarketing campaigns). But Google Ads is the largest buyer in the world. And Google refuses to offer AdX in header bidding.
• Google also refuses to integrate AdX with competing server-side header bidding systems (such as Prebid Server or Amazon Transparent Ad Marketplace), or with marketplaces and other competing ad servers.
• Consequence: if you do not want to cut yourself off from the main source of demand (namely: putting AdX campaigns in competition with the campaigns of other marketplaces at the same level), you must use AdX, and therefore you are strongly encouraged to use Google Ad Manager.

Can you avoid Google Ad Manager and take advantage of AdX? Yes, but only via an incredibly complex mechanism, set up by Axel Springer (the largest European media group) with the help of its partner AppNexus (a Google Ad Manager competitor offering an ad server and SSP) and allowing AdX to compete with other demand sources:

If you don't understand anything about this diagram, that's normal.

Necessary regulation

If Google has been able to establish itself so effectively in the advertising market, it is not only because its tools are "better" (the argument should not be underestimated: Google offers good tools with an excellent user interface), but above all because it was able to take advantage of an absolute absence of regulation to buy up multiple companies and abuse its dominant position.

The different documented practices are not exclusive to advertising markets. As Dina Srinivasan documents so well, we can find these practices in other markets such as financial markets, online ticket sales, etc. However, other markets are regulated and sanctions are applied when a player abuses an advantage. As a result, these markets perform better.

How was Google able to escape any regulation in these advertising markets? Without being a legal expert, one of the elements is undoubtedly the complexity and opacity of adtech, which you may have noticed if you stuck around to read this entire article... The options available to regulators are numerous; here are some suggested in recent studies:

• Structurally separate (sale, or independent divisions with strict controls) the ad server (DFP) from the marketplace (AdX).
• Structurally separate its purchasing activities (Google Ads, DV 360) from sales activities (AdX, AdSense, AdMob).
• Structurally separate Chrome.
• Activate all Google Ads demand on other marketplaces.
• Allow ad servers simple access to AdX demand.
• Allow third-party buying platforms to purchase YouTube inventory.
• Facilitate the purchase of advertisements on Google search results through third-party platforms.
• Force Google to provide full transparency on auction and impression data.
• More generally, regulate and sanction the entire advertising ecosystem: Google is the company that abuses the most, but it is not the only one.

More fundamentally, what is the real usefulness of this ultra-complex programmatic ecosystem, a source of massive personal data leaks? We can clearly see the negative effects: catastrophic user experience, trivialized and generalized surveillance (thanks Google), extreme media dependence on Google... If we look at its cost and opacity, the results are shocking:

Study on programmatic transparency: yes, the publisher only receives 51% of the amount spent by the advertiser; also note that the study cannot explain 15% of the money spent...

As a publisher, if you don't offer a subscription, is life outside adtech possible? Yes, if we believe the experience of Dutch public broadcaster NPO, which saw its advertising revenues increase after abandoning targeted advertising. Of course, the experiment is easier to run when you are a public broadcaster, but it is worth trying.