The move to first price cannot cure what ails programmatic auctions

In today’s digital landscape, programmatic auction structures are changing at a rapid pace in media planning, often leaving buyers losing auctions when they shouldn’t. While supply-side platforms (SSPs) and demand-side platforms (DSPs) have shifted toward first-price auctions, claiming this will help solve the problem, they have disputed that this change will increase costs for buyers. We recently put these claims to the test in the real world and discovered that moving to first-price auctions increases costs, even with mitigation measures in place. Hearts & Science Executive Director of Digital Activation Ben Hovaness has a three-tiered plan to make bid strategy straightforward and help clear the air around programmatic auctions.

Executive Summary

Increasing amounts of programmatic inventory are being sold in first-price auctions (“1PA,” winner pays their bid) in many of the major SSPs, a departure from the second-price auction model (“2PA,” winner pays second-highest bid) that has historically prevailed in programmatic ad sales. SSPs moving to 1PA have claimed that this change should be beneficial for buyers, as it resolves the current 2PA design flaw that sometimes results in the highest bidder losing an auction. Certain SSPs and DSPs have also claimed the move to 1PA should be cost-neutral for buyers once they adapt their bidding strategies.

Auction theory predicts that this cost-neutrality claim is inaccurate and our real-world testing bears this prediction out. Based on our data, for a given set of inventory, advertisers are better off buying through an SSP conducting a 2PA. This is true even when a DSP auto bidder is used to buy the 1PA inventory; in this case, the 2PA-priced inventory still has a lower price. Advertisers can and should adjust their programmatic supply path accordingly, both by shifting investment toward 2PA-priced inventory when possible and by negotiating fixed-price private marketplace (PMP) deals with their biggest publishers. However, these near-term mitigation strategies are not the end of the story.

The programmatic auction system should be restructured as a classic single-stage 2PA. This would retain the benefits of the present 1PA model (highest bidder always wins) while adding the benefits of a 2PA system (bidding truthfully is optimal). This could be achieved through the following reforms:

1. SSPs must pass all bids for an impression, not just the highest, through to the publisher.
2. Publishers must conduct a clean second-price auction. Price floors can be set for inventory, but the practices of conducting a first-price auction and price discrimination (variable price floors per advertiser not tied to a PMP deal) must end.
3. Both SSPs and publishers must submit to audits by a neutral third party to ensure compliance with points 1 and 2.

This combination of changes would make the marketplace much more advertiser-friendly by making bid strategy straightforward and clearing the air around programmatic auctions while maintaining or improving yields for publishers. Hearts will be working with partners across the programmatic supply chain to bring this vision to life.

Auction game basics

Figure 1: Private value, clearing price, and buyer surplus

Auctions have been used for millennia to allocate difficult-to-price items to buyers. While a tremendous variety of auction forms exist, this paper focuses on two: sealed-bid first-price auctions (1PA) and sealed-bid second-price auctions (2PA), the two forms that dominate the digital media landscape. Buyers must pursue different bid strategies as the auction form dictates (e.g., 1PA vs. 2PA) but will typically seek to maximise the difference between the value they or their clients place on an item (their “private value”) and what they ultimately pay for it (the “clearing price”).¹ This difference is called a “buyer surplus.”

In a 1PA, the highest bidder wins and pays what they bid, regardless of the behavior of other bidders. By contrast, in a classic 2PA, the highest bidder wins the auction but only pays the second-highest bid amount. This is true even if the highest bidder values an item dramatically more than the second-highest bidder; in every case, the winner will pay the second-highest bid amount. By extension, the same bid amount will yield a different clearing price depending on whether the auction is 1PA or 2PA.

This has a direct bearing on the optimal bid strategy for each of these auction forms. The dominant strategy in 1PAs is to guess the lowest bid amount between the bidder’s private value and the second-highest bidder’s bid that will result in an auction win. Contrary to what some commentators have written, it is suboptimal to bid your private value in a 1PA; doing so eliminates the possibility of a buyer surplus. On the other hand, 2PAs are “dominant-strategy incentive compatible” (DSIC), meaning that the best or “dominant” strategy for bidders is to bid their private value. ²

The simple dominant bid strategy afforded by the 2PA form benefits both buyers and sellers. Bidders do not have to concern themselves with determining an optimal bid distinct from their private value. Sellers, assuming a constant set of bidders, should receive an average selling price identical to what they would have received in a first-price auction (more on this later).

Thinking about the marketplace more dynamically, the elegance of the 2PA should give additional bidders the confidence to buy ads at auction—driving up publisher revenues!  

Programmatic second-price auctions stand apart

Because of these benefits, second-price auctions have been widely adopted in digital media, with Google and Facebook most prominent among the firms using second-price auction variants to sell billions of dollars of digital ad inventory each year. Since all demand for Google search ads and Facebook social ads flows to those firms directly through their proprietary systems, all bids are aggregated into each relevant ad auction. As such, these auctions follow the classic 2PA auction design closely. The programmatic 2PA, on the other hand, diverges from the classic design dramatically.

Figure 2: Header bidding’s position atop the programmatic “waterfall”

Demand for programmatic ads is diffused across many DSPs, so digital publishers rely on SSPs to coalesce demand for their inventory. Not every DSP is integrated with each SSP, so publishers employ multiple SSPs to capture as much demand as possible. SSPs were originally given the opportunity to bid in sequence, with each publisher working its way down its list of partner SSPs until one transmitted a bid above the price floor. This sequential “waterfall” approach was simple enough but resulted in forgone revenue for publishers since the SSPs never had to compete directly in a unified auction. “Header bidding” was developed to solve this problem, forcing all SSPs to bid simultaneously into an instantaneous, yield-maximising publisher auction.³

The implementation of header bidding, however, broke the logic of classic 2PA. Now, the SSPs would independently receive bids from DSPs, conduct a 2PA, then pass the clearing price to the publisher. When presented with multiple clearing prices from SSP 2PAs for a given impression, the publisher conducts a 1PA, selecting the highest clearing price from the set of SSP auctions. The two-stage nature of this auction—2PA at the SSP, then 1PA at the publisher—destroys the DSIC quality of a classic 2PA because the highest initial bidder has no guarantee of victory!

The diagram below illustrates the problem:

Figure 3: The problem: in 2PA, sometimes the highest bidder loses

The shift to first-price auctions

The use of 2PA at the SSP in header bidding created bid-strategy problems for advertisers, who could not be assured that a high bid would result in an auction win. And for SSPs, it created a business problem: SSPs have a business model based on earning a commission, a “take rate,” on winning bids, so their inability to submit the highest received bid unaltered to the final auction hurt their win rates and, therefore, their revenues.⁴ Even when an SSP won an auction, their take rate was a function of the second-highest bid only, further depressing their earnings. As a countermeasure, SSPs often non-transparently adjusted the bid transmitted to the publisher such that it was no longer a true second price.⁵ This boosted SSP revenues but damaged advertiser confidence in the integrity of the auction system.

Finally, while the move to header bidding did force SSPs to compete with each other head-to-head, it did not fully aggregate advertiser demand. There is still tremendous bid attenuation between advertisers and publishers. As the simple model in Figure 3 above illustrates, four advertisers may bid into two SSPs, but the publisher will only see two bidders. This means that the publisher can’t conduct its own 2PA without suppressing revenues.

First-price auctions in the SSP were envisioned as a way to resolve all of these issues in one stroke. After all, if the two-stage auction is a 1PA in both stages, a bidder can be confident that she will win if she has the highest bid among all bidders, across all SSPs. Concordantly, SSP win rates should be identical to the percentage of the time a given SSP has the highest bid among its competitors. The business implications are positive: For a fixed set of bids, SSP revenues should increase. Advertiser confidence in the integrity of the auction system should be improved, too, as winning bidders’ clearing prices should be identical to the bids submitted by their DSP.⁶

Figure 4: First-price auctions ensure the highest bidder always wins but drive up cost

Unfortunately, the move to 1PA is not a win for advertisers for two key reasons. First, costs will increase. Second, it is impossible to construct an optimal bid strategy. Both predictions are grounded in existing auction theory literature and supported by our testing. The remainder of this paper examines the theoretical basis for our claims, the results of our field experiments and recommendations for advertisers.

Auction theory

Why does moving to 1PA directly increase advertiser costs? William Vickrey is the father of modern auction theory, so much so that the sealed-bid second-price auction (introduced above) is commonly referred to as the “Vickrey auction” in his honor. Vickrey’s 1961 paper Counterspeculation, Auctions, and Competitive Sealed Tenders not only created the field of auction theory but also established a core concept called “revenue equivalence.” Vickrey used tools from game theory to demonstrate that if certain conditions were met, four standard auction types would generate equivalent revenues for sellers (and prices paid by auction winners):

1. Second-price single-bid sealed-bid auctions (Vickrey – classic 2PA)
2. First-price single-bid sealed-bid auctions (Blind – 1PA)
3. First-price ascending-bid open auctions (English)
4. First-price descending-bid open auctions (Dutch)

The Vickrey auction and Blind auction are commonly analogised to programmatic advertising’s 2PA and 1PA auctions, respectively. This is a mistake, because programmatic 2PAs deviate from the classic design in a way that violates the revenue equivalence requirements. Vickrey’s revenue equivalence theorem requires the following conditions to be met to apply to a given auction design:

1. The auction mechanism must always allocate the item to the highest bidder.
2. Bidders must be risk-neutral.
3. Bidders have independent, uncorrelated values for the item.

Programmatic second-price auctions, as shown in Figure 1 above, do not satisfy the first criterion: The impression does not always go to the highest bidder. Sometimes, a lower-bidding advertiser can win. Therefore, programmatic 2PAs and 1PAs cannot be revenue equivalent.⁷

We can take this a step further than merely saying the two forms are not revenue equivalent. Since the programmatic 2PA sometimes allocates impressions to non-high bidders, it is a mathematical certainty that average clearing prices will be higher in a 1PA auction form.⁸

Why is it impossible to construct a dominant bid strategy in programmatic 1PA? Constructing a dominant strategy in a second-price auction is easy: A classic 2PA is fully incentive compatible. All a bidder must do is be aware of his private value and then bid it. The auction form takes care of the rest, ensuring that the high bidder captures the maximum possible buyer surplus.⁹

Sealed-bid first-price auctions call for a more complex strategy that is rooted equally in probability and the bidder’s private valuation of an item. After all, if a bidder values a media vehicle at a $10 CPM, bids $10 and wins at $10, no surplus is captured. Bidders, therefore, need to construct a strategy that includes “bid shading”—the reduction of their private value to a bid that captures a surplus while still beating competing bidders with lower private values. Put simply, the more competition you face, the smaller the gap between your private value and your optimal bid. The underlying math is presented in depth in a blog post at Mathalicious, but the takeaway is straightforward: Against “n” opponents, the dominant strategy is to bid n/(n+1) of one’s private value. The visual below illustrates how this plays out in a case where a bidder’s private value is $5:

Figure 5: Optimal Bid Strategy in 1PAs: “bid shading”

Unfortunately, this ideal approach to bid shading is technically impossible to execute in the world of programmatic advertising due to information deficiencies on the advertiser side. While advertisers do have access to certain metrics related to the auction like submitted bid values, clearing prices and win rates, one thing they never see is the number of competing bidders. In fact, no single party in the programmatic supply chain sees all bids:

• An advertiser can see their own bids but not those of other advertisers.
• A DSP can track all bids it submits for a given impression but cannot see those of other DSPs.
• An SSP can see all bids that it receives for a given bid request but not those received by other SSPs.
• A publisher does not see all bids, only the subset that survives each SSP’s own 1PA.

Constructing a dominant 1PA bid strategy requires information that no one in programmatic advertising has: the number of competitors a given advertiser has in each auction. Without this knowledge, constructing a dominant bid strategy for 1PA is impossible.¹⁰ Bidders will necessarily realise suboptimal outcomes compared to what they could achieve with a classic 2PA, which does not require knowledge of competitor bidder volume.

Auction theory predicts that the move from the programmatic 2PA to 1PA will be negative for advertisers. Irrespective of any changes advertisers make to their bid strategy, it is a mathematical certainty that clearing prices will increase. Separately, the information-poor auction environment that bidders encounter makes it impossible to implement a dominant 1PA bid strategy.

Empirical results

We set out to see if the predictions of auction theory would be supported by field tests in the current programmatic auction environment.

Hypothesis

Acknowledging that, in a two-stage auction, using a second-price auction in the first stage can lead to undesirable losses due to an uncompetitive second price being passed to the final first-price auction.

We hypothesise that these losses are nevertheless outweighed by the superior buyer surplus produced by the second-price auction format.

Experimental design

Our test was designed to be as favorable to 1PAs as possible. Of our three cells, only one was a 2PA. 1PAs were granted the benefit of a cell with an auto bidder. We assumed that if our predictions about the effects of changing auction types were inaccurate, 1PAs using a DSP’s bidding algorithm would easily outperform 2PAs using a simple flat bid strategy.

Figure 6: Our three-cell test

To construct the test cells, we relied on SSPs’ own reports of their auction type, our DSP’s guidance and confirming coverage in the advertising trade press. The 1PA test cells only utilised SSPs that had publicly switched to this auction format. For our 2PA test cell, all media ran through a single SSP that was widely known to only conduct 2PAs.

The test cells ran concurrently through a single DSP, all with a simple demographic target serving across the United States. All media was bought exclusively from authorised sellers as declared by publisher ads.txt files, so we know that none of the inventory was “spoofed.” Each test cell also had the same daily rate of spend in order to control for volume of demand as a confounding factor.

Analytical controls

Conscious of the complexity here, we took steps to control for certain factors when conducting our analysis of performance:

• We restricted our analysis to the publishers where we had significant delivery across all three test cells.
• Since different ad units (300×250, 728×90, etc.) vary in price, we controlled for ad unit type.

Findings

We found that our 2PA test cell with flat bids outperformed both 1PA cells. While the use of an auto bidder reduced clearing prices relative to the 1PA cell with flat bids, these prices were still higher than those returned by the 2PA auction cell.

Figure 7: Aggregate Results: CPM and vCPM (second-price flat bid CPM indexed to 100)

There was no significant variation in either viewability rates or in-demo delivery accuracy across our test cells, which tells us that we were serving inventory of a consistent quality and reaching the same types of users. That is important because it contradicts any claims that 1PAs provide access to “more premium” inventory.

Figure 8: Aggregate Results: Viewability and In-demo Accuracy

We further saw that these patterns were repeated publisher by publisher, even when we controlled for variations in ad unit composition:

Figure 9: Results by Publisher

In 14 out of 15 publishers we analysed (these were the publishers with significant delivery across all three test cells), we saw that our 2PA cell with flat bids outperformed the 1PA cell with the DSP auto bidder. It is likely that if we had run a fourth cell, using 2PA combined with the DSP auto bidder, the relative performance of 2PA would have been better still.¹¹

Avenues for future research

It is possible that improvements in DSP bidding algorithms could shrink the gap between 2PA and 1PA clearing prices. If the difference were to become small enough (<2 percent), arguments about auction mechanism design would be relegated to the realm of the academic. We will continue evaluating DSP bid algorithms as they improve.

SSPs are also getting into the bid optimisation game: Certain SSPs offer a bid-shading algorithm at no additional cost, which our preliminary testing shows reduces 1PA clearing prices below the levels described above. 2PAs are still cheaper than 1PAs when we use SSP bid shading, but the approach bears further exploration.

What now?

Here are our recommendation for advertisers. Theory and our experimental data both point in the same direction: 1PAs are broadly bad for advertisers. Whenever the same inventory is accessible via both 1PA and 2PA types, advertisers should generally prioritise 2PA. To be clear, our recommendation is not to avoid 1PAs at all costs and in all circumstances, because there is trade-off to the two-stage 2PA auction form: It delivers lower prices, but the highest bidder will sometimes lose the auction. As a result, advertisers should calibrate their inventory strategy to the needs of their campaigns.

The guidance to prioritise 2PAs is most applicable to large-scale, broadly targeted buys that aim to acquire only a small fraction of the relevant inventory. In circumstances where winning as many auctions as possible is essential, advertisers should be more open to bidding in 1PAs as long as they have bid shading active in their DSP. For example, a local-market campaign aiming to saturate a target audience, an e-commerce retargeting campaign or a campaign targeting a small CRM list of potential customers would all be ill-served by eschewing 1PAs entirely.

Private marketplace (PMP) deals  offer a way out of the programmatic auction quagmire, with some limitations. It is possible for advertisers or their agencies to negotiate fixed-price PMP deals, wherein the buying party is guaranteed a stable clearing price through time. The trade-off is that each deal must be negotiated and monitored individually: The onus is on the buying party to ensure the price is fair and the deal is liquid. After all, a deal that is more expensive than open market or unable to deliver any volume is no deal at all. Ultimately, the utility of these PMP arrangements is limited both by an advertiser’s negotiation and administrative skills, as well as the willingness of publishers to make favorable terms.

Most advertisers will need to participate in some quantity of 1PAs, so they should undertake experiments such as those described in this paper to determine what combination of DSP and SSP bid-shading methods yields the best results for their campaigns. Advertisers should also direct investment to prudently negotiated and administered fixed-price PMP deals. For most advertisers, it would be advantageous to retain an agency such as Hearts & Science to design, implement and evaluate the results of such a bid-strategy testing regime, as well as to negotiate and administer PMP deals.

Proposal for the future development of the auction system

The adoption of header bidding and 1PA has come with benefits but left us with a profoundly flawed auction system. From a bid-strategy perspective, programmatic auctions are advertiser-hostile when compared with their search and social counterparts. This article does not suggest the current programmatic 2PA system is the answer. On the contrary, the specific claim is merely that for advertisers. The existing programmatic 2PA is generally superior to the 1PA system presently being promulgated by various interested parties.

Going forward, we should work as an industry to migrate to a single-stage 2PA. Facebook and Google figured this out years ago and constructed their auction systems accordingly. Aggregating all advertiser bids into a single second-price auction is the optimal design for online ad sales because such a system:

1. Always allocates a given impression to the highest bidder
2. Allows a simple dominant bid strategy
3. Only requires a single auction (low latency)
4. Maximises publisher yield because
   • All bids are surfaced to the publisher—there is no bid attenuation
   • Revenues rise as more bidders join an auction¹²

For this move to be publisher-friendly, it is critical that all advertiser bids are being transmitted to the publisher before the change to a true 2PA is implemented. Moving to 2PA without making this change first would result in depressed publisher yields due to the current environment of artificial bid sparseness.

Restructuring the auction system will be a significant undertaking. There are also multiple technical means of achieving the goal of a unified single-stage 2PA that captures all advertiser bids. A future paper will explore the mechanics of how this could be achieved from a technical perspective.

Even if we bring that structural change about, however, mirroring auction design alone isn’t enough: The programmatic ad auction must also become as trusted and predictable as those conducted by Google and Facebook. To establish the credibility of this new system and escape the long shadow of past misbehavior by various stakeholders, publishers should submit to an auction audit conducted by a disinterested third party such as the MRC.

Conclusion

Auctions are the beating heart of modern digital advertising. Every day, billions of impressions costing hundreds of millions of dollars are auctioned, sold and delivered across the United States.¹³ Auction design therefore has commensurate economic importance to all digital advertising stakeholders. Somehow, though, auctions have managed to avoid the close scrutiny—and accompanying calls for standardisation and transparency—engendered by topics like brand safety and viewability.

That needs to change. Right now, advertiser resources—which fund the entire ecosystem—are being squandered on the needless opacity and complexity of the current programmatic auction system. While advertisers can and should take measures like bid shading to cope with the present fragmentation, it is in their long-term interest to demand a transparent, independently verified auction system that supports straightforward bid strategies. Hearts & Science looks forward to working with its clients and partners to make this a reality.

Acknowledgments

Elijah Halpern, Sam Hochman and Ryan Verklin on the Hearts & Science Digital Activation team were invaluable in designing, running and analysing our test of different auction types. Without their contributions, this article’s arguments would lack empirical heft.

Chris Principe, senior director of digital activation at Hearts & Science, patiently educated me on the evolution of waterfalled SSPs to header bidding. Any coherence in that section of this article is solely attributable to him. Chris was also indispensable to the review and integration of feedback on the first draft.

Greg Dreifus and Mark Oster at Resolution Performance Media provided extensive feedback on this paper, as well as further insight into the technical underpinnings of the auction system. Our conversation about the trade-offs between the different auction systems shaped the “recommendation for advertisers” section.

Megan Pagliuca, chief data officer at Hearts, offered steady support, encouragement and patience.

Michael Griffiths lent this project his inimitable critical eye, spotting awkward phrases and tracking down intellectual forerunners.

Julie Van Ullen wrote the first article I am aware of, recommending that programmatic advertising move to an entirely new auction system, with all advertiser bids flowing into a classic second-price publisher auction. This paper’s recommendations on reforms of the auction system are materially similar.