I’ve got a bit of catching up to do – I originally wrote this post up on Monday, and while I feel like a lifetime has transpired since then, I think it’s still worthwhile to respond to a few great articles that came out in the last week.

The first I’d like to explore is a post from Scott Messer that got great engagement – https://www.linkedin.com/posts/smesser_programmatic-adtech-curation-activity-7383464041798623232-7iNT?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAOyvR8BP2jkp5oV9vvOrVvzgEItupNdUsc - on Friday, after the podcast, Scott pinged me with a thought – that Prebid would be for SSPs and OpenAds would be for DSPs – and I said you need to post that right now. It’s a very interesting thought.

I cheekily withheld from him why I thought it was interesting. It’s an interesting premise because when I reflect upon the reasons that Jeff outlined as his justifications for their creating OpenAds, I realized none of them materially affect SSPs. This alone, without transaction ID, is indicative enough of the diverging interests of legacy SSPs and legacy DSPs to justify separate auction mechanisms to satisfy their unique requirements. Let’s dig in.

Last friday on the show Jeff outlined three reasons to create OpenAds – Obfuscation, Duplication, and Sometimes Lying. People have been up in arms about these characterizations, but broadly speaking, I think they’re objectively true. Without re-explaining them for the 1000th time, I came to the realization that SSPs overwhelmingly benefit from these behaviors while DSPs optimizing to performance are actively hurt by them.

SSPs benefit from them in numerous ways.

Obfuscation? They would be the ones doing the obfuscating, so they’re always aware of it. And if an SSP is choosing to obfuscate, it means they’re doing it because they think the DSP is going to bid more money with less information. It’s a tool in the SSP kit. I can’t say it’s my favorite, but hey, until literally the past 2-3 months DSPs haven’t seemed to care so who am I to judge.

Duplication? From a curation perspective, SSPs are immune to lots of duplication – if anyone is duplicating things, it’s either happening on a page where they ostensibly can see it (and would take the time to), or they would be the ones doing it. They can also work with vendors like Gamera to gain powerful optics into on-page stuff, and a few SSPs do (we work with a couple of curation platforms – as always, hit me up to learn more). From a non-curation perspective, where SSPs have a different job – to just get the right user IDs in front of DSPs as many times as humanly possible - duplication is so effective that it has become a business model unto itself.

Sometimes Lying? I think this is a nod to ID bridging, but there are plenty of other forms of fraud. ID bridging satisfies the unsophisticated DSPs insatiable thirst for User IDs – this, like duplication, has literally become a business model unto itself.

All of this being said, assuming these things are problems for buyers, in the current DSP SSP paradigm, curation actually solves all three of these problems pretty darn well. This is because SSPs aren’t shackled with the sins of the father, namely, they’re not expected to be a “neutral ad purchaser from all SSPs.” In my estimation, most of these problems are only really issues if you can’t buy directly from publishers, and by virtue of the fact that curation gets to just buy from within an SSP, curation is essentially each SSP building their own version of OpenPath (which is brilliant, and certainly good for buyers who use it – as long as competition exists).

What makes this whole debacle 5-D chess is that the SSPs get to have their cake and eat it too. By manipulating the auction infrastructure to support the three sins, and also offering curation, they get to create the disease while also innocently selling the cure. DSPs, meanwhile, look selfish when they roll out their own version of direct to publisher technology, and for certain unsophisticated buyers (folks just trying to deliver against UserID lists), buying across the three sins actually performs better (later in this article on bad incentives fucking up our whole ecosystem). Hats off to the SSPs for majestically satisfying both ends of the performance and brand spectrum simultaneously.

So yes, I think Scott’s description is interesting – because we now know that Prebid is largely built by SSPs for SSPs, and OpenAds was built to solve problems that SSPs don’t have! We’ll see if other DSPs get the memo and actually try to create efficiency and performance for their clients the way that TTD is. Is OpenAds the only solution to this? No, I think not. I think people will come up with other creative ones, there will be iteration, and there will be improvement. But did we need a catalyst for change? Yes, we certainly did.

Another closing thought for this one – with the Google remedies coming up, there will be a proliferation of Adservers that offer meta-auction management and direct sales management that are super cheap. If I were a prebid management company, I’d be building this, like, yesterday. Ping me if you’d like some advice!

The other article I’d like to address is my buddy Rob Webster’s https://www.linkedin.com/pulse/saving-open-web-why-we-need-stop-selling-programmatic-robert-webster-sdhoe/?trackingId=8BAH2wLgQtyq778WOrwE5A%3D%3D who is a buy side guy doing a lovely job of getting the message out that Programmatic doesn’t perform and some heads need to roll about it (shout out to Jana as well).

Rob captures a lot of things very well and I generally love this piece (especially his vision of the future, which I can get behind), but I want to dig into this because I think it’s a refrain that goes unexamined :

“This is the structural rot at the heart of the open exchange. It’s not that people are evil; it’s that the incentives are backwards. As Matthew Prince of Cloudflare puts it, traffic has never been a good proxy for value, yet we built an entire industry around maximizing it. Publishers were caught in a system designed to fail them.”

The question I’d like to ask is which incentives and why. Mr. Prince seems to say that publishers were just “incentivized to get traffic,” but I don’t think he digs deeper. And then Rob digs into quality and exchanges – but I think this is wrong, or at least, incomplete. And I think it’s because as an industry, we’ve become confused (or even worse, hoodwinked) about what open web programmatic is for. Once again, a reminder, facebook is generating meaningful CPMs for UGC content, and we can’t seem to get high prices for new yorker essays about luxury vehicles. There must be more to this story.

I, for many years, believed I worked in a business designed to deliver effective digital advertising for buyers.

We might not.

There is a nonzero chance that to date none of us in programmatic have worked in a business built to deliver effective digital ads.

The ugly reality might be that the primary consumers and clients of programmatic to date have been Agencies primarily concerned with a way to capture more margin. Making ads work as well as possible was simply not their priority. The job of DSPs (on behalf of many of these agencies) has been to :

1. Deliver ads to a list of User IDs

2. Facilitate kickbacks at every point in the value chain in order for agencies to recapture margin downstream that they’re not allowed to put in their initial proposal. DSP, SSP, Sales House – you name it, they were all in on it and getting squeezed.

People keep painting the commoditization of publisher inventory as technological or structural, and blaming programmatic for it as a tool. In reality, it’s not the tool that did it, it’s the users of the tool – the target audience – and that audience consisted of buyers that don’t give a shit if their ads perform or not. How on god’s green earth do we expect publishers to be able to charge anything if buyers don’t care about performance?

That’s what’s been causing the race to the bottom. It’s actually not even the user-based buying in and of itself – it’s user based buying with a complete disregard for post-delivery outcomes. How could publisher inventory be worth anything or be at all differentiated if the buyers aren’t optimizing? And I think we’re finding out now that lots of buyers haven’t been. Now, I don’t mean to paint with too broad of a brush – I’m sure there are plenty of buyers who just stopped buying programmatic because it didn’t work, and maybe even some who want to do the right thing – but there were enough of these “unsophisticated buyers” to line the pockets of ad tech companies for about 20 years.

So don’t blame OpenRTB. Don’t blame the structure. Blame the fact that we’ve built an industry to serve a set of clients that will never pay more than bottom of the barrel rates because they have absolutely no reason to – they don’t care if the ads work or not, they just want to hit a userID as cheaply as humanly (or machinery) possible without serving to something that lands them on the front page of adweek in a negative way (cue our two billion dollar cover your ass publicly traded companies).

That being said, things are changing. Chalice does case studies about buying at super high CPMs for advertisers actually concerned with outcomes. The Trade Desk, much to my boyish delight, has decided to actually build infrastructure that is dedicated to making advertising campaigns perform better, and even though all of you keep giving them shit for it, they barrel forward undeterred. It is from actors like this that the new future will flow forth, and their inevitable competitors. This is why I’ve started a company dedicated to making the web perform better by helping create high-value metadata and analytics in this new efficient ecosystem.

In response to Rob’s article I will go back to my refrain – Facebook average CPMs are between $10 and $25. This means that advertisers, at times, are willing to pay $25 CPMs for banners because the post-click CPAs are so competitive.

That world can exist on the open web. Even if we have DSPs and an “Auction Management Company” charging 30% across the two of them (I can dream can’t I?), that’s $7-$17.5 net to publishers. Can you imagine how the internet would look if that’s what website CPMs were? Hell, can you imagine how the global economy would look? And even if open web CPMs were 50% of that, we would be in a magnificently superior place to where we are now. Thank goodness a few companies are trying to make it happen.

TIDs have been beaten to death, though I know a lot of you will want to talk about them. Extending the topic, people may want me to try to talk about TTD’s press releases and what they’re working on.

But I think you’re going to get lots of that. I think the thing that is worth doing right now is trying to contextualize the moment. And if we do, I think it will provide all of us with clarity.

Before I dig in – a disclaimer. This article is not meant to condemn or pass judgment on any of these practices. If I’m being honest, I personally have both benefitted from them and been massively damaged when the industry shifted to do away with them. So don’t think of this article as judgment from an ivory tower; rather, this is my attempt at a relatively objective commentary on celestial bodies.

The thesis : The past 15 years of ad tech have included a series of improvements and paradigm shifts towards quality (damnit, Erez, might’ve been right about this whole thing, but don’t tell him).

Quality Reckoning 1 - Domain Spoofing & Toolbars

Originally, the first big scandals were related to domain spoofing (this was prior to header bidding existing). In the original ad tech platforms, in order to deal with the fact that ad tags sometimes rendered into unfriendly/hard iframe environments (sorry, this is a relatively technical concept – the idea behind a hard iframe is that the code executing inside of the iframe can’t talk to anything outside of it. Think about your ad rendering inside of a box that isn’t allowed to know anything about what’s outside of it) those ad tags allowed you to hardcode a domain into the tag. Basically, you could type “&domain=” at the end of the tag and the ad exchanges would pass that domain into the bidstream to be purchased from buyers.

Can you guess what there was a ton of in this era? That’s right, domain spoofing and toolbar traffic. Exchanges were flooded with Facebook inventory, Yahoo inventory, you name it – whichever domains drove the highest CPMs (I’m sure there’s a Fou out there who will reply and say THIS IS STILL HAPPENING!) produced by people fabricating the domains using &domain=. This is because the market at the time basically just wanted to “buy domains” for certain brands – and the technology that existed could be exploited to meet those demands. There also wasn’t a lot of post-bid detection, and as long as the report said you served to “foxnews.com” the buyers were stoked. So, participants in the ecosystem realized they could make meaningful amounts of money by “misrepresenting” their inventory – or in the case of toolbars, just not disclosing that it wasn’t being sold by the publisher itself, rather by another technology provider.

There was a massive reckoning. AppNexus shut off dozens of ad networks and significant portions of their traffic. Many ad exchange volumes precipitously declined. Companies went out of business and the ecosystem transformed. This wasn’t done because the sellers wanted to – sure some publishers were mad, but other “sellers” were the ones reaping the benefits! The change was made because buyers demanded it. And, at the end of the day, as Brian explained with many graphs – what happened was a redistribution, not tremendous shrinkage. And this will be a refrain if we look closely –

Effect of tactics on Buyers : Uncertain what they’re buying, often buying something that isn’t what it says it is. Less effective optimization.

Effect of tactics on Publishers : for publishers being spoofed, money going to spoofers and not to them. For rest of publishers, money being routed away from their supply.

Reckoning #2 - MFA

The next reckoning in my wildly incomplete history, other than the introduction of header bidding which wasn’t really a reckoning, was MFA (MADE FOR ADVERTISING). The MFA business model consisted of driving traffic to websites with hyper-engaging content (all you elitists out there, slide shows are still entertaining. Instagram is basically a giant slide show and Snapchat is disappearing slideshows) with very high ad density.

These websites came into being because buyers, at the time, wanted viewable inventory on specific audiences. Those were the only two things they were looking for – and an intelligent MFA site could deliver each of them at scales unprecedented on other websites. In case any of you were wondering, the largest MFA operations I’ve seen were capable of driving between 500 million and 2 billion advertising impressions per day. Comscore top 10, easy.

However, based on data, some buyers decided that MFA wasn’t providing incremental sales. A number of players entered the space to clean up MFA, and low and behold, MFA was rooted out of the ecosystem. Once again, certain companies went out of business. Certain sellers are making a lot less money today because we rooted out MFA. Once again the predilections of buyers dictated the behavior of the market – both in the creation of MFA, and then in its destruction.

Effect of tactics on Buyers : a combination of bad press and less-than-stellar performance.

Effect of tactics on Publishers : spend routed away from non-MFA publishers due to volume bias.

Reckoning #3 - Bid Jamming and ID Bridging

Which brings us to the current moment. The buy side has gotten progressively smarter, albeit slowly, over the past 15 years. The current paradigm on the buy side is still buying user IDs – with the caveat that the sites aren’t classified as MFA (most buyers still haven’t fixed outcomes, which would be the right way to stop buying MFA). Within our current infrastructure, there are two ways to get this money that have absolutely nothing to do with your website or your traffic – 1. Send more of those IDs (ID-bridging), 2. Make your website seem bigger than it is, so that your surface area to the bidder helps you get around throttling and pacing.

What you’ll notice is that each of these phenomena have gotten progressively less egregious. The ecosystem is undeniably getting cleaner and cleaner. Domain spoofing was basically outright fraud, MFA was certainly not outright fraud (I will fight people on this one, I wrote an article about it), but it was deemed unsavory, and ID Bridging and request inflation to get around algorithms are decidedly not fraud at all (well, some id bridging is fraud lol) – they are, however, practices that make it harder for discerning buyers to do their job, so it’s not a surprise that buyers don’t like them.

Effect of tactics on Advertisers : Uncertain what they’re buying, often buying something that isn’t what it says it is. Less effective optimization.

Effect of tactics on Publishers : Volume bias – publishers not rewarded on the performance of their inventory, rather on the ability to build a stack that duplicates and id bridges as much as possible.

This Is The Publisher’s Moment

I think at this moment we are presented with an opportunity to slay the dragon at the heart of what has hurt publishing on the web. At its core, programmatic disconnected the user and the advertising experience. What I think many have failed to realize is that the true destructive component of this is not necessarily buying “your audience” cheaper elsewhere – the real destructive component of this is that buyers became convinced that just buying “the user” was enough.

This is because programmatic has been, admittedly, built like shit. It’s been hacked together by systems that weren’t built to do this, all under the suppression of a monopolist who had everything they needed to make it not work like shit. I believe the most pernicious knock on effect has been that inventory description has been left by the wayside – and buyers exclusively optimized to UserID x Domain for many, many years. All of the inefficient (and totally rational) tactics above are downstream of this highly unintelligent and mediocre buyer preference.

This meant that all inventory itself was a commodity. This is why MFA worked. This is why the only way to increase your CPMs seems to be to directly negotiate. This is why ID bridging works, this is why bid jamming works. It’s because the primary determinant of CPM in this ecosystem is the user ID and maybe domain. In that world, all banner slots are made equal – it’s the user who drives the value. How are we at all surprised that ad density is a mess? The only way to make more money is to place more ads! When you get one of the valuable users you need to squeeze as much value out of them as possible (Gamera measures this BTW, I promise around 20% of a publisher’s users make all of the money). Simultaneously, if all ad slots are the same, how do you get share of voice? You get your commodity ad slots in front of the bidder as many times as humanly possible.

Sincera and Jounce, to their credit, began the trend of changing thinking about this. Things like ad to content ratio being valuable and correlated to outcomes is actually a big fucking deal. The foundation is laid for buyers to start actually evaluating advertising opportunities and not just evaluating users – and Gamera (my company) exists to continue this precise trend by adding real time metrics and some other fancy stuff into the mix to complement systems like Sincera or Deepsee.

As a corollary to this, buyers also want to become more intentional in what they’re buying. In a world of duplication and ID bridging this gets harder and harder, because discerning which unique advertising opportunities are which, and then buying the ones you actually want while dealing with pacing, is inherently difficult in such a noisy environment with all kinds of throttling.

This “cleanup,” if it actually happens which I hope it does, I believe will have very similar effects to the MFA cleanup. Just as MFA publishers winning budgets meant that those budgets didn’t go to other publishers, technologies that are superior duplicators and bridgers take budgets away from technology platforms that aren’t superior duplicators and bridgers. And if you as a publisher aren’t working with those platforms, or are limiting the number of exchanges you’re working with because reasons, you’re probably getting less of the pie than you would be in efficient equilibrium.

I think it is important to internalize this. The budgets are finite. Right now, we have an environment where supply competes on the ability to ID bridge (normally this is the purview of the middle men) and the ability to duplicate (which overrides bidding algorithms), not on the quality of the advertising experiences and the outcomes they drive. The drive towards quality that Jounce and Sincera tried to initiate was partially foiled by the fact that even with the data, the decision systems were being overridden. You don’t need to be a psychic to see that buyers don’t like that, and they’re going to do whatever it takes to unravel those things. I actually don’t think the minutiae matters – rather in this moment, we have practices that actively make it more difficult for publishers with quality advertising experiences who aren’t good at duplicating and bridging to be discovered by buyers, and we have a number of buyers who are coming out and saying that they aren’t up for an ecosystem that works that way anymore.

This is, in my mind, an undeniable good. There will be casualties as there have been in every quality paradigm shift. But wow – this might be the one. This might be the one that restructures the ecosystem so that it actually starts to perform. This might be the moment that grows the digital advertising pie instead of finding a way to redistribute branding budgets amongst those who are the best at manipulating the systems involved. I’m not so naive as to think that there won’t still be people who figure out new tricks, and there are always tricks to figure out. But between the line the trade desk is taking and curation (SSPs are a fantastic place to seat logic that also undoes duplication and id bridging BTW), I’m feeling as optimistic as ever. At least that’s what I’m betting my business on, and I’m planning on bringing as many publishers as I can with me to the promised land. And even if you’re a cynic about the big players, I think the toothpaste is out of the tube for this – the ability to compete on quality amongst the buy side companies will be a forcing factor for the rest of the ecosystem.

The equilibrium for facebook is around $20 CPM. That’s how well their inventory performs – that’s how much they can charge buyers, pocketing all of it themselves. And it’s ads on UGC. Can you imagine if programmatic reached an equilibrium even 50% of the way there? It isn’t impossible. We just need to break out of the old ways of doing things. I will leave you with a fact that should haunt all of us at night – Facebook has about 5-15% of the traffic of a large ad exchange. And it commands 4x the banner budget of our entire industry. Maybe it’s time to hold the “experts” to account for the fact that the ads on our websites aren’t working.

After you’ve read that and you’re suffering like the rest of us, it’s time for you to engage intellectually with what prebid has just done – available here https://github.com/prebid/Prebid.js/pull/13800 . If you’re not familiar with reading PRs, this PR makes transaction IDs bidder specific. This means the little ID associated with an auction, which DSPs wanted to use to understand all of the different ways they receive requests for a single auction from all of the exchanges, can no longer be used for that purpose. Ostensibly, this means it can now be used to de-duplicate requests within one exchange and not between them…but then again, not even really that, because exchanges are now solely responsible for grading their own homework and could theoretically fabricate them.

I want to talk through the technical details of why the old version of transaction ID (the one that was consistent across exchanges) could be bad for publishers, because I hear a lot of things bandied about as “reasons” that I’m not sure hold up to deep intellectual scrutiny. By the way, I would be happy to be wrong about any of this – just let me know in the comments with insults and ad homs if I’ve mischaracterized something, or if you’re a real lunatic – post the actual explanation of why I’m incorrect.

I will then conclude with my thoughts on the question behind the question here – because I think this is a turning point for our industry, and right now, we’ve taken a turn backwards. But let’s get on with it.

As a refresher – the old transaction ID passes an auction unique identifier downstream through all supply paths so that a buyer knows a given bid request is from a given auction for an ad unit.

Let’s hear some arguments

It allows for data leakage

The argument here is that transaction ID provides a join key that can be used to synchronize audiences across exchanges. Let’s chart this out

Exchange A

PMP 1 - Auto Intenders

Exchange User ID 1234

Exchange B

Open Market Ad Request

Exchange User ID 5678

In the current state, without cookie syncing one cannot discern that this is the same auction and therefore the same user. However, it’s worth noting, that while these ad exchange user IDs are distinct – each ad exchange needs to do a cookie sync with the DSP. This means that in reality, this request to the DSP looks like this :

Exchange A

PMP 1 - Auto Intenders

Exchange User ID 1234

DSP ID 0101

Exchange B

Open Market Ad Request

Exchange User ID 5678

DSP ID 0101

Therefore, in this scenario the DSP is fully capable of discerning that the open market ad request user is the same one as the auto intender PMP one – and could ostensibly “take” that data.

So why are people worried about the addition of a transaction ID? What data leakage are they referring to? From my understanding, here’s why :

Exchange A

PMP 1 - Auto Intenders

User ID 1234

DSP ID 0101

TransactionID 9999

Exchange B

Open Market Ad Request

User ID 5678

DSP ID 0101

TransactionID 9999

In this scenario, only the DSP has access to “DSP ID” – the join key – the synced cookie – but with the addition of the transaction ID, anybody with access to logs from both exchanges has a join key for the audience. Exchanges and DMPs have all sorts of deals for ID syncs and usage and hashing and permissioning and all kinds of madness – which means that the addition of this join key has the potential to create a data leakage issue via logs for anybody who has the logs. It has been voiced to me that this concern is about curators – and because curators have log access, ID vendors and publishers are worried about their audiences seeping out.

Is this a problem? Probably, for ad exchanges honoring their agreements. Is this transaction ID’s problem? I don’t think so. Exchanges don’t have to share these things with people. They ostensibly don’t share the DSP User IDs today, because if they did, they’d have the same exposure they’d have with transaction ID. So why do they have to share the transaction id?

Transaction IDs will expose flooring strategies

I’ve also heard concerns about price floors, but they normally just consist of the sentence “my price flooring strategy will be exposed,” occasionally with the word “sequential” or “tiered.” In light of this, I’d like to explore what data DSPs already have when it comes to price floors – and what additional data they’d be able to procure about price floors should they have a “unified” view from transaction id.

Right now, DSPs receive user IDs, pages, user agents, and a bajillion other things. This means that even without a transactionId join, their data set looks like this (assuming no ID bridging, which further complicates things as we’ll explore in a moment) :

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 1.00

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 2.00

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 0

In this scenario, the current state of things with no transactionid, the supposition is that the various price floors – even with all other components of the bid requests held constant – result in additional yield. I’m not sure I understand how or why this would happen, but it’s certainly possible I guess.

The argument is then that if DSPs see this

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 1.00

TransactionID : 5678

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 2.00

TransactionID : 5678

Page : xyz.com/yay

userID : 1234

User agent : 1310923

Price floor : 0

TransactionID : 5678

And can use that transaction ID, the price flooring strategy that’s creating additional yield breaks. I’m not sure if I buy it – but I guess maybe?

Price floors alone don’t seem that compelling to me – because if the DSP is receiving the same user, on the same page, with all of the same attributes, a bunch of times (and has historicals for this) – but with very inconsistent price floors, why wouldn’t they just wait and bid for their valuation? Even without transaction ID, they have a join key – the user id – and they can see all the floors passed at the placement level for a given user. The transaction ID doesn’t actually afford them that much different data, and I’m hard pressed to come up with a “bid low” strategy that works with transaction IDs but doesn’t work with User IDs.

Transaction IDs break ID bridging

The real beast comes in when you add in the concept of ID bridging. When I look at the bid requests above, the thing that really stands out to me is the UserID. It’s a trivial deduplication effort, and trivial to just ignore the price floors, when the UserID is the same across all of the bid requests. Basically, any argument that you could make for using Transaction ID you could effectively execute using UserID with the same logic.

Frankly, user IDs should look like this and this should work. ID resolution should happen at the publisher level. But it doesn’t always – it often happens at the middle man level – and I think that’s why middle men are able to so effectively harvest profits from this (and not necessarily pass those through to publishers BTW).

When you add ID bridging, and don’t have transaction ID, the bid requests to the DSP now look like this :

Page : xyz.com/yay

DSP userID : 8574

User agent : 1310923

Price floor : 1.00

Page : xyz.com/yay

DSP userID : 2634

User agent : 1310923

Price floor : 2.00

Page : xyz.com/yay

DSP userID : 0984

User agent : 1310923

Price floor : 0

If I’m an algorithm, or even a human, and I’m looking at this – this really looks like different people getting different ads. I have two reactions to this – 1. I’m not at all surprised that this is an effective yield optimization technique, 2. I think this is a form of fraud.

It makes more money because different users will have wildly different valuations, and now the variable price flooring strategy can really work.

The reason I think this is a form of fraud is this impression can only be one person. It can only be one DSP User ID, at least that’s how it’s supposed to be in the buyerUID field (which is what I’m referring to with user ID here). This means that someone in this scenario is lying – advertently or inadvertently. I think that person is almost never publishers, and almost always an intermediary.

But we need to ask ourselves as an industry – is this acceptable? At what point does this become willful misrepresentation?

So, DSPs are fed up with this. They want to have a transaction ID tied to this – so now these requests will look like this –

Page : xyz.com/yay

userID : 8574

User agent : 1310923

Price floor : 1.00

TransactionID : 5678

Page : xyz.com/yay

userID : 2634

User agent : 1310923

Price floor : 2.00

TransactionID : 5678

Page : xyz.com/yay

userID : 0984

User agent : 1310923

Price floor : 0

TransactionID : 5678

This join key makes it very, very easy for them to see “bid request A from partner A came in with this userID for the same bid requests from partner B with this other userID.” And then they can drop the hammer on partner B if they want to and also ensure they’re not bidding against themselves on what they think are different users but are actually the same.

Now, is the transaction ID actually necessary? Eh – Kind of. The DSP could also check its logs once it serves to the page and true up the IDs declared in the bid request with the IDs they found in the browser. I would argue this is just as effective for finding ID bridging, and just as effective for dinging supply paths. And if they unraveled the ID bridging by refusing to pay for every ad rendered to a misdeclared buyerUID, they can now use UserID (with other fields) as a transactionID.

So while the transactionID is a valuable datapoint in hunting for ID bridging malfeasance, I’m not certain it’s a necessary one. But I do also think that ID bridging is actually a far larger problem than has been discussed to date.

Downstream of ID Bridging you have Bid Jamming

If you look at every one of my examples, the assumption is that DSPs are going to have the same thing presented to them multiple times, often with radically different valuations each time because of a bunch of different fields within each bid request. There’s a reason this isn’t something mentioned explicitly by publishers in all of the transactionID discussions – and it’s because people tacitly know that this isn’t something present in a healthy market. Basically, the ID Bridging problem gives birth to a bid jamming problem, where bid duplication has the effect of making the algorithm think you have a lot of differentiated inventory when in reality you’re selling one thing. You get to test out a bunch of different values in a bunch of different fields to see what elicits the highest bid from the algorithm, and this crushes it for yield.

The elephant in the room is that there’s a universe where transaction IDs, when leveraged to their full extent, could theoretically help to kill all of these things at once. A DSP could theoretically bid once per transaction ID. That is, assuming they’re capable of maintaining state in real time for “have I served to this transactionID” across like 10 gazillion QPS, which like, maybe?

But if this were to actually happen there are a lot of people who will make less money – a supposition substantiated by the fact that today they indisputably make more money by engaging in bid jamming combined with ID bridging.

While people may cite other things, this is what scares people. Because it’s created so much lift for publisher revenue the notion of losing it is a frightening one.

Conclusion About Stated Arguments About TransactionId

My goal with this examination is not to dismiss the publishers’ concerns with transactionId. Well, actually, it kind of was. But in the course of doing so I hope I also drove home the point that we wouldn’t necessarily need transactionId to solve a lot of the problems that transactionId solves if there weren’t a bunch of things happening at once that back DSPs into a corner. For instance, I think if we successfully killed ID bridging, it would be a lot easier for DSPs to SPO and deal with bid jamming, because user IDs would be a lot closer to transaction IDs. But that’s not happening – and I’m not sure whose fault that is at this point.

This is where we get into the foundation of the “public” sell-side argument. If these problems can theoretically be solved without transactionId, why do we need it? Why not just do what gareth says and fix id bridging and then build your own strategies for bid jamming?

And I understand that, I really do. However, the notion that taking away something that makes auctions more efficient (transactionId) could be good for the long term health of independent programmatic is a hard pill to swallow for anyone who believes in the future of the industry. Furthermore, I’d be remiss if I didn’t lament the lack of public debate about this outside of my articles and a few slack channels that most of the industry doesn’t participate in.

What We Should Actually Be Debating

But this is all for the “public” argument. I think there’s a far more consequential “private” debate that we should be having – and that’s “is bid request duplication sustainable?”

I don’t think people spend a lot of time thinking about this. Smart people know if you do it more, you make more money. But just because smart people can tickle the ivories of programmatic to make more money doesn’t mean that this is a “sell side” desirable thing. What’s good for the goose isn’t necessarily good for the gander, and not all publishers were created equal. Just because you’re a publisher doesn’t mean bid jamming necessarily benefits you – in fact, if bid jamming becomes an optimization vector, and you’re not a master of it, you’re being harmed by it. Spend is being routed away from you if you aren’t doing it as aggressively as possible today. Transaction Id therefore benefits publishers who are not bid jamming beasts, and hurts publishers who are. I would expect the publisher debate about this to be more robust, which is kind of why I find it surprising that this is a “DSP vs publisher” debate and not a “bid jammer vs not bid jammer” debate. DSPs are not going to suddenly “spend less money” because transactionId allows them to deduplicate bid streams. That’s as bad for them as it is for publishers!

Even beyond that, and beyond the arguments that have been publicly discussed, the problem here is a fundamental independent ad tech one. It’s that we are running our industry like Jon Hamm in Mad Men. In writing this article I remembered that originally people managed sales conflict by obfuscation – by hiding what people were actually buying from the buyers. I think this mentality hangs over to today – where people think “information should command an additional arbitrary price,” and that somehow there’s more money to be made when you withhold data.

I have news for you. This strategy has failed. Programmatic is not gaining marketshare, it is not growing, and it is not trusted. Programmatic’s competition, the walled gardens, do not withhold information from themselves. From their clients? Maybe. But not from their algorithms.

We have enough challenges to overcome – it’s an own goal to keep trying to run our business like a 1950s advertising agency <> magazine publisher negotiation. There’s a simple heuristic to live by in programmatic – More Statistically Significant Data is Better. Full stop. I’m a believer that sharing data will result in everyone making more money – even if some buyers are able to get things cheaper.

Programmatic is a 30bn business. Facebook is over 100bn for display banners. There’s a lot of money that can come our way if we can make our channel actually work instead of infighting over the scraps of the walled gardens. And the buy side of the programmatic ecosystem has its role to play in this – if more data doesn’t lead to more advertising dollars and therefore more competition and money for publishers, then their home’s foundation will erode out from under them. I think they know that.

There are certainly people who benefit from obfuscation. We need to decide if the obfuscation is worth stifling the growth of our industry.

Thanks to a LinkedIn post from Mr O’Sullivan, there has been a heated debate about something called a transaction ID in bid requests. I’d like to explore this topic in detail, because this is one of those scenarios that manages to make everyone look bad, which is an opportunity that a hater like myself cannot let go unattended.

The thesis behind transaction IDs is simple – in the Prebid auction, which is really the core programmatic auction, we would assign a unique “transaction” ID to each execution of the publisher ad stack. This transaction ID would be signaled out to DSPs, letting them know that regardless of how a given bid request gets to them it’s still part of a unique on-page auction. This would allow them to know that bid request A from partner X and bid request B from partner Y are actually from the same prebid auction, compare the metadata in the bid requests, ostensibly not bid into that auction more than once, ensure things are as they’re supposed to be, detect ID bridging and other tomfoolery, and all manner of good things for transparency and programmatic health.

O’Sullivan innocently asked “why wouldn’t we do this?” And personally, I think it’s a radically fair question. Purposefully obfuscating the actual impression opportunity auction from the buyers just seems like bad faith to me – it’s one of the myriad reasons that programmatic works like shit, because all of this garbage programmatic shenanigans makes it impossible to actually optimize anything and bid intelligently.

However, not so much to my surprise, the publisher community reacted to this with vitriol, or at least a few people did. The defenses were about what you’d expect, mostly flavors of “yeah so you want me to stop playing the game and make less money? You can pound sand.” Here’s a brief summary of the arguments I saw:

1. Obfuscating final auction allows us to use price floors to manipulate bidding more effectively

2. Obfuscating final auction allows us to bid jam / request duplicate more effectively and evade detection

3. Obfuscating final auction makes id bridging reconciliation hard, and id bridging makes us mucho dollares

4. Obfuscating final auction prevents people from mapping user IDs between exchanges without paying hash tollkeepers (it’s a join key, the primary threat coming from middle men like curators who could use it to back into data segments

The first three are what I normally refer to as “gaming bullshit (GB) .” And I get it. I really do. It makes more money – I mean hell, I am the bid jamming guy. I have been advocating for bid jamming since before it was cool – see my article here from september 2023 https://www.garethhatesadtech.com/p/bid-jammin .

But I’ve evolved, and while I still advocate for some GB, it’s not for the same reasons as many of you rapscallions out there who love GB.

A bad argument for GB is that “it makes me more money.” It’s bad because it’s potentially myopic – we don’t know what the knock on effects of these techniques are in terms of making programmatic work badly and nobody is doing a good job of measuring programmatic health over time as opposed to in this moment. It’s on my roadmap, okay?

However, because publishers aren’t thinking this way, it’s perfectly possible that the extra 3-5% you’re squeezing out of your programmatic auctions is making your site perform 30% worse on a ROAS basis for advertisers who actually optimize things. This is because the GB that results in 3-5% yield for publishers could quite feasibly be resulting in 20-30% higher CPMs for buyers, with the excess going to the GB middle men platforms. This generalization won’t be universal, some very talented GB people own their own pipes, but for many publishers it’s the case – the GB might be doing long term harm in the name of short term good.

However, in the interest of truth, we cannot let the buyers off unscathed. There exists a great argument for GB. And that’s that most advertisers don’t give a shit about performance.

If you’re running a DSP, and your buyers don’t give a shit about performance, that’s why you can’t buy through more direct supply paths. It’s not because “publishers and middle men aren’t doing XYZ.” If you don’t have buyers clamoring to shut off bad middle men who take high fees, you don’t need to deduplicate your auctions.

This is all to say that if your buyers actually wanted to fix things, we don’t need transaction ID to do it. Just buy through a single path. We know paths are commoditized, and we know the only “value” being added in the middle is by GB. So why keep it alive? Because unfortunately, the phone call is coming from inside the house – the primary facilitator of the shenanigans is the prioritization of middle men by your own agency clients. We don’t really need to save them from themselves. They don’t need a white knight – they just want to deliver to their list of user IDs and get back to politicking for more budgets. The tricky part here becomes when DSPs have one advertiser who cares about performance, and another who insists on their looney tunes supply path. In this scenario, when they bid against themselves, theoretically the performance advertiser might be getting punished from having additional demand in the auction from the goofball advertiser and the bidder might’ve prioritized pacing things differently – but that’s about the only use case I can think of that’s really a “rock and a hard place.”

Thus we have a situation where frankly everyone has some responsibility for a really shitty nash equilibrium. Sellers don’t want to make a change that will effectively make them less money. Buyers can’t do what needs to be done to solve this without seller action. We therefore have a Gordian knot that we need to cut.

There is but one way (well, one nice way).

Incentives.

If buyers can properly reward publishers who pass a deduplicating identifier, if they can make more money from the added transparency than they make from GB, we have an escape hatch. Do I know how this would work? No. I’m not on a product team at a DSP right now. But I think there absolutely could be a way.

Without the incentive built properly, you are effectively asking publishers to make less money for doing the right thing. We all know that sucks. That being said, there’s nothing inherently wrong with transaction IDs. I think a global join key is actually a fundamental component of a healthy and transparent marketplace. But we’re not going to get there unless the incentives are there – and while they don’t seem that complicated to me, they do seem rather difficult to align.

For the uninitiated, a dynamic revenue share refers to a pricing dynamic within an ad exchange wherein the exchange charges a different revenue share fee on an auction by auction basis. For the ultra uninitiated, ad exchanges charge publishers a percentage of revenue for the honor of selling publisher inventory to ad buyers (buying via DSPs), and this revenue share is typically written out and negotiated in a contract but not always.

Let’s start out with why exchanges would do this –

In 2025, all ad exchanges have to submit their bids to another auction (at least in banners and native and OLV) – the Prebid auction – that occurs on the publisher page or in the publisher’s instance of prebid server. This bid has to be net of their fees – namely, they can’t submit their gross bid and then extract their fees later, as this would make the auction clear on false pretenses, so they have to submit the amount the publisher would actually get paid if their bid wins. Because of this, there exist scenarios wherein the ad exchange’s fee, and not the bid they were able to procure, preclude them from winning an auction – an example :

Ad Exchange A receives a $1 bid from DSP A, charges a 20% fee, so submits a bid of $.80 to prebid.

Ad Exchange B receives a $.90 bid from DSP B, charges a 5% fee, so submits a bid of $.855 to prebid.

In this scenario, even though Ad Exchange A received a significantly higher bid, Ad Exchange B wins the prebid auction because they’re charging a lower fee.

Ad exchanges 10 years ago didn’t have to worry about this scenario, because they weren’t bidding into prebid auctions – which means their greed basically went unchecked. But now, there’s competition out there, and they have to compete both on how high of a bid they can get from DSPs and their fees.

In its most basic form, Dynamic Revshares were born out of the fact that their fees actually affect their ability to win auctions. Ad Exchange A sees the above example happening (probably because some sales person complains about Ad Exchange B eating their lunch even though Ad Exchange A is more prestigious), and decides to do something about it. They realize if their machine learning teams (lol) can properly predict what the market clearing price for an impression will be, they can ratchet down their revenue share to win the impression (because a smaller percentage of something is worth way more than nothing at all).

So, in scenario above,

Ad Exchange A receives $1 bid from DSP A, their machine learning says it thinks the clear price will be $.86 because data scientists are infallible geniuses, and recommends that Ad Exchange A takes a fee of 11% just to be safe. They submit a bid of $.89.

Ad Exchange B, who doesn’t have a team of perfect AI data scientists, doesn’t have dynamic revshares. They lose the auction with their $.855 bid.

This, in my opinion, is the cleanest form of dynamic revenue sharing and is basically totally okay. This is because the dynamic revenue shared “flexed” down, and exclusively down. There was no “harm” here because they charged less than they normally do, and this article is going to be about harm, but I would advocate that this scenario in every instance will just make programmatic more efficient.

But what about less straightforward circumstances? Hearing about dynamic revenue shares, we should all be reminded of Project Bernanke

https://www.adtechexplained.com/p/google-project-bernanke-explained

– wherein dynamic revenue shares were being used in unsavory ways.

The unsavory way I want to talk about today where instead of decreasing their fees to win marketshare, ad exchanges increase their fees to maximize their profits. This can be as a result of simple greed (if your contract permits it), or a function of trying to compensate for when they flex down and their take rates look lower than their investors would like. And if you’re a real sketchball, sometimes you flex down so hard that you actually add money from your other advertisers to your bids. Literally, taking a negative revenue share as a company, and then making up for it later by charging the snot out of publishers. This is a cousin of Bernanke, just executed with a little less market control, and it is absolutely happening.

This, in my mind, is bad dynamic revenue sharing. This is just auction manipulation, where ad exchanges really look a lot more like arbitrageurs taking primary positions on media than they do neutral ad exchanges hosting unbiased and fast auctions. But I’ve been doing some soul searching on why it’s bad, because while lots of people complain about this behavior, I have yet to see someone articulate the actual negative effects this will have on the ecosystem, outside of some loose assumption of “publisher opportunity cost,” which I think is shaky.

It’s bad because in ad tech, we have a fundamental opportunity and a fundamental problem with transparency. And I don’t mean this as a buzzword, even though it is one. I mean this from the perspective of digital advertising is fundamentally broken when it doesn’t distinguish between working media and fees. And digital advertising is uniquely in a position to fix this.

There are companies that try. Fiducia is trying. Sellers.json is maybe kind of trying, if dangling your bare foot on the ground counts as getting off of the couch. But no principal actors (ie. DSPs and SSPs) are really doing it, and that’s who we need to fix the problem.

The reason this lack of transparency is bad for publishers is because these auction shenanigans (and the surrounding lack of transparency) create scenarios where the cost of publisher media looks artificially high. In the name of middle men taking their fees, a report consumed by an advertiser, or consumed by an advertiser attribution data science god(dess), is basically saying “publisher inventory cost X,” when in reality, the cost of publisher inventory was “Y Media + Z Middle Man Fees,” where Y and Z will function in crazy ratios sometimes (there certainly exist impressions where the middle man fees exceed the cost of the media).

It is then fully possible that the advertiser or data science creature decides that inventory doesn’t work for their campaign and pause all future spend to that publisher, when in reality, that publisher might’ve been ROI positive if not for the stinky middle men.

This is harm. Hell, this is, like, meaningful harm. I am not a lawyer, but there’s a non zero chance that the way ad tech has been constructed has led to unfathomable damage to publisher revenue by not disclosing what was working media and what were fees to middle men.

And that’s the solution here – it’s not complicated – we need to denote the cost of working media and the cost of middle men fees in every transaction in the ecosystem. It wouldn’t be hard to do, we already have supply chains “modeled,” and this is a simple thing for people to append. But it is a major uphill battle to make it happen, because I’m willing to bet these shenanigans are the tip of the iceberg.

All of that being said, if you’re a lawyer who works at/with a publisher, we were able to prove damages with Bernanke. Maybe you could be the ones who light a fire under the ass of transparency with a couple of strongly worded letters…

Today I’d like to reframe things. I think ad tech has done that good old ad tech thing where shit has gotten super confusing again. People are talking about Curators, Signal, Agents, you name it – like we’ve invented a bunch of new things.

We haven’t.

We’ve just made up names for the same things happening in different places. Don’t get me wrong, the same thing happening in new places can drastically change the landscape of an ecosystem – but it can also serve to confuse and befuddle, when in reality, I am of the opinion that ad tech is healing itself and becoming simpler.

I will tease all of you with my shower thought that spurred the rest of this article, and it was a silly “omg this is true” moment for me. That thought is that I am fairly certain, with a bit of shoe-horning and an army of ops people (or agents perchance?), I could recreate the entirety of Ad Tech in Google Ad Manager (GAM).

All of it.

Creative trafficking, placement level targeting and optimization, data. You name it. I could do all of it in GAM. I’m just gonna need to export a lot of ad tags and upload a lot of line items. This is because fundamentally, all of ad tech consists of a few basic functions. Sure we’ve built a bunch of companies, and we’ve invented the ridiculous concept of “buy side” and “sell side” ad tech, and we’ve modularized the components of the system – but at its core, what OpenRTB, and what all of independent ad tech seeks to do is create a simple marketplace of supply and demand, targeting the demand to the supply. Because of this, with sufficient will, we could absolutely execute that vision within a single GAM account. It’s a wild thought. And it should scare you, because it’s indicative of the unnecessary complexity that we’ve introduced through all of our ad tech shenanigans.

With this logic in mind, I’d like to reframe the emergence of Curation. I love Ciaran and Kev’s “SSA (super signal aggregator),” framework, and their investment thesis around signal, but I think it overcomplicates things. Signals have always existed in ad tech, every component of a bid request is a signal, and data signal vendor aggregators are actually nothing new (in fact, this was the original job of the DSP). I actually think focusing on signal, as opposed to function, makes the landscape look confusing. Curation begins to make a lot more sense if you do a feature comparison of each of the companies in the value chain, and we plot those over time. So that’s what we’re going to do!

Let’s start here – pre-curation and pre-header bidding life :

I don’t expect this image to be exhaustive, sure there are probably a few features and functions in there that I missed, but this is the core of it. This was a simple world where DSPs ran campaigns, and SSPs executed their most important function – “Buyer Auction Hosting”. In this scenario, the SSPs are the holders of auctions. There are other auctions to be sure – the internal DSP auction, the ad server auction, but those auctions are not aggregating prices across many DSPs for a given impression. The SSP auction is the programmatic auction in this era.

And SSPs were closed systems, which actually really sucked. In your core adserver, because nobody was bidding, you had to traffic an estimated yield per SSP, which of course would vary wildly based on how many exchanges were called before them, and I feel quite comfortable saying that this era, while discrete and easy to understand (DSPs = Buyers, SSPs = Marketplaces), the publishers got totally screwed..

The reason is simple – the closed nature of SSPs meant that demand was wildly fragmented. That fragmentation meant that inter-SSP competition was really almost non-existent, and publishers actually had zero clue on an impression by impression basis what each impression was actually worth – and rarely were there other bidders to push the price up. So things sold cheap in auctions with relatively few participants – especially compared with a modern header bidding auction! Imagine, a single SSP, vs a header bidding auction with 30 exchanges calling 200 DSPs each – the bidder density difference is easily an order of magnitude.

So yea, this era, while simple, sucked.

But then, we invented header bidding

Header bidding, at its core, created a second auction outside of the one that occurred inside of the SSP. At first some SSPs resisted, because the fragmentation that drove their margins and market positions was directly threatened by HB (especially if you were early in the stack). But HB won. HB really won. Look how much it won here :

That, my friends, is Rubicon’s stock price between 2015 and 2018. And that’s the visualization of someone betting against header bidding, which is the embodiment of a competitive market as opposed to a fragmented one. This is because 3 b-grade SSPs participating in a competitive auction mops the floor with a single a-grade SSP in a fragmented implementation.

But I digress. The actual thing I’d like to illustrate in my first diagram is that the most important function of an SSP (in my opinion), the Buyer Auction Hosting, is now shared. Two things are doing it. There’s a SSP buyer auction, and there’s a Prebid buyer auction.

That’s redundant. It was redundant in 2016, it’s redundant in 2025. We simply don’t need two systems to manage auctions with bidding systems.

The obvious upshot of this is that the bidders who traditionally bid into the SSP auctions can now leap-frog the SSPs and bid directly into the prebid auction. I swore until I was blue in the face running RTK that this would happen and happen fast.

But it didn’t.

There are probably 5x the number of Prebid bidders that there are DSPs – what we saw is that DSPs had drunk their own kool-aid of being buy-side people, and had somehow convinced themselves that bidding into prebid was a “sell side function,” even though it was literally technologically identical. Oh, that and the fact that we came to learn that tons of Agency Holding companies have deals in place with ad exchanges, where they can extract margin at the ad exchange level and engage in shenanigans, but this isn’t a missive about corruption.

What we did see is one enterprising DSP figure this out and begin to get involved with Prebid – The Trade Desk. I remain gobsmacked that more DSPs have not gotten in on the game – but I guess I’ve heard about a few internationally. But let’s visualize TTD’s OpenPath :

Zoinks. As you can see, the redundancy (the features shared in the Venn diagram) that came about as a result of header bidding created an opportunity for disintermediation.

This makes perfect sense.

The universe hates inefficiency. It took us 10 years, but eventually this redundancy was laid bare – and it resulted in the worst case scenario for SSPs, which is one of their primary revenue sources cutting them out of the equation.

I don’t think anyone was actually surprised by this. The moment prebid emerged anyone with a brain knew “wait, if there’s another auction, why don’t DSPs just bid into that?” – It just took the DSPs awhile to pull their heads out of their keisters and get the work done. But to be honest…most DSPs still haven’t done it. And the same way that Prebid took off like a rocket as a way for SSPs to equal the playing field with Google, those wily exchanges figured out a way to prevent this DSP disintermediation.

That way is curation.

The crux of this article is that Curation is simply SSPs playing defense. Prebid was the first domino to fall, moving SSP functionality out of the SSP and making them redundant. SSPs were now resigned to slowly watching their businesses dwindle as more and more innovation happened in the primary buyer auction, the Prebid auction.

But in their infinite wisdom, they realized “If Prebid can take functionality away from us, what if we could take functionality away from the DSPs?”

Getting people to change DSPs is impossible. Everyone knows that. Ask Chango. But Prebid’s appeal was that it didn’t replace SSPs, it wasn’t an alternative, it just took one or two things that they did and moved them while still cooperating with them. In fact, it benefited them and they are the ones who designed it!

So, if I’m an SSP, I’m wondering what parts of a DSP could I move out of the DSP, thereby entrenching myself in the value chain?

Well, turns out most. And it turns out that most DSP functions actually work better when you’re close to the inventory. This is because Header Bidding created a glut of duplication and noise, and DSPs haven’t done what they need to do to make things work better (ie. pick 2 exchanges, ditch the rest). My thesis on this one is that they became addicted to buying UserID lists, and that’s an easy business to run, so they didn’t think to invest in actually optimizing what they were buying. As long as they “got their ID,” it was gucci. So they’re now sleeping in a bed that they’ve made.

So what does this functionality shift look like?

We now see the Venn diagram creeping the other direction. SSPs, through deal ID infrastructure that they’d built ages ago, actually already had the skeleton of targeting systems – all they needed to do was a little bit of workflow work to turn this into a UI a media buyer could use to change what inventory went to campaigns. And because SSPs were so close to the inventory, and inherently didn’t have the ridiculous duplication that all DSPs can’t seem to convince themselves to turn off, campaign optimization works better in the SSP than in the DSP.

And that brings us to the current day. One thing I’d like to emphasize is that this trajectory hasn’t actually created any new functionality. What it has done is moved features and functions from one class of company to another – which brings me to what I think is the important conclusion of this article

Buy Side and Sell Side frameworks for viewing the industry are outdated.

There are Bidder Networks and there’s Prebid. That’s it. Prebid is the monetization auction, and the Bidder Networks are the bidding mechanisms. Maybe they’re DSPs – maybe they’re DSPs + SSPs + Curators – but at the end of the day, the output of those systems is a bid, and the functionality within them is identical in that their simple goal is to derive a bid and spend budgets.

This is a good thing.

The concept of buy side and sell side holds us back when competing with walled gardens – it puts two hands into the cookie jar when walled gardens only have one, not to mention the data asymmetry and inefficiency that comes from integrating two ancient ad tech companies. Andthe Google ruling means that things are going to get even more egalitarian if AdX has to bid into prebid, at which point we’ll finally get pretty close to recreating walled garden performance (yes, the final hurdle for the open internet is the artificial insertion of GAM into the monetization flow).

But the world is indisputably getting simpler and therefore better. We now have bidders, in a few different flavors, and we have Prebid. Soon, we might even be free of GAM. And just in time for LLMs to destroy the internet as we know it, we might actually build systems that rival Google and Facebook.

The issue, and one that is quite possibly seriously compromising the performance of independent Ad tech, is Throttling.

What is Throttling

In the world of ad exchanges and real time bidding, things are communicated via packets of information called “Bid Requests.” A bid request is a bundle that includes metadata about an advertising opportunity – IP address, user IDs, page URL, etc – and these bundles are used to evaluate the expected value of a given advertising opportunity for an advertiser within a bidding environment.

From the ad exchange perspective, and getting a little technical, depending on your infrastructure (but especially if you are working with off the shelf cloud providers like Amazon), one of cost centers can be something called “egress.” Egress refers to the cost associated with transmitting information to other systems, or in this scenario, the cost of sending bid requests to bidding environments. This means that just by virtue of working with a bidding environment, exchanges incur additional costs – the costs associated with sending out the requests to that environment.

Simultaneously, from the buy side perspective, there is a cost to evaluating the given advertising opportunities against your expected value model. This is generally related to “compute,” or processing costs, but there are other costs as well. It generally costs even more money than egress does. This means that for open market bidding systems, there are fixed costs to operate the infrastructure regardless of spend.

In general, companies don’t like costs, and want to avoid unnecessary ones, or even “non-optimal” ones – which brings us to the birth of throttling.

Throttling refers to the act of limiting the number of bid requests that go to a given bidder, at least originally, to optimize costs. Because of egress, and because of compute, it is in the interest of both the sell side system and the buy side system to not incur costs for bid requests that will never result in an impression transaction. So, basic initial implementations were “I, the world’s greatest DSP, have no advertisers in Brazil. Do not send me Brazilian traffic mr. ad exchange. Or do, and I’ll just stop working with you, because you’re a commodity.”

Throttling Grows Up

But then the nerds got involved. I’m not sure if this started on the sell side or the buy side, but I’m assuming it’s the buy side, and some machine learning person went “hey, I bet I can design an algorithm that dynamically listens to more of what we want and less of what we don’t want. It will improve outcomes for our advertisers, because I’m a data scientist and I know what’s right, and also save us money. Give me a promotion please.” And then, in the infinite wisdom of DSP leadership, they went “Fuck yea, let’s give this person more money and power.” And thus dynamic DSP throttling was born, where DSPs had internal algorithms to choose which advertising opportunities they evaluated for bidding and which ones they just ignored completely.

SSPs all of the sudden saw bid rates (the rate at which DSPs bid on their traffic) wiggle all over the place. Some poor account manager probably reached out to the DSP and went “what is happening?!?!” to which the DSP replied “lol, get better traffic bro,” and then the SSP leadership team got fired by their board for not knowing what was going on. But then, genius idea, someone at the SSP realized the inverse of the DSP realization “It would seem the DSP bids more often according to some kind of horrible arbitrary pattern. What if we send more stuff according to that arbitrary pattern?” Honestly, it was probably the same machine learning person who built the dynamic throttling algorithm at the DSP. And they said to their leadership “Hey, if we dynamically change which bid requests we send to the DSP, we can manipulate their bid rates and get them to spend more money with us. I know, because I built the DSP algorithm, and I know how to trick it. Wait did I say that out loud? Nevermind, oh, and by the way, egress costs will go down too!” The new leadership team, put in place by the board, saw revenue going up double digit percentage points with their magic new algorithm and made this person a C level.

And this brings us to where we are today. Throttling is a mess. It is a game of machine learning algorithms trying to learn and optimize to other machine learning algorithms, based on signals that probably have very dubious statistically significant optimizable value in the first place. This has resulted in two even worse things :

1. This dynamic throttling war is partially responsible for bid jamming. The more systems throttle, at least initially, the more money you make by slapping as many bid requests into their face as possible. This is not publishers’ fault. This shit worked because of these dumb throttling algorithms.

2. Now, publishers are implementing their own machine learning algorithms manipulating the ad requests that go to exchange from their prebid auctions – because now exchanges throttle inbound too, it’s just great – the cost incentives are different because sending requests from the client doesn’t cost publishers money, but certainly ad exchanges throttle what they listen to from publishers, and building algorithms to game their algorithms indisputably can make more money from publishers.

We are now in the endgame of this vicious cycle of throttling spend down and then building algorithms to game and throttle spend up. I put it at about the 4th circle of hell – we aren’t quite with Satan in the ice dungeon in the middle of the earth yet, but we’re basically swimming in feces up to our necks. Or maybe that was the 6th circle, I don’t remember.

And this bullshit makes it really difficult when you want to actually fix the ecosystem. I know this because my company is fixing inventory categorization – I think ad placements, and sites, and everything having to do with inventory classification in bid requests could be much, much better. And we built a better mousetrap. Gamera classifies inventory more effectively than a GAM ad unit ever could, models performance, and creates stable identifiers that increase the performance of ad campaigns – helping publishers put their best foot forward to buyers and helping buyers reduce CPC and CPV and CPA by 35%+ (I’ll stop selling now I promise). But, because programmatic is busted, I can’t transmit my data directly through the pipes, so I’ve given up on that for now.

But what we do do is curate. We build PMPs (and help others build PMPs in combination with our inventory signals) using our data. And because of those stupid throttling algorithms, that are optimizing to signals that are awful, it’s a magic guessing game of the percentage dropoff of signals we send in via the client, signals sent out in bid requests, and therefore signals heard by DSPs.

In some circumstances, we see 99.9% dropoff rate from signals sent in via the client to signals heard in the DSP. 99.9 fucking %. Nobody at the DSP or the exchange can explain why – because it isn’t humans doing this, it’s machine learning algorithms optimizing to machine learning algorithms optimizing to other machine learning algorithms. I swear, the amount of money paid to these throttling analysis people probably could’ve cured some horrible disease by now.

And this leads me to my conclusion for this article – which is that these throttling algorithms, because they’re making decisions based on imperfect bid request data (read : garbage inventory identifiers and user IDs) and based on other machine learning algorithms are indisputably making the industry more inefficient. It means there are advertising opportunities that buyers are missing out on, which means buyers get shittier results and that publishers make less money. This is a direct result of how the infrastructure is designed, and the vicious cycle of bid jamming and throttling on throttling.

So how do we fix it?

I think the fix is rather simple – in 2025, there’s no reason to be listening to more than 2 exchanges, and DSPs should compete on who throttles the least (and therefore who can perform the best). I remember a call with Jeff Green, well, my only call ever with Jeff, where I pitched Jeff on the direct, fee-less exchange I’ve written about before. His response was “we’re TTD, the fact that we can listen to everything means that we can find opportunities our competition can’t. I don’t need one exchange.” His position was an anti-throttling one at its core – basically, throttling makes our adversaries weaker. I couldn’t argue with him, because he wasn’t wrong. And his statement rings true to this day, probably 8 years later.

There will always be listening and bidding optimization – a tradeoff between cost efficiency and performance – but Ad Tech needs performance to win if it’s going to survive.

Thanks for reading! And try out my data, it works great, or if you’re a publisher, make sure you’re using us for your direct sales (amongst all of our epic publisher products). Or that we’re at least implemented so that when we fix the internet for advertisers and DSPs you’re included in the party.

No, this is not a retraction.  I wasn’t fully wrong about curation.

As a technologist, curation still doesn’t make sense to me.  If we’re looking for optimal outcomes for advertisers on the open web, all of my prior criticisms still apply.

I was wrong about the focus of my criticism – and about the reason for curation existing.

My prior assumption was that curation existed because of fragmentation of data.  That isn’t actually what it is, though that is the guise it hides under.

Now that I’ve seen the guts of curation, curation exists because DSPs are dropping the ball on performance.  And enterprising SSPs, being wiley as they are, have discovered a way to take advantage of this.

I am reimagining curation.  It is not a data play.  It is not a technology play in any way.

It is a pure business model play, and that business model is that of the affiliate network.

Exchanges have built their own agency sales/adops affiliate networks.

For those of you unfamiliar with the affiliate model, it’s a pure play performance model wherein the affiliate gets compensated when they cause an outcome to occur – and they’re often compensated a % of the revenue generated by the outcome.  In this scenario, the outcome is getting an Agency to buy against a PMP, and it’s an affiliate platform on steroids, because the affiliates get to construct their own custom offers (custom PMPs) based on a menu of things provided by the ad exchange.  This is genius.  In the curation affiliate model, the curator then gets to set their own terms – often up to 30% of total spend!  An unimaginable amount of compensation to an adops lead who creates tremendous results for their brand or agency – but not for a curator!

But, like I said in my prior article, these crazy numbers are not meant to be a condemnation of the affiliates (the “curators”) or the exchanges.  They’re executing a business model that is inarguably adding value for someone, because their fees are deducted in-flow, and the bids through their dealIds are still higher than all of the other bids in the auctions (assuming they win), at least in header bidding unified auction environments.

The question that we need to be asking ourselves is…why is this possible?  Why is it possible for a sales person, often with zero proprietary technology, to bundle up some inventory and sell it directly to agencies and those bundles outperform other strategies in the DSP?

There is a potential explanation.  Many industry old-heads repeat the refrain that the only system in ad tech that ever worked for performance was Right Media.  The theory that I’ve heard put forward there is that unlike OpenRTB, which allows broad-pipe integrations between demand and supply, Right Media had a very “manual” linking process for what were essentially mini-exchanges within Right Media.  This meant that the optimization algorithms got a new bite-size chunk of inventory to optimize to with each new link, but none of those chunks were ever anything near the size of the modern ad exchange (especially in the world of header bidding).  The optimization algorithms, therefore, were able to build more accurate predictive models for performance, because they weren’t constantly trying to “boil the ocean” with every new campaign.

There’s a nonzero chance that Curation is simply narrowing down the pool of inventory that DSPs optimize to, and by virtue of that fact (limiting the number of places that the optimization algorithm has to consider), improving performance by enough to justify a 20-30% fee.   Also, if you’re a pro-trader on a DSP, and you’re not reading this article and thinking about starting a curation business, I don’t know what you’re doing.  Because it seems to me that you could be making 20-30% of the budgets you manage :D.

Now this could be BS, and my goodness, Right Media is 20 years old.  I can only imagine that data science has come a long way since then.

Other explanations also exist. For example, many DSPs are relatively religious about not pooling client performance data — Pepsi would probably be pretty unhappy with you if you looked at their top performing campaign and then shot a note to Coke “Hey guys, check this out, it works AWESOME!” Curation, on the other hand, revels in the performance data from one agency or brand enhancing performance for another, and operates nice and neutrally as a vendor on the “sell-side.”

But the point remains – people are doing this and it’s making campaigns work better.  I was wrong because I painted curation as a data play, it’s actually much less sophisticated than that.  It’s an affiliate sales play.  And algorithms worth hundreds of millions of dollars are getting upstaged by sales/adops people.

This all being said, DSPs could of course build their own version of curation – allow members within your platforms to sell targeting and optimization templates to one another for a % of spend going to the person selling the template.  I bet it would work even better!  I also bet all of the current curation companies would get in on the game.

But like, isn’t this kind of the job of a DSP?  To help individual DSP users run campaigns super well?  Hopefully curation just ushers in a new era of focus on outcome-based optimization, but people need to come to terms with the value it’s actually providing first.

If we engage in deep thinking about the various forces within the ad tech value chain, we can uncover how everyone acting in good faith can result in a remarkably inefficient ecosystem.  It is only with this understanding that we can transform it.

Please note, as with most of my writing, my intention is to bring the ideal independent digital marketing world into being, not reflecting on the “reality” of the world today.  I also have no horses in this race, and enjoy partnerships with all of the entities described here.  This is not meant to castigate anyone – merely to explain why the world as it is probably isn’t going anywhere soon unless there’s a meaningful disruptive force.  I also believe that independent marketing infrastructure is the lifeblood of the open web, and to the extent that we care about the decentralized distribution of information, it is our duty to improve these technologies to the best of our ability.  On to the discussion.

The villain of today’s article is something we all know well.  It’s been around as long as I’ve been in ad tech, and certainly before, and it’s not going anywhere any time soon.  The villain in ad tech is the Revenue Share.

Loss Aversion

For the uninitiated, a revenue share refers to paying a vendor a % of revenue as opposed to a flat fee – the attractive part of revenue shares being that they’re directly proportional to the amount of money spent on or paid for media, with no “guaranteed” fees..  The reason revenue shares work is because of a game theory concept called Loss Aversion (read more here https://en.wikipedia.org/wiki/Loss_aversion), the TLDR of loss aversion is that people hate paying and losing money, especially publishers in my experience, and they hate it so much that they will sacrifice potential expected value (ie. make less money over all) as long as they can avoid something being framed as a loss.  An example, which all media people will identify with – which do you think your average publisher would prefer : 

• Publisher pays vendor $3 guaranteed, has a 70% chance to make $10 (publisher is net +7)

• Publisher pays vendor $0 guaranteed, has a 70% chance to make $10, but has to pay $8 of the $10 (publisher is net +2)

Publishers will pick the second option almost every time.  This is because the 30% chance of losing $3 is untenable to them; the notion of a guaranteed cost without guaranteed revenue is unacceptable.  This, even though the expected value of the first option is meaningfully higher!

And the rabbit hole goes further!  Not only does loss aversion cause non-optimal decision making, which you could also say is just an outsourcing of the risk associated with technology costs, but because of the nuances of ad tech, it actually creates the illusion of fully aligned incentives that are not quite there.  In many scenarios, it would make sense that a revenue should align two parties who both want to maximize the total value of a thing – a % of revenue means you want the revenue number to be as high as possible.  However, in the world of unified auctions, it does not totally mean that. This is kind of a complicated theory, so buckle up, but it is actually the case.

Demand Inelasticity

The thing that breaks the incentive structure of revenue shares is bid jamming (article here https://www.garethhatesadtech.com/p/bid-jammin).  The net effect of bid jamming is demand inelasticity wherein the bid variability for differential supply paths within a single auction is so high that the additional share of voice generated by a reduction in vendor take rate (read: taking a lower revenue share) is actually negligible.  I know because I tried this.  I built an SSP, got bids from the trade desk, and took zero vig, and guess what?  It didn’t win any more than any other supply path!  

When it comes to incentives, SSPs/Exchanges now have a strange decision to make.  I, as an SSP, do not increase my share of voice (or increase it minimally) by reducing my fees.  Therefore, I can assume that my a priori share of voice is actually somewhat static – so I am now incentivized, in order to maximize my margin, to either increase my SOV in other ways (hello ID bridging and curation), or to maximize my margin, because me taking higher fees won’t decrease my revenue, just that silly publisher’s.  Additionally, for ID bridging and for Curation, I only care that they increase my share of voice with their differentiated bids – if I can, I want to scoop off as much margin as possible and not pass additional cash to the publisher.  Because of this demand inelasticity, my optimization is not to maximize the size of my payout to publishers, my optimization is to maximize my SOV and my margin.  Increase Publisher Revenue? Yes.  Maximize? No.

The Aggregate Effect is not Intuitive

Let’s illustrate this. For example, each new ad tech entrant adds 15% to the total CPM in the transaction, through some sort of value add – be it increased outcome rates, some kind of auction manipulation, whatever. Without demand inelasticity, aka in a perfect market, if they pass 100% of the 15% through to the publisher, they would maximize their share of voice, and for every reduction to that 15% they extracted in fees, their share of voice would decrease proportionally because the bid would decrease.  However, because demand is inelastic, this is not the case.  The additional fees subtracted from the bids generally do not lead to a reduction in share of voice, or at least not cleanly or linearly. So what’s the net result?  The vendor is incentivized to take as high of a fee as possible while still maintaining their edge in the auction – which is often super, super, high.  We’re talking 15% of value and a 14% fee – but still transact.

This math, I believe, is what has sent our industry awry.   Every vendor does this math, and if they’re truly adding 15% of value, they can charge the 14%...and publishers are still happy with their 1% lift because they’ve taken no risk!   There’s no real incentive to upset the apple cart here either – you’d think vendor competition would have them squeezing one another, but with everyone adding different kinds of value (and weird price inelasticity in a given auction due to throttling and whatnot), it’s a phenomenon that hasn’t come into vogue.  Even with SSPs, the biggest commodity players in the industry, I’ve observed that meaningful reductions in take rate don’t lead to proportional increases in share of voice.

Publishers don’t care because they live in a world of header bidding – even if the SSPs and Curators generated $10 of surplus value, if they take $9 of it and pay the publisher $1, the publisher is still coming out ahead.  

And this, my friends, has been happening over, and over, and over with tech layer and tech vendor after tech layer and tech vendor.  The revenue share, and specifically the sell-side revenue share, has created an environment where the lion’s share of the surplus value created by independent ad tech vendors has gone to those ad tech vendors.  But, I would argue, the vendors are not to blame!  This is the expressed preference of publishers, who are allergic to costs and loss averse. Things are especially muddy in the exchange space, due to price inelasticity and entrenched relationships between DSPs and SSPs, and that’s why we do see innovation in the header bidding management layer (RTK, my company, did not charge revenue shares for our software.  We were the only ones in the space not to do so at the time, this was in 2015.  We included an ad exchange in it, but even with zero revenue share, my friends at TTD meaningfully throttled down our supply path, though it was probably our fault – RIP).  This is also why I don’t like Curation.  It is inarguable that Curation helps publishers, just like it is inarguable that SSPs help publishers, but the problem I have with it is that it’s just another rev share – publishers have become vessels for new technology to ride along, but get paid a measly wage for their services.

Why should we care?

We should care because our competition here, X, Google, Facebook, etc, does not have this value siphon draining dollars from their ecosystem.  Every incremental % of value that they create is funneled into making their technologies more effective or their content better.  And until we figure this out, and fix the massive revenue leaks in our industry, we will lose to them more and more.  AppLovin, in an ecosystem that grew up and came into being after the DSP and SSP existed, does not operate and make most of their money in RTB.  It is important to understand why.

What can we do?

Apart from publishers changing their behavior, which I will not hold my breath for, I think the onus is on DSPs to really dig into bid jamming and select single supply paths to publishers.  I don’t mean SPO to 3 or 4, I mean SPO to one.  If the demand inelasticity goes away a lot of these problems fix themselves and people actually have to compete on take rate.  I won’t hold my breath for this either, but it would go a long way.

There are other solutions, but I’m going to use them in my products, SO I’m not going to spill all of my secrets to you here :).  

I have been given a gift, not-so-rarely afforded to those in my position, wherein Ad Tech has created another thing that I think is dumb, but all of you seem to be buying.  That thing is “curation.”

In light of that, a disclaimer – I may at some point sell some of you some kind of product that does this.  I will not be happy that I am selling it.  I will take no joy in it.  But damnit, if you want to pay for something that does this, do not consider this article a statement that I will never build it, because I might.

With no further ado, let us explore Curation in the depths of its silliness.

Subscribe now

What Is Curation

In simple terms, Curation is a kind-of new form of “bundling.”  Bundling means grouping the inventory of multiple publishers together using some kind of logic, and then going out and pitching agencies/advertisers directly on your shiny new “bundle.”  The defining logic of the bundle could be anything – certain pieces of inventory, data, sitelists, whatever.  You name it, someone is bundling it, and someone is pitching it.

Historically, bundling was done as a standalone platform business model.  This is why people refer to curation as the new form of Ad Network – because this was literally the pitch of ad networks for a decade.  Have a contextual data set?  Great, let’s sign up some publishers, layer the contextual data set on top of it, and close some direct advertiser deals.  How about cookie data about users?  Awesome, let’s start TACODA, the 2005 Behavioral Ad Network.  Unique (or not-so-unique) data has been and continues to be the lynchpin of countless ad network businesses.

But now it’s 2024, and appropriately, bundling in its modern form uses the favorite mutant offspring of the Ad Tech middle man – the Deal ID.  Deal IDs are the foundational technology of PMPs (Private Marketplaces), which have been “Curating” for years.  Deal IDs are logic-based labels that are dynamically applied to bid requests by an ad exchange.  In direct opposition to bundling of old, where you had a separate technological entity, Deal IDs allow you to bundle within a pre-existing piece of tech that already has inventory integrated with it – the ad exchange.  We’ve made a frictionless transition to this for reasons I will describe later.

Deal IDs are a strange creature.  They were originally invented for publishers to do their own bundling, and I would argue that this was in a sense “okay,” while still not being optimal.  Their initial use was meant to be a semi-direct sold methodology, something in between programmatic guaranteed and open auction. But in truth,  the notion of a publisher needing to label something that was typically already readily available for targeting to DSPs always rubbed me the wrong way, often tacking on some pulled-out-of-a-hat pricing to sweeten the inefficiency deal.  Don’t get me wrong, I’m sure there were some people who used them properly and had a good reason for doing so, but over time they became progressively more and more meaningless.

I think the reason that Deal IDs became so fraught is because they’re by design adding an inefficient component to programmatic auctions. This is because a Deal ID system is actually a targeting system – wherein the system user sets up a list of things they’d like to apply this Deal ID to, and then the Deal ID system applies it when appropriate.  In the case of modern curation, the targeting system is used to integrate the dataset, whatever it may be, with the bid requests generated by the ad exchange.  So the flow is as follows : 

1. Publisher loads website.  Prebid runs, sends requests to ad exchanges.  Maybe publisher or a publisher vendor passes in some KVPs for targeting, maybe they don’t.

2. Requests land in the ad exchange.  Ad exchange executes a lookup, where they have a BUNCH of Deal ID rules set up each with its own set of qualifying targeting (inventory, geo, data provider, blah blah)

3. Qualifying requests get the appropriate Deal ID appended to them.

4. Requests go out to DSPs, with a subset of requests having the relevant Deal ID(s) appended, to be labeled as “PMPs”

The problem I have with targeting in this context is that it’s way too close to the concept of a “campaign” in a DSP for me to believe it’s efficient.  In fact, the universal Deal ID troubleshooting email consists of the following “HEY DO YOU HAVE TARGETING TURNED ON IN ADDITION TO THE DEAL ID BECAUSE ITS NOT DELIVERING AND THAT’S PROBABLY  WHY.”  A DSP campaign contains a list of rules of things for the campaign to target, meanwhile, the deal ID system is executing a…list of rules of things for the deal ids to target.

Subscribe now

When things are being done twice in the ad tech supply chain  it’s normally the sign of a problem.  And not only are Deal IDs watered-down forms of DSP targeting, in their modern form, they’re often hurting more than they’re helping.  This is because Deal IDs are arbitrary, middleware-introduced signals that could mean absolutely anything, including nothing at all.  It could be “homepage ATF on News Sites” or it could be “users who like potatoes.”  It could be a “whitelist” of literally every domain available in that exchange, packaged under a Deal ID because you’ve probably turned off your targeting for Deal IDs and this is a way for an ad exchange to slip mr. advertiser impressions that they don’t want.  And this brings us to the krux (I spelled this like this in my first draft and I refuse to fix it) of the issue – we are using Deal IDs badly, and I’m not certain there’s a way to put the toothpaste back in the tube.

And it gets worse.  Their original invention, while introducing targeting inefficiency, at the very least meant that Publishers or exchanges were using a shitty way to communicate optimization information to the buy side while keeping 100% of the fruits of their inefficient method.  But modern “curation” has added a new component – the Deal ID middle man.  

There are a few flavors of these middle men : 

1. Inventory Curation – “we’re going to put websites and ad units into a Deal ID so that you don’t have to build your own whitelists of these things”

2. Data Based Curation – “we have contextual or behavioral data we’re using to govern when this Deal ID applies”

Where this gets spicy in the modern era is the sneaky-and-gross-but-totally-clever introduction of the third party curation provider.  You see, dear reader, exchanges have been scared of becoming pure proxy servers for some time (or, disturbingly aware of the fact that that’s what they are).  Given this unpleasant truth, they quite cleverly had the idea “what if we allow other companies to enrich the bid requests we’re sending out, without having to implement code anywhere else, and then they get a vig on the PMP.”  In other words, exchanges had the realization they can become an integration point for data, and subtract a revenue share en-route that they then pay to those third parties, while also outsourcing the sales against that data.

This is great for exchanges because they make a vig on their PMPs and have someone else doing sales for them.

This is great for the “curators” because they make a vig on their PMPs, and they don’t have to provide any technology other than “when” to apply the PMP – they can be basically pure sales companies.  Plus they can often activate without additional technology on publishers already integrated with the exchanges.

This sucks because the signals going to DSPs aren’t what they could be, and we’ve managed to introduce yet another middle man into the ad tech supply chain.

Share

Per my many articles on trying to fix ad tech, there are certain places where certain activities should occur in the programmatic supply chain.  Targeting and optimization are the purview of the DSP.  This is a good thing.  We shouldn’t all be trying to build targeting systems and optimization algorithms, let some people specialize and get super good at it. Ultimately, I don’t like Curation, or Deal Ids, because I think bundling is bad.

Bundling is bad because the application of a given bundle may contain many different components, some of which may or may not be transparent to the DSP (or, frankly, might be totally value-less and already available without the bundle).  This results in signal reduction and signal muddying – where the primary differentiated signal the DSP has is the deal ID, when in reality, the deal ID itself might be composed of signals unavailable to the DSP.

By reducing and muddying signals sent to DSPs, we’re making campaigns perform worse. By making campaigns perform worse, we’re making ad tech worse.  Now, the counterargument to this is that some signal is better than no signal – and while that may be true, the argument of “well this makes more money than the alternative so we’re going to let it happen” is the justification for most of the bad things in ad tech.  And because so many people aren’t actually optimizing their performance properly, bundling becomes another way to make middle men rich and hoover more money out of working media.

So what’s the solution here?  My solution is that any signal that would be used to append a Deal ID should be communicated in its raw form to the DSP, and in doing so, the DSP will more effectively optimize the campaign to the signals that lead to outcomes.

Curation Shouldn’t Exist, DSPs should just be better listeners

I’ve done a lot of complaining about exchanges and middle men in this article, but at the end of the day, I believe the party most responsible for the emergence of curation middle men is the DSP.  This is because I believe the right way to do this would be for Publishers to append whatever first (or third) party data they had to their bid requests in its raw form, instead of using a grouped Deal ID proxy, and DSPs should consume that data for optimization.

This would involve some OpenRTB cleanup – you might have a dozen different fields from a dozen different data providers flowing into the DSP, but in the age of artificial intelligence, more data is a good thing and should improve the performance of biddable media.  DSPs should always want as much data as possible to back-correlate to campaign outcomes, and if they don’t, then there should be new DSPs who fire up who want to listen to that data and use it to back test versions of their optimization algorithms.    Allow me to illustrate this future with an example : 

A curation provider provides “MFA-free” deal IDs.

Subscribe now

Current Curation Implementation

Curation Provider crawls websites, has a list of websites it thinks are MFA as a result of its 10-point assessment system (i’m being generous here).

Bid request comes in, curation provider reads page URL, replies with a binary “Deal ID or No Deal ID”

All bid requests have normal information associated with them + non MFA requests get DealID:NOMFATKS appended to them

Agency buys Deal.  Agency Logic consists of “well, NOMFATKS performs better than Open Exchange Buying, so I guess it’s worth the 15%” (once again, being generous here, it could be that it performs worse for their campaigns but they just really don’t want that MFA)

Future Perfect State Implementation

All bid requests have normal information + 

fpd.InventoryData:

AdDensity:1234

AdCount:6

InView:Yes

AttentionScore:75

Basically, we pass each of the components that the Curator used to make the determination that it was MFA into the bid request, and we optimize to those.  This means we do surgery instead of chopping with an axe.  Theoretically, this could get messy if a publisher is working with a bunch of different data providers, but I would argue this is no different than working with 20 different userID providers, which people do today and nobody seems that upset about.

Now the DSP correlates campaign performance to the components that comprise MFA.  Maybe some impressions that would have been purchased before for being “MFA”, now get purchased because of nuance in their scores, or some impressions from “non MFA” sites that would’ve been purchased, don’t get purchased because they have a component score that doesn’t correlate to outcomes.

The tricky part of this implementation is billing – Deal IDs are easy because middle men just get a vig (once again, the plague of all ad tech).  This means that to do this the right way we need some prebid work.

1. Each additional Data Provider needs to be A/B tested on a subsegment of traffic, and performance needs to be measured.  Test needs to happen for a long enough time for DSPs to adapt to the signal.

2. Data Providers can then bill on a usage fee, or on a % of lift, whatever they prefer.

I don’t just think this reform is important, I think it is foundational. Deal IDs are a core thing that holds back open web buying from performing as well as it could.  My implementation would mean that we harness the power of open source to build optimization data that isn’t just “as good” as walled garden data, it’s better because of how many different data providers can compete on the lift they can create for advertisers.  In this architecture, which is death of 3pc proof BTW, Open Web buying could enter a new era of increasing performance, and DSPs truly compete on their ability to find signal in noise to drive outcomes for their clients.

So yea, I think this would be awesome.  Oh, and I’ll let you guys know on Linkedin when/if I have any curation products available.  It’ll probably be soon.

Subscribe now

Welcome to a flash-installment of GHAT, which overrode my scheduled programming (me complaining about Curation, which you will all get in a week or two), as I think this is a time-sensitive article that everyone should read.

I want to talk about what the world would be like without GAM.  There’s a non-zero chance that GAM and ADX will be separated by uncle sam, and I believe that this could be the beginning of the end of GAM.

My reasoning for this is that 95% of the people using GAM don’t actually need it.  And the remaining 5%, Prebid could build a competitor that displaces GAM in a matter of months, with the actual migration process taking 3-5 years.

This is not to say that Prebid could function as an ad server right now, per Tom Kershaw’s testimony for the DOJ it is missing a bajillion core features.  But those features are missing not because they’re impossible or even difficult to build in a superior way to GAM, rather because nobody at Prebid wanted to waste their time building something that very few people would use.  Hell, we BUILT one of the core ones, even though I was bitching the whole time that no one would use it.

So, without further ado, the world without GAM.

First, The History

The concept of the publisher ad server is ancient.  In a training I gave all of my RTK employees, and maybe even had a training video or two on back at AppNexus, I discussed the emergence of the ad server.  In a sentence, the ad server is an abstraction layer that serves to swap out, target, and count the ads on publisher websites without an engineer having to change the ads on that website manually.

This needed to exist in 2005 because many things were direct sold (ie. the publisher made a deal directly with an advertiser), and things that weren’t direct sold went out to Ad Networks that weren’t interconnected.  The Ad Server lived, thrived, and was necessary in a world where the systems didn’t actually communicate with one another.  Rather, you ping ponged advertising impressions from system to system using javascript tags resulting in client side redirects, and every advertising system had its own internal economics that weren’t exposed to other systems.

This Began To Change With RTB

The first RTB systems, and I’ll just go with Right Media for this, changed the game by allowing networks to expose their economics to one another – in a simple sentence, instead of ping ponging advertising opportunities between one another with no communication about price, they could bid into one another’s advertising opportunities. 

This concept, of one advertising ecosystem (ie. a network) bidding into another advertising ecosystem is the foundation of modern ad tech.  

The trajectory looks like this : 

Phase 1 - Early RTB (RMX, AdECN)

A single platform that houses multiple ad network entities.  These ad network entities leverage the platform itself to bid into one another.

Phase 2 - Inter-Platform Bidding / OpenRTB Emergence

Instead of the various participants in the advertising auctions being housed within one platform, we develop technology for platforms to communicate with one another.  This is when the concept of the DSP and the SSP emerged, and OpenRTB attempted to standardize the communication between those companies.  Ad Exchanges are born here.

Phase 3 - Header Bidding

Header Bidding applies the same bidding theory that existed for Ad Networks (now called DSPs and SSPs) to a level of abstraction lower, within the publisher monetization stack, previously the exclusive purview of the ad server.  Various Ad Networks now participate in a meta-auction at the publisher ad opportunity level, where the bids they get from their technology platform partners are passed along into an ultimate publisher level auction conducted by a javascript client side header bidding framework.  Currently, but not necessarily, this then gets pushed into GAM.

The difference between Phase 3, where every system is capable of communicating live bids to one another for every single ad opportunity, and Phase 0 where the ad server was born, is tremendous.  

In a world where every system is capable of bidding, Publishers without direct sales truly don’t need an ad server.  This is 95% (or maybe 99%?) of publishers.   And to be clear, this means that 99% of publishers who are using GAM are only using it because adx is the only participant in this supply chain that refuses to bid into other technology platforms.

So, in light of this, I’d like to describe Phase 4 - A world without GAM

In a world without GAM, we centralize all direct sold and programmatic inventory into a unified, single on page auction.  Let’s make this auction perfect – here we go –

Step 1 – Inventory Definition

I’ve written about inventory definition in other articles, it’s how a publisher labels the inventory they have available.  Publishers shouldn’t be responsible for this.  Publishers should put ads where they want them to go, and then an automated system like Adelaide or Contextful or some other startup that I’m an advisor to (feel free to hit my dms) dynamically labels the inventory in a way that gives it stable, predictive performance value for advertisers.

Step 2 - Publisher First Party Data Addition

Publishers use first party data already for all kinds of stuff – typically communicated into GAM via key-value pairs.  In Step 2, the publisher appends this first party data to the various ad holes on their website, and also uses the Prebid Real Time Data Module to append any other data from its vendors to the upcoming advertising auctions.  This is standardized, and all enriching data for this ad advertising opportunity is appended here in a single step.  Publishers can exert control over who sees what data, or they can blast everything to everybody as they see fit. This almost entirely exists today in Prebid.

The reason things aren’t all consolidated here is primarily because OpenRTB also has to get its shit together regarding taxonomies for this stuff, and DSPs need to start ingesting and optimizing to the data fields more flexibly and stop driving people to use PMPs for dumb shit (article to follow next week).

Step 3 - Live Auction For Inventory

Requests are sent out for bids.  In this Platonic future, the publisher has two sets of bidders.  

1. Direct Sold – they’ve configured a bidding platform with their direct sold campaigns, they can use a DSP for this.  Using a DSP actually has meaningful benefits, as their campaigns will be able to simultaneously target O&O inventory and do audience extension in a unified way (appnexus can already do this, and has been able to for over a decade).  DSPs are also fantastic at outcome optimization.  GAM is terrible at or incapable of both of these things.  

DSPs already have access control management, the concept of teams, and the ability to create tremendous hierarchy in their uses that is almost always superior to GAM’s ridiculous Teams feature.

We would need to introduce the concept of priority level (ie. guaranteed, p1-1000, etc), but that’s not super complicated and with the right incentives DSPs will build it quickly under the tutelage of a knowledgeable sell sider.

2. Programmatically Sold – all programmatic bidders compete in a unified auction.

In addition to the DSP housing the managed campaigns, the publisher has all of its other bidders.  Ad Exchanges, DSPs who are now going Prebid Direct, you name it, everyone can participate in a single unified auction with modular implementation.  Prebid Server becomes a big deal, because it provides basically infinite biddable scalability with zero latency impact.

Step 4 - Delivery

Prebid now renders the ad to the page directly.  This saves us around 800ms of GAM doing bullshit, making ads across the internet perform better, and also exposes the auction dynamics via Open Source, so it literally becomes impossible for anyone to engage in shenanigans.

Fin

This is Ad Tech 3.0.  Let’s Make It Happen.

Feature disclaimers

Forecasting, or a tool to help publishers plan their direct sales, is something easily built as a plug-in to Prebid.  In fact, forecasting coupled to an ad server sucks, if it were in Prebid you could have multiple forecasting vendors compete with one another and pick the one you liked best.  It would be epic.

Ad Quality Controls for Publishers  – easily accomplished by prebid, in fact I’m pretty sure Prebid is carrying this football forward and its always been one of my pet projects.  This would objectively work better in Prebid than it does today.

Anything else an ad server can do Prebid could do better.

Secondary Benefits of Ad Tech 3.0

The Industry Grows Like a Mofo – using Prebid Server, anyone who wants to start a DSP can buy directly from any publisher using Prebid in a biddable way with trivial integration costs (zero latency cost, the cost of additional fanout in server, and prebid server could really easily add a throttling methodology for testing new bidders to control costs).  We have tremendous new competition in the DSP space, because Ad Exchanges go away.  Fewer Middle Men.  This has a bajillion benefits.

Publishers Make More Money – by reducing auction latency, and making it much easier for bidders to integrate directly with publishers, and by simplifying the competitive workflows, Publishers undoubtedly make more money in this scenario.  They have more competition for their ads, they have more transparency, and they own their future.

Advertisers Get Better Results – better data, communicated in a standard way, with better transparency on the auction methodology.  Campaigns work better, more people spend money on the open web.

This would possibly save internet publishing.  It would certainly make everyone more money.  And there’s only one thing stopping this optimal outcome from happening –

ADX DOESNT BID INTO PREBID.

Thanks for coming to my Ted Talk.  Happy Antitrust. Also I won’t hold my breath

Today’s dead horse that I’m going to reanimate solely for the purpose of sledgehammering once again is MFA.  I know, I know, another article about MFA?  Please will people shut up about MFA?  

No!

And I won’t because in my semi-humble opinion, most of the things I’ve read about MFA have completely missed the point.  And just so we get it out of the way, if you’re reading this and you’re an MFA publisher, this article isn’t for you nor is it meant to be a defense of MFA.  So don’t clog up the comments!  

I think the right way, and frankly the only way, to discuss MFA is incentives.

It is my supposition that MFA is not a surprise, is not some “damnable” thing, and is in fact a predictable product of the systems that we’ve created.  Allow me to explain.

Cue the Time Machine

I’d like to begin by focusing on something a little different – something I would call the “non programmatic internet.”  These are websites whose monetization methodologies do not utilize programmatic advertising – rather, from their inception, they make money from other means.

This is readily apparent in some of the largest websites out there that conspicuously lack banner ads.  Their business models are simple  : 

1. Content that gets indexed effectively by SEO

2. Layouts, and content, that drive CPA performance marketing (known as commerce, or affiliate)

By performance here, I mean primarily post-click performance advertising – where the advertisers only pay when someone clicks and then does a thing.   This is because the early ad networks were really built around this, including Google Adsense which still pays out on clicks today, ostensibly because historically those clicks would either lead to outcomes or were the desirable outcome themself.

One of my favorite examples of this type of website is the wildly successful Credit Karma (one I’m intimately familiar with, because within the graveyard of Gareth’s failed tech startups lies credit-authority.com, Credit Karma before Credit Karma was big).

Credit Karma, as a website, is a brilliant example of performance marketing and innovation in upselling products after reviews.  They originally provided reviews of credit cards, those reviews driving users to their site primarily through SEO (maybe they did some paid?  I would love to know if they did…), and if you look at their homepage you can see it’s solely dedicated to driving users to products and to reviews of products (the main button is their in house product, a paid credit monitoring service, and then the little boxes below take you to their reviews of other products) : 

https://www.creditkarma.com/

A nice example of one of these reviews, meant to be discovered via SEO, is one of these, a description of Balance Transfer credit cards 

https://www.creditkarma.com/credit-cards/balance-transfer

In fact, if I type balance transfer credit card into Google, low and behold our good buddies at Credit Karma are indexing really well in the SERP (Search Engine Results Page) 

But the long and short of it is you know what you don’t see on this website?  Banner ads!  Not a single one.  There’s no programmatic here – this site monetizes almost entirely through conversions on their own products or through post-click attributed applications for other products (starting with credit cards, but expanding into other financial verticals).  This is the internet built in the image of affiliate performance – and you can see this methodology implemented in tons of different web layouts.  The page is built for a single purpose – to create clicks on that little continue button to get people to apply for credit cards.

The insight I want to drive home here is that the monetization methodology informs the structure of the website.  A user clicking on information, and filling out an application, is the payable event.  This means this website has evolved in such a way that it maximizes those payable events.

This brings me to MFA.  What is the primary desirable payable event in programmatic for many advertisers?  Now that we’ve moved away from raw impressions or CTR, it’s viewable impressions.

And that’s the outcome that many brands are optimizing towards.  “Here’s an audience, make sure they see my ads per this little javascript measurement I execute” So what do you end up with?

Site’s whose primary existence is to create as many viewable impressions as possible as quickly as possible!  This is not surprising at all.  In fact, it’s stunning that it’s a revelation for anyone.

Note, the optimization goal of “viewable impressions” is not : 

• Viewable impressions with not very many other things on the page (like other ads)

• Viewable impressions on highfalutin content that we think is intellectually stimulating

If it were those things, we would’ve seen websites writing more high-falutin content and ad impressions without lots of other stuff on the page.   The failure of MFA is not MFA websites.  The failure of MFA is that we built an incentive system in programmatic that essentially necessitated their existence.  And we’re now demonizing websites who were just giving us more of what we wanted and patting ourselves on the back.  Shame on us.

I would say that MFA exposed two negative things while also signaling at least two positive things  – 

As for the negative things : 

1. The outcomes being defined by advertisers were not actually producing results for them (by many accounts, MFA sites looked like they performed great)

2. A bunch of people were not reviewing their sitelists as precisely and often as they could’ve been

Outcomes

It is my firmly held belief that badly structured outcome goals are to blame for programmatic being subject to so much bullshit.  When outcomes are poorly structured, and are not tied to explicit measurements of performance in some way, the industry naturally becomes a hornet’s nest of misaligned incentives and tomfoolery. 

My reasoning here is simple – if the structure of advertiser outcomes doesn’t actually provide growth for a given advertiser, budgets can never increase to that channel outside of essentially arbitrary manipulation.  It should simply never, ever, ever, be a bad thing to get more of your outcomes.

At worst, this is killing programmatic.  It means programmatic is waging a war of reputation, not a war of performance, and after scandals like MFA how could a responsible CMO bet on the open web?  If all they’re looking for is safety, and they can manipulate the data to make whatever channel look like it’s hitting their targets, they’re almost always going to pick the source that appeals to brand recognition – and when Google, Facebook, and big reliable direct sold names exist, open web programmatic is in trouble.  I think that TTD does a great job of branding itself here – but ultimately you are what you eat, and TTD eats the open internet (this is why they have to do all this press about a sitelist – when performance is a commodity, and “branding” is the goal, the number one thing that hurts your campaigns is bad reputational press).  And, by the way, thank goodness that TTD is there to do this – without them, programmatic would be in serious, serious trouble.

At best, these misaligned incentives create wealth for middle men and people who are good at selling the technology that establishes these incentives.   It is no coincidence, by the way, that the same companies that sell much of the measurement technology responsible for advertisers caring about these outcomes are also the companies selling additional products to cure the mutant offspring of their outcome products.  These non-outcome-producing-outcomes naturally lead to a fixed-pie ecosystem vying for branding budgets allocated not on substance, but on internal marketability.  Because there’s no demonstrable ROAS, there’s no scalability.  I believe that unless we can make programmatic demonstrably more marketable as a portrayer of value when compared to other channels, site owners are going to be stuck fighting like rats over cheese on a sinking ship. 

And an important point here for all of you – if you were optimizing to an outcome before the MFA scandal, and then implemented a blocklist, but you’re still optimizing to the same outcome, I don’t think your campaigns are driving any more value for you than they were before.  Because how could they be – if your outcome was broken on MFA sites, it’s broken on other sites too.  If you’re now implementing a secondary measure to assess the performance of your blocklists…why wasn’t that your optimized outcome in the first place?  If it had been, your DSP would’ve been optimizing to it, and your spend wouldn’t have been allocated to sites that for all intents and purposes were performant in the eyes of the DSP.

Asleep at the wheel - Programmatic’s Albatross

MFA is not a transparency problem.  There is no part of the MFA scandal that was not painfully reportable to anyone with a modicum of experience running a detailed report on the delivery of their advertising campaigns.  And if a DSP withheld this information, that’s profoundly problematic, and really like, not cool dude.  But even if they did, your buy-side ad server should be able to report on delivery, or any of your verification vendors that you’re paying through the nose for, or any of a number of technologies.  I really think the DSPs are the last people in the world to blame for this unpleasant surprise.  

Given this, there are a few foundational pieces of trust that need to exist, per my last article about what to give a shit about : 

1. Advertisers are buying what they bid on, as described

2. Advertisers have the ability to verify after the fact what they bought

Technologically, this is utterly trivial.  It seems to me that there are some business practices at play here, and that have been at play for a long time, that are keeping this from happening – namely, that people aren’t actually monitoring where their campaigns are running and SSPs are sending garbage openRTB data.  And time is not an excuse here – the cost of overseas employees to review campaign delivery is astoundingly low.  And these teams are tremendously capable with regards to campaign management as well – if anyone wants introductions to a few solid firms, I’m happy to provide them.  

But accountability needs to exist, and if you’re a brand or an agency, you need to know exactly where your money is being spent (or at least, what pages are ostensibly driving performance for your campaigns).  If you run somewhere for a long time, and it looks like it’s driving performance for you, the responsibility is yours and yours alone.  TTD is taking a major step in basically reviewing your sitelists for you with their Programmatic top 500 – but once again, they’re doing this to basically do traders’ and agencies’ jobs for them, so now everyone has a “spend more money than you have to, but don’t give me any surprises” button inside of their DSP.

Now, for the positive side of MFA-

I know, how could MFA possibly have a positive side?!  It does, I assure you, it truly does, in fact it probably has more than one.  Here are two I think worth mentioning : 

1. MFA reveals that programmatic budgets are successfully inspiring web evolution, and publishing businesses are capable of evolving to optimize to programmatic

2. There exist economics to funnel traffic out of walled gardens onto the open web that can possibly work

Evolving Internet

Our first example in this article, cited Credit Karma (and the millions of websites like it) of an economic system inspiring the construction of websites.  

MFA is an example of the programmatic ecosystem inspiring precisely the same thing, just for viewability.  The reason this is a good thing is because if we fix our outcomes, it means there are companies out there that will reimagine their businesses to optimize to those outcomes.  That is, as long as we don’t go totally Luddite and move entirely to allow lists.  Rather we need to let the invisible hand of the market inspire new business models as programmatic properly evolves to meaningful outcomes.  This is actually epic.  We just need to optimize campaigns to outcomes that we want, and websites will come into being that drive those outcomes, and the programmatic might see some market share growth.

There Exists a non-SEO Publishing Model

MFA sites do not get organic traffic.  But the largest MFA sites were as big or bigger than the largest content publishers on the internet.  That’s an astounding feat.

While bad for just about everyone in the programmatic supply chain, this should be incredibly inspiring for the publishing industry.  This means that there exists a business model wherein content can be promoted successfully with advertising, siphoning very expensive audiences out of places like Facebook or Google Search.

MFA made this work by stuffing tons and tons of ads on the page, which we all know is no bueno.  However, if there exists a way to achieve ROI on users without insane ad to content ratios it means that publishers can effectively convert audiences from walled gardens into consumers of their content by paying for it instead of praying for it.  

This business model is one of the reasons we started Gamera – the way to do this is to measure the lifetime value of users, not single sessions, and to build systems that re-engage users effectively (of course, in combination with providing them with content that they actually want to read) to achieve positive ROI and business growth.  I am firmly convinced that this is the key to transforming online publishing from a dying industry into a growing one.  If you’re a publisher and you’d like to learn more about this, I’m happy to talk to you about it.

Conclusion

So there it is.  And now I get to use the phrase that I’ve been sitting on for months and months  – If you have an MFA problem, MFA isn’t your problem.  MFA could end up being a good thing for programmatic, as long as it spurs reform and innovation in advertiser outcomes and publishing business models.  And please note, this is not meant to be a defense of MFA – there are definitely sketchy business practices there, not in the least bit indicated by it being difficult to navigate to the experiences being sold programmatically.

But if the MFA scandal doesn’t inspire reform and innovation, and it just creates an excuse to slap another vendor layer onto poorly defined advertiser outcomes, it will make our future more dire than it was before.  This would be a regression, not a progression, of programmatic.  And the dire future we would be barrelling towards would not be because we served our brands to some shitty looking websites.

I feel this is necessary because so much in ad tech is sold on FUD (fear, uncertainty, and doubt), when in reality, ad tech is astoundingly simple in abstraction and almost every problem can be deconstructed into my magical overarching theory of everything (you’re welcome).

The foundation of my framework is that Ad Tech is a numbers game.  It’s a quantification, and through quantification it removes human biases and creates efficiency.  And if we want to get really deep, once you include attribution, Ad Tech is a tangible, scaled, measured, quantification of human response to stimulus, the likes of which has really never existed before.

What do I mean by quantification?

The fundamental building block of ad tech is the bid request.  It’s a packaged up (using some kind of object) bundle of information about an opportunity to advertise.  In drastically simplified example form, this looks like this : 

domain : website.com

userID : 1230912301923901

Useragent (browsing environment) :  12309sasciawj:1230812

1230912301 other fields like these, with ever growing complexity

A bidder receives _billions_ of these little packets per day and has to make decisions about which ones to buy and not buy.  How does it do that ?

Simple!

Machine learning and statistics of all sorts.  It looks at the correlation between the fields in these little bundles of information and advertiser outcomes – which are defined by the campaign configuration in the platform.  An outcome, in this context, refers to a signal communicated from the advertiser/agency/placer-of-ads to the platform making inventory evaluations that something desirable occurred and it should try to make more of them happen.

What does this process actually look like?  There are lots of flavors of this – but at its core, every campaign goes through a “learning” process by which it correlates outcomes (everything from clicks to purchases depending on how you build the campaign) to the numbers it receives in the fields in the bid request bundles.  

Some people try to “bootstrap” this process, or make it happen faster using a bunch of methods you’ve almost certainly had pitched to you by a vendor.  It could be through targeting, either behavioral/user-based or contextual, or by sharing data from previous campaigns or even from other advertisers present in the bidding system.  In this context, allowlists and blocklists are bootstrapping tools – if there are websites you are basically certain will perform or never perform, you can limit the scope of the learning process to a smaller pool which makes correlations happen more quickly due to decreased cardinality.  This comes at the cost of potential performance – ostensibly, the more places you learn, the better your correlation machine is able to find outcome bundle combinations.

That’s ad tech.  That’s it in its beautiful entirety.  In fact, that’s online advertising in its entirety.  And that we’ve built pipes that surface almost every impression opportunity on the web to these bidding systems is totally epic.

The Framework for Evaluating Everything

In light of this, I’d like to posit that, at its core, there exists a single problem in all of programmatic advertising.  The problem is fundamental, and any other “issue” is an extension of this problem.

That single problem is when our magical machines can’t find a correlation between outcomes and the attributes/fields/numbers in bid request objects – or when they can’t find it fast enough.

When the magic machines aren’t correlating, it means that advertisers aren’t achieving results.  And no matter how you spell it, if there aren’t results, the industry dies.  Right now the problem is that walled gardens, like facebook, are driving better results with their correlation finding machines than programmatic correlation finding machines.  This is the lens I use to view the world – will X fundamentally disrupt the functionality of the correlation machines?

Now some of you might be like “Gareth there are so many other problems!! You’re so wrong!” so let me pre-empt the whining for a few of these : 

1. Advertisers run on websites that they don’t think look good/have bad viewability/have lots of ads/ blah blah blah!

   1. This is not an ad tech problem.  Ugly websites might drive outcome correlation for you.  The name of websites does not matter past the values in the bid request objects.  The ad density does not matter past the fact that it impacts expected value in correlation calculations (and ad density measurements can be be included in bid request objects, at which point they’ll create an optimization node).  If your attribution system communicating outcomes is configured properly with your magic correlation machine, no attributes of the website matter past their relationship to outcomes over time.  This means properly configured campaigns will stop buying from websites that don’t work for them – if your campaigns aren’t doing that, the problem is not inventory, it’s your outcomes or your magic machine.

2. CPMs from this system are too low!  Publishers don’t make enough money!

   1. Make a website, or attract users, that drives better outcomes based on the bid packets it sends out.  If your bid request objects and impressions don’t correlate to advertiser outcomes, your business does not “deserve” to get advertising revenue.  Reputation does not matter.  Only advertiser outcomes do.

3. There’s so much inefficiency!  Middle men are taking money left and right, and buying on shitty websites!

   1. As long as advertisers are pumping meaningful goals into the magic bidding machines (and as long as there aren’t monopolistic forces at play, which god knows there are), the inefficiency will be squeezed out over time.  We are indeed seeing this right now with header bidding squeezing down exchange and network take rates.

I believe that every scandal, every problem, in ad tech should be viewed through this lens.  Is there something happening that is breaking our correlation machines?  If we’re breaking the correlation machines, we need to mobilize like white blood cells, surround, and extract.

Notice, this does not mean that I think there is a dire need to change inventory curation, SPO, DPO, do direct sales, be more premium, blah blah blah.  Why?  Those things aren’t solving problems that break correlation machines.  Sure, the problems they’re attempting to address might slow them down or help them, maybe?  There’s only one person who knows the answer to that question – and that’s the DSP executing the correlations and their data scientists, and I trust them to find market efficiency on this one (as long as their outcomes are properly defined, which is a whole separate set of issues but the programmatic supply chain isn’t really responsible for that).  It’s their job to evaluate the fields inside of the bid request objects – and their job to determine whether fields correlate to outcomes or not.  

If I can send one message with this piece it’s this – I’m a deep believer that an ecosystem with properly configured outcomes, and survival of the fittest amongst the magic correlation machines, will naturally steer itself away from inefficient configurations and inventory.  And history supports this.  We just need to let it keep playing out.

SO LET’S RUN THROUGH SOME EXAMPLES JUST FOR FUN

NOTE – for all of these examples where I say “test,” the only way to test things is with a test/control framework.  Within a single campaign you need to be able to do a thing a % of the time – if you have two separate campaigns that are identically configured that’s “okay,” but if there are any different variables at all, you ain’t testing a thing.  This goes for running campaigns in general as well – in fact, I think all campaigns should serve a statistically significant % of PSAs to prove that the campaign is doing anything at all.

Bootstrapper = potentially makes campaigns optimize more quickly and perform better

Destroyer = always makes campaigns perform worse

3rd party user segments  - When I include this 3rd party user segment, does my cost per outcome improve when compared with a control group (ie. a campaign configured identically other than the targeting of the segment)?  Give a shit category – low, “bootstrapper,” “I’ll test it.”

ID Bridging and ID Fraud - My correlation machine is being fed falsified data.  Holy shit, this breaks all of my processes, this is probably the worst thing that can happen.  Give a shit category – maximum, “destroyer,” “I’m literally calling the police”

MFA Vendors - When I include MFA sites in my campaign, does my cost per outcome become worse than when compared to not including MFA sites? Give a shit category –low, “bootstrapper,” “I’ll test it.” Zero difference between an MFA vendor and a contextual data provider.

Domain Spoofing - My correlation machine is being told I’m on domain X, but I’m actually on domain Y.  Give a shit category – maximum, “destroyer,” “I’m literally calling the police”

Anti Fraud Vendors - When I include anti-fraud prebid targeting, for people that this vendor thinks are fraud but others don’t necessarily, does my cost per outcome decrease? Give a shit category – medium, “destroyer,” “I’ll test it.”

All Campaign Targeting  - When I manually add targeting to a campaign, does my cost per outcome decrease faster/further than when I don’t have targeting? A note – I don’t believe anyone should ever be putting targeting on their campaigns.  DSPs should provide correlated bootstrap data for the learn process by advertiser category, and algorithms should be able to handle the rest, but I digress. This is a future that needs to come sooner rather than later. Give a shit category – low, “bootstrapper,” “I’ll test it.”

SPO / available purchase channels - does this publisher sell on lots of exchanges? Should I pick a single place or a small number of places to buy their traffic? Give a shit category - low, “bootstrapper,” “I’ll test it.” Note : I don’t think this will do anything because Prebid creates a downstream unified auction

Google Being a Monopoly and controlling access to bid requests - Is google manipulating auctions to limit what information can be communicated to who, when?  Are they introduce errors into the delivery of bids and creatives? Give a shit category - high, “destroyer,” “Call the FTC”

Please feel free to pepper me with more requests about things you’d like slotted into the framework / disagree with my characterization of things! Have a wonderful week everyone!

It seems to be a universally loathed thing when discussing buying advertisements on websites programmatically, so I’d like to unpack it and dig a little deeper – and hopefully provide some insights and maybe even change some minds.  Here we go!

1. What is paid traffic?

Paid Traffic, in the context of this article refers to visitors to a website that a publisher has paid to attract.  How would one do that one might ask?  Easy – a publisher shows an advertisement, somewhere, a user clicks it and then navigates to their website.

Most people think of outbrain and taboola - with native tiles at the bottom of a content well advertising what many would refer to as “low-quality” content (which I think is an elitist way to speak about the things that people read, but I digress).  But, the fact of the matter is that any kind of content, or any kind of experience, can advertise using any number of methods – in fact, via all of the same methods that agencies use on behalf of brands.  Publishers, in this context, are the advertisers.  If it can be part of a brand’s marketing mix, it can be part of a publisher’s.

2. What is organic traffic?

People almost always describe “organic” traffic in contrast to paid traffic.  For the purposes of this article I’m going to lump together a bunch of things that should probably be separate – but really, organic traffic typically refers to traffic that is sent to a website by some kind of “organic” discovery vehicle.

This could be google search – where for whatever reason, Google’s search algorithms choose to send traffic to the website because they think it will satisfy the intent of the user typing their prompt into the search engine.  It could also be a social network or facebook – where a user shares an article, a given audience picks it up, and Facebook decides that this article will enhance the user experience of certain audiences in facebook and will show it to them (this is rarer nowadays).  

But the notion behind organic traffic is that some kind of content discovery platform is choosing of their own volition to send traffic to the website, typically with basically zero control exerted by the website itself.  Sure, there are things like “SEO” – where a content creator tries to guess what the organic platforms are going to recommend to their users, but at the end of the day these algorithms are purposefully opaque and unpredictable.

3. Other types of traffic?

There are other types of traffic that I’d refer to as “owned” – and what I mean by owned, is that the publisher has direct control over their relationship with the user.  The two that jump to mind here are “direct/type-in,” which refers to users that type the URL of the website directly into the browser, and e-mail/newsletter, where the content creator promotes content directly using a unique identifier with their user.  Push is also probably in this category.

4. Why should I care about paid traffic?

We typically hear about paid traffic in the context of ad fraud (bots) or MFA (bad user experiences).  However, I would like to make the argument that paid traffic is one of the only things on the web that provides content publishers with control over their own destiny.  

This is because the primary difference between organic traffic and paid traffic is incentive.  Here’s what I mean : 

Organic

Organic Traffic Provider (OTP) has some kind of “discovery” service it provides to its users where it provides them with content > The more appropriate the content is to the desire of the users the more they use OTP’s application > If content provider can guess what kind of content will perform well for OTP then OTP sends traffic to website

In this relationship, the content provider is valuable to the OTP to the extent their content makes users want to use the OTP application more.   The salient point, that should not be lost on anyone creating content, is that if OTP could produce their own content they would (see zero-click search) and their dedication to their content creators is entirely self-serving with zero financial commitment.  In fact, OTP often runs ads next to the content they’re promoting, creative all kinds of perverse incentives as we’ve seen with the latest google scandal https://www.wheresyoured.at/the-men-who-killed-google/ (ironically, link chosen because it’s the first search result on google, what a dark twisted world we live in).

Paid

Paid traffic source (PTS) has their own content / application that they want to make money from by displaying ads (PTS and OTP are often the same company, but the two sides of the businesses rarely interact).  As a business line, they are incentivized to make as much money as possible > They allow advertisers to place ads on their application, ostensibly in a perfect-ish market the advertiser that pays more gets to show their ad > the user clicks the ad and goes to to advertiser defined landing page, if user has high post-click/action value, advertiser is willing to pay more > advertiser pays more gets more clicks

In this relationship, the incentives are fundamentally different from the OTP side of the house.  The user’s behavior and engagement with the advertisers is advantageous purely to the extent that it generates additional advertising revenue for the paid traffic provider – which means that the decision making power actually lies with the advertiser.  Or in this case – the PUBLISHER! – because in this workflow, and in the world of paid traffic, the publisher is the advertiser.

 This means that control over traffic flows is actually governed by the market – and imperfect as it may be, it is meaningfully more egalitarian, transparent, and scalable when compared to the OTP flow by virtue of the fact that market forces are highly influential over what ads get displayed on a page.

Of course paid traffic sources will exert controls over what type of advertiser are allowed to advertise where, etc, but the more artificial controls they insert into the market the less money they will make to the extent that they don’t start losing users – and the users aren’t using their platforms for the ads, they’re using them for something else, so we no longer have this strange relationship between traffic allocation and user satisfaction.

What does this all mean?

To me, this means that demonizing paid traffic is literally one of the worst things we could be doing.  Paid traffic is the method by which publishers can take control of their future – they can successfully advertise whatever content they may have to audiences that want it, and then find ways to properly engage with those audiences such that they make more money from their users than they have to spend to get additional users. PTS providers have to make money – and as we’re seeing, in a battle between user experience on these platforms and money, they often choose money.  It’s a lot easier to alter search results without considering the downstream effect on publishers than it is to turn down advertising dollars.

This means, in my opinion, the solution to all of this mess on the open internet is very simple – publishers need to become sophisticated advertisers, and paid traffic (and properly executing paid traffic strategies) is the foundation to breaking the dependence on organic discovery vehicles.  We’ve already seen the appetite for the migration to the inbox with the meteoric rise of substack – there’s certainly appetite for direct engagement with high quality niche advertisers of content – and publishers owning their paid strategy and scaling is a foundational brick in the wall. By having a strategy to reach new users and build relationships with them, website owners create sustainability, safety, and scale.

It also means that it’s important that publishers and content creators have tools to treat their businesses more like advertisers and brands treat theirs – a focus on ROAS and LTV – which is why we created Gamera (please enjoy my shameless plug of my new company – https://www.gamera.ai/value-analytics/overview.html )

The Caveat

Bad paid traffic exists.  Bots exist. Fraud exists. Bad website experiences exist.  These are all separate problems from “Paid Traffic” that often get lumped in with them.  But I see no reason why “Paid Traffic,” outside of MFA concerns (which are really ad density concerns), would be any more likely to subject an advertising campaign to shenanigans than organic traffic.  Sure, it’s not a great sign if all of a websites users are churned-single-session-clicks – however, there’s nothing inherently wrong with this as long as they aren’t bots and the site performs for your campaigns.  It’s perfectly possible that traffic source is an optimizable parameter for your campaign as a brand — lovely! So are a million other things. These need to be handled algorithmically, and more signal is generally always good, so there’s no need for you to really have an opinion about any of them, leave it to your machines. So don’t fear paid traffic!  Fear fraud and things that hurt advertiser performance.

Welcome to the latest installment of Gareth Hates Ad Tech.  Today I want to step a little outside of ad tech into a wider topic that encompasses ad tech – a discussion of the open internet.

I was inspired to write this article by a few things, but most importantly, by Eric Seufert’s article, The Open Web Is Whistling Past The Graveyard.  I’m writing this not because I necessarily disagree – but because if he’s right, we need to reflect deeply on what we might be losing.  I also think that we need to develop solutions to fix the problems he outlines, and I am happy to tell you all about my current project dedicated to doing so (so this is also a soft pitch, but I’ll try not to make it too unbearable, and focus on the why and not the how).  

Let’s start with Eric’s twitter post, and then I promise to stop co-opting his content and I will launch into my own diatribe :) 

The open web seems to be whistling past the graveyard in the face of third-party cookie deprecation. The economy of the open web, empowered through third-party cookies, is not too big to let fail.

In my view, the collective, unwarranted optimism of the open web with respect to third-party cookie deprecation in Chrome is anchored to three core misapprehensions:

One: Privacy Sandbox will ultimately deliver advertising targeting and measurement functionality that is comparable to cookies.

Two: Alternative identity solutions will meaningfully supplement the loss of user-level addressability instituted through third-party cookie deprecation.

Three: The open web is so vital to the consumer internet experience that it simply won’t be allowed to wither and deteriorate.

First, let’s lay out some table stakes – 

What is the open web?

The open web as defined in this article refers to the spider web of websites accessible via web browser using DNS (Domain Name Service) to access the website you’re looking for.  DISCLAIMER ENGINEERS PLEASE DON’T COME FOR MY HEAD,

What’s the Domain Name Service you say?  I’m so glad you asked!  It’s actually a wild service administered by the US government that maps all of the word-names of websites (ie. cnn.com) to the servers that resolve to where their content is hosted (ie. 192.241.241.134, aka an IP address).  The Domain Name Service, interfaced with on your behalf by a number of DNS providers like Amazon or Namecheap, allows you to register the names of your websites and the servers they need to resolve to, and then actually routes them accurately!  It’s pretty insane that this piece of vital open web infrastructure is controlled by a country, but I digress.

After DNS registry is done, you type cnn.com into your browser and your browser gets redirected to the servers where the content is hosted.  Nowadays, almost everyone uses a CDN – a content delivery network – to house their content.  CDN’s are special because they’re companies that specialize in distribution – IE. instead of a few central servers that everyone’s computers need to communicate with (if you’re in Ad Tech, you’ll be familiar with the notion of “Data Center,” Data Centers house servers) CDNs have bazillions and bazillions of data centers and servers all over the place, so they’re close to the users who are trying to access the content and it delivers more quickly.  This comes with some caveats in their functionality and flexibility (which is evolving right now l, and is a very cool space), but they’re real good at loadin’ websites.  They’re a fascinating form of “decentralized centralization,” which is a good reminder the “centrality” is a spectrum and not a binary.

What role does a browser play?

So, in this summary of the internet, we have the domain name service mapping website names to servers, and those servers loading the content via CDNs.  That is our high level architectural map – but even more important, how do end consumers access this infrastructure?  The answer is a web browser – a fundamental piece of internet infrastructure that has been a warzone since its inception.  It’s a warzone because of the tremendous amount of influence it wields – the web browser a user is utilizing is basically the tool by which they engage with the internet.  That’s why netscape was worth a bazillion dollars, why Microsoft was so insistent on bundling in IE with their operating system, and why Google went to tremendous (read: sketchy) lengths to get Chrome installed absolutely everywhere they possibly could (bundling, etc).

Browsers, when they connect to these servers, literally download files (html, css, javascript, stuff) and then assemble those files on your behalf.  Then, poof, magic, a website shows up in front of your face!  The reason I’m highlighting this is that this abstraction, this concept, is often poorly understood – browsers literally download and assemble files to create websites.  They’re not magic windows into some magic thing that exists somewhere else – that’s not how they work – they’re downloaders and assemblers. 

Why is this architecture important?

The reason I think this is so important is because of how flexible it is.  Because they’re downloaders and assemblers, the contents of those files can be virtually anything within the confines of supported web languages and market (ie. html CSS and javascript).  This provides content creators with a tremendous amount of flexibility.  Combine this with the fact that you have tons of DNS vendors, tons of CDN providers, and multiple browser options, and you get to what I want to be the brunt of this section – the architecture of the modern open web is decentralized and democratized.  Website owners have nearly unlimited power to choose their own destiny.  They can choose how their websites look, what they do, how they’re monetized, everything.  They can also, very importantly, provide access to anyone using a web browser, with open standards.  

This stands in stark contrast to what I would consider closed platforms.  There are two types of closed platforms that jump immediately to mind for me, but there are probably lots more  – 

1. Walled Gardens - Twitter, Facebook, etc.  These are platforms that have significant limitations around how content is discovered, displayed, and monetized – as the experience is mediated by a single platform.  

2. App Environments – App Environments tend to be owned by OS providers – which is actually interesting semi-vertical integration compared to the desktop software distribution model. This means those providers get tremendous say in similar realms to the closed internet platforms - discovery, functionality, and monetization.

Why does closed vs open matter?

In 2024, everything is trending towards closed, and I think beyond sucking for Ad Tech (where I spend a depressing amount of my intellectual energy), it is going to suck for society and the world.  With the rollout of privacy sandbox, Google is trying to vertically integrate monetization with their browsing environment, essentially making Chrome into a centralized monetization platform on top of the decentralized nature of the open web.  It’s the first step in turning browsers into an increasingly closed environment, with the rules written by Google.  It has the lovely side effect of significantly reducing the amount of revenue that independent site owners can generate from their websites, which alongside content summaries directly in google search results or feeding their LLMs with the crawled internet, looks to me like a giant middle finger from Google to the publishing and independent site world.

But this article isn’t just about Google! Let’s try to talk through some distinct ways in which Closed environments are just, like, super lame dude for lots of use cases: 

1. Closed Environments Exert Control Over Information

One of the hallmarks of a closed browsing environment is they control who sees what, when.  When you log into facebook, facebook controls what posts you see.  When you log into Twitter, twitter controls what shows up in your feed.  When you go into the app store, the app store controls which apps you see.  If you search within facebook, a centralized authority chooses what pops up in that search – with the ability to choose what appears or doesn’t appear.  There’s no such thing as a “non facebook facebook search engine.”

This is a scary future.  We are already struggling to deal with understanding the effects of these systems on our psyches – this closed architecture means that companies, and in some cases individuals, have the ability to exert control over information in their platform.  We certainly don’t need more of this.

The architecture of the open web stands in stark contrast to this.  Anyone can access anything they like, and there is simply safety in decentralization – taking a website off of the internet requires significant intervention into tech companies that have zero stake in controlling content and actually differentiate themselves on their neutrality.  You have to work pretty hard to get “de-platformed” from the internet.

Additionally, if we feel like the results we’re getting from our search engine are biased or not what we want, we can just use a different search engine.  Poof, problem solved.  Google’s search dominance is a result of people wanting to use it, not a result of there not “being any other way to search.”

2. Closed Environments Insert Themselves as Economic Middle Men

Another hallmark of closed environments is when they involve themselves in the financial transaction generated as a result of the usage of the platform.  In facebook, for example, often content creators actually get no money at all.  In Twitter, they’re rolling out a monetization strategy, but other platforms are more generous, like Youtube, where content creators get a meager cut of the money generated by ads shown to their users.  App Stores, similarly, take revshares on money generated by payments processed through their systems, and are notoriously greedy.

This model, which underpins most of the closed systems, creates strange bedfellows and incentives.  Firstly, it means that these platforms are constantly doing a balancing act between surfacing content to their users that their users actually want to consume (more consumption leading to more revenue) with surfacing content that makes them more money within the parameters of the closed environment (some pieces of content lending themselves to revenue more effectively than others).  This battle, which exists in all publishing, and let's be clear these content platforms are publishers, is magnified exponentially in these large scale platforms, where I’m sure they have entire teams dedicated to doing the math here.  And once you’ve committed in some way to using a given content platform, these biases become inescapable (and strangely influential in your life).

It also means these platforms typically engage in sketchiness around monetization.  When they become the bidding platform, the auctioning platform, and the entire ecosystem for tracking campaign performance, you encounter all sorts of shenanigans.  You get facebook gaming video quality metrics, you get Google manipulating auctions to make more money (seems to be a topic du jour, but they’ve always done auction madness), and in general, you get non-transparent systems that the greater internet community will have less and less of a say in.

There are some great examples of the lengths to which these closed platforms are willing to go to exert their economic control.  One that pops immediately to mind is Epic Games, the creator of the ubiquitous game Fortnite, enjoyed by 500 million human beings across the globe.  The structure of Apple’s App Store is such that they demand a percentage of subscription revenue for apps delivered via their platform, and that percentage is meaningful.  Epic told them to take a hike, that ruins their unit economics, and Apple held strong.  They’re in a lawsuit - but my goodness – the most popular game on the planet is not available in the App Store because the closed environment is that serious about its stranglehold on the economics.  There are examples of this all over.

3. Closed Environments Stifle Innovation

Most importantly, I’m a believer that the nature of these closed platforms inhibits innovation.  When content can only be delivered via a social network, there are significant limitations on how that content can be displayed and what it can contain.  These limitations are not up for debate and it is inherent to the construction of the central platform that they’re not democratic.  How things show up in Facebook is dictated by Facebook and Facebook alone – and you don’t need to be a believer in Marshall McLuhan’s the medium is the message to see how placing limits on the various mediums that can exist for delivering content can meaningfully alter the effect that content has on our psyches (imagine a world where all we had was twitter and tik tok – what would conversations look like? What would humanity become?  I shudder to think.)

This is true in App environments as well – Apple and Android both exert significant control over the types of applications that are allowed in their app stores, which means that lots of new app developers spend significant cycles just trying to figure out how to keep Apple and Android happy.  There are also plenty of examples of Apple and Android de-listing apps “because they didn’t like them,” and I’ll spare you the chat gpt lookup of “examples of popular delisted apps” because nowadays you can do that yourself and by giving you the query I’ve basically delivered the content 🙂.

The fact of the matter is that not only is the open web a flexible platform for developing new media and methods of delivering information and activity, it’s an open and flexible platform for distribution of software.  It’s like a giant, relatively free system that allows builders of things or writers of stuff to freely deliver that stuff to anyone who can click a link.  We need to keep this around, and in order to do that, we need to make meaningful improvements to how site owners make money.

So what do we do?

To bring this full circle, I’d look to loop back to Eric’s post and dissect it : 

The open web seems to be whistling past the graveyard in the face of third-party cookie deprecation. The economy of the open web, empowered through third-party cookies, is not too big to let fail.

One: Privacy Sandbox will ultimately deliver advertising targeting and measurement functionality that is comparable to cookies.

Two: Alternative identity solutions will meaningfully supplement the loss of user-level addressability instituted through third-party cookie deprecation.

Three: The open web is so vital to the consumer internet experience that it simply won’t be allowed to wither and deteriorate.

I’d like to synthesize this a bit – really, the brunt of this post is that the economics of the open web are reliant on cookies.  The explanation for this reliance would be that in order for site owners to generate money, they need the performance benefit provided by persistent cross site tracking to advertisers via cookies in order to generate CPMs to stay viable businesses.

This is not an unreasonable supposition.  CPMs have been going down as it is, and online businesses have been failing.  A big hit to CPMs, some estimates have on the order of 40%, might just sink the internet altogether – and in his words, it’s not too big to fail, and if we just keep cramming on more ads to make up for the decreased CPMs it will get so shitty to be on the web that everyone will just stop.

This is the pessimist’s take.

HOWEVER – distilling this even further, what he’s really saying is “a 40% hit to CPMs is coming and the web might die”  That, my friends, sounds like a problem that we can figure out some solutions to.  It’s time to fill that glass up halfway, so let’s do some things.

1. Increase CPMs on the open internet

I have a lot of blog posts written about this already.  But the method by which we auction programmatic ads, the data included in those auctions, and the tools we implement (ad units, placement IDs, identifiers) are in a savage state of disrepair.  I don’t think it’s unreasonable to think that we could meaningfully increase the CPMs of every website on the internet by fixing the infrastructure itself.  I’ll put this into bullet points, but probably best to read some of my other articles, and then readers of this please go start some companies : 

• Standardize inventory classification using a stable, persistent, performance-difference indicating inventory identifier.  Give advertisers something meaningful to optimize against over time.

• Standardize how we pass third party sources of information.

• Make sure your auctions are not staggered, supply path optimization does not matter if you do not have a single, centralized auction where everything competes on bid price.

• Make sure your ads are viewable, lazy load ads, think about the things that will improve advertiser outcomes not things that you think “look nice.”  There’s a lot written on the internet about “ugly ads performing better,” in my experience, the same holds for websites, ugly websites tend to perform better for campaigns.

• Run everything on earth through a single, optimized prebid instance managed by someone who knows what they’re doing.

One of my overarching themes for this is that websites need to work for performance advertisers.  Most of the bellyaching I see on linkedin is about “brands” being defrauded, or not trusting programmatic, or programmatic blah is doing this to big brand x.  This is all a distraction.  Closed platforms don’t succeed because they work well for brand advertisers, even though they do.  They succeed because they work for performance advertisers – simple, post click, performance advertisers.  If your website works for that, it works for everyone.  This is the goal we need to achieve for programmatic to survive, not to roll out some bullshit enhancement that incrementally improves a deck built by a trader for an agency shown to a CMO somewhere.

2. Site Owners need to rethink their businesses and incorporate unit economics

Prepare thyself for a soft pitch of my new company, which makes this blog post our official exit from stealth mode.  The open internet, as it has existed to date, is lacking something that every closed ecosystem has and is actually the foundation of most of their businesses - unit economics.

What are unit economics you ask?  Unit economics is the study and understanding of the cost and revenue associated with a single customer for a business.  So, in simple terms, how much money do you make from each user vs how much money does it cost you to acquire that user and then provide whatever services you do for them.

For advertising in apps, this is the lifeblood of the app install industry.

For meta, they can tell you this with excruciating detail.

For the ad-driven open internet, this data is nearly impossible to get.  If you are a site owner reading this, I put to you this question - how much money do the different types of users who visit your site generate over their lifetime?  What’s the difference between logged in users and not logged in users?  How about users who read your sports section versus your entertainment section?  Or users who found you through search vs social media?  And I don’t mean estimates, I mean for each of these different subsections, what does the distribution look like and how long does it take to reach the asymptote of revenue generation over the lifetime of a user?

If you can’t answer those questions, easily and offhand, how can you possibly profitably grow your business?  

I would argue that cookies have been a crutch – that website businesses haven’t “gotten with the times” the way that app businesses have because they haven’t had to.  The old school mad men days of “well, we just need to charge higher prices for our inventory and do more sales” compared to how much you pay your content creators, combined with free traffic from organic sources (which, lo and behold, is drying up) gave them the semblance of being reasonable businesses, when in reality there were some major blind spots in how to logically scale and operate the business.  AKA - Free Traffic + Unsophisticated Ad Dollars from Brand Advertisers = shitloads of low-complexity revenue that looks extremely profitable.

This is why we started Gamera.  Gamera is a Google Analytics competitor / alternative / complement (they’re not mutually exclusive at all) that incorporates the notion of value into your site analytics, especially for companies with on-site advertising revenue.  We don’t think that it’s a coincidence that the most popular website analytics platform was built for things that primarily benefit search engines and tangentially site owners – that site owners have been told to purely optimize to experience and behavior without demanding any insight into revenue and value.  We don’t think that company necessarily has your best interests as site owners in mind, through no fault of their own, I wouldn’t expect them to do extra work for no real benefit on their side.

The foundation of Gamera is that we provide you with precisely these insights – we don’t just think about single user sessions, we don’t just think about bounce rate – we answer questions like : 

• For your consumers with the highest lifetime value, what parts of your website do they regularly engage in ?

• Who are these users?  Would you like to segment them and do stuff with those segments?

• How do you get more users that are exceedingly profitable on a user by user basis, not by comparing aggregates with other aggregates that don’t actually tell you how to grow your business?

• Which users will have a net LTV benefit to paywall?  What are the tradeoffs in various forms of paywall for the bottom line of my business, not on an aggregate basis, but user by user?

I think we actually need these analytics in order for site owners to properly improve the performance of their websites for advertisers – because there’s no better feedback loop than yield!

If you’re interested in learning about value analytics, and LTV, please reach out!  We’re entering a closed beta now, and I am personally working with our first cohort of customers.  You can find more information here : https://gamera.app/value-analytics/overview.html 

It’s time for a new generation of thinking – the good old days of scotch and compelling powerpoints are not going to protect the internet against the walled gardens.  We hope to be one brick in the foundation of a new (very not exclusive) clubhouse for the open web.

My thesis here is that doomers who believe that online publishing is broken and that all online publishing is going to die are wrong because while they have successfully diagnosed the symptoms afflicting legacy publishing (less moneyz), they have overlooked the root cause of the illness and therefore the cure.  I’m also going to give you guys some free business ideas, and I would gladly chat about them in more detail if someone was really interested in fixing these things.

Background

The commonly held belief at many “premium” publishers is that programmatic CPMs are so far below direct sold that they need to focus on direct sales and PMPs, and not only that, if they didn’t have direct sales and PMPs they’d be out of business.  And based on current data, that’s often true!  

But I’m a firm believer that current data is actually a misrepresentation of what open programmatic could be.  The CPMs generated by open auction today are crappy and low and unsustainable, yes, but that doesn’t have to be the case! And my thinking for why this doesn’t have to be the case is that the average CPMs on Facebook (and even sometimes on GDN, but I digress), are actually insane, like, $10,$20,$30. And while I’m not certain we can get programmatic all the way there (wouldn’t that be nice), my belief is that programmatic is broken, but broken in such a way that it can be fixed.

For our prototype, this article will look at Facebook, a wildly imperfect, impossible to use marketing UI, that has generated CPMs for Facebook’s inventory (read : banner ads with little squigglies on them on a UGC forum publisher) that would rescue just about every publishing business on the internet.  We’re going to start with the nature of facebook inventory and media buying, see if we can identify some deltas, and then run through some possible solutions.  The question we’re working to answer here is “can we make programmatic behave like facebook,” or at least, “can we make programmatic behave enough like facebook that our average CPMs increase 5x.” And no, I do not expect that this article is exhaustive, so please feel free to mention other things in conversation but don’t yell at me for “forgetting XYZ.”

What is the facebook advertising experience?

Facebook is, at its core, a user-generated content website.  It’s a forum.  It’s a forum that connects you to your real-world (and non real-world) peers, and allows them to share images and content pertaining to their lives and interests in a curated feed, a feed simply optimized to engagement metrics (like viewability or “attention” or CTR).  It is not a magnificently difficult thing to build – Facebook’s difficulty comes in its network effect scale, which is nigh impossible to replicate as the switching costs are massive for users who have uploaded their lives to their system.

But this means the advertising experience is simply not all that different from the advertising experience on a normal website.  

There are ads in the sidebars, ads in the content well, and on your mobile phone there are big 300x500 ads that occur probably at a higher density than your average publishing website.  On my mobile feed, every 2-3 tiles with content I get a gigantic ad.  This is comparable ad density to MFA inventory, and as Ari illuminated on his podcast, not only is the experience similar – MFA sites are often facebook traffic!

This brings us to the crux of our discussion — if Facebook has MFA ad density, and MFA users, why is facebook a viable channel for every D2C advertiser I’ve ever spoken to?  On a post-click CPA goal no less?  And while charging freaking $30 CPMs sometimes?!?!

Two Possible Explanations

Explanation #1 - Facebook’s behavioral data is simply that valuable. This is perfectly possible. Facebook has social graph data, it has demographic data, and they certainly say that they use it! But programmatic has data providers too. Programmatic has credit card data, has perfect deterministic user behavior data (or did when it had 3p cookies), and all sorts of other fancy stuff. Hell, programmatic companies spent and spend literally billions https://www.forrester.com/blogs/epsilonpublicis/ building data solutions to improve the performance of their campaigns. But even with all of that, is programmatic really 10-20x worse at data than facebook is? I’m not so certain…

Explanation #2 - My controversial opinion here is that the inefficiency of ad tech is what causes the lion’s share of the delta in price for facebook traffic versus your average programmatic website.  The high prices are not a result of their behavioral user data, though that certainly helps, rather there are a number of things in the guts of programmatic that are keeping us from reaching these CPMs, or at least closer to them.  If I’m wrong, then I blame all of you data companies that are supposedly adding so much value to the ecosystem while you funnel money away from publishers.

A Few Potential Differences

So, on to the assumptions – what are some issues that facebook doesn’t have that modern programmatic does?  Let’s start with the construction of the inventory itself, and then move on to general industry inefficiencies that I think are “in the weeds” but are low hanging fruit.

1. Fix Inventory Categorization

Problem

Inventory Categorization on the open internet is fundamentally broken. This manifests itself in a few ways – but the one I’d like to focus in on is the concept of “placement” or “ad unit” or whatever.  For the uninitiated, these are definitions that map loosely to ad slots on publishers pages, but also might not, but also sometimes do, but also sometimes exist across multiple sites, but also are expected to be the point around which all optimization rotates…. Get the picture?  The notion of ad slot in programmatic, and across exchanges is nonsensical.

Know what Facebook’s bidder isn’t stuffed with?  The same ad opportunity being described as placement 12345, ad unit 938123, inventory_code Purple_Elephants_Ate_My_shoes, and cid 92XsasjS2 – when it’s literally the exact same ad slot in the exact same experience as before.  Or, even a step further, we don’t have different employees at facebook adding their own method of categorizing ad units to the page for different sections of facebook.  Facebook’s inventory categorization is static, predictable, and reliable from an optimization perspective.

We have structured independent programmatic advertising in such a way that arguably the most important thing for monetizing a website, the structure of the ad unit objects, is determined piecemeal by each of the hundreds of thousands of publishers in the space.  This is actually fucking bonkers.  Literally every publisher is expected to categorize their own inventory and lay out the ads on their websites.  

WHY!?

WHY IS THERE NOT A COMPANY THAT CENTRALIZES THIS AND DOES THIS FOR EVERYONE!?!

This is probably the deepest issue with the construction of independent publishing that inherently makes significant swaths of the internet basically unbuyable and unusable for the purposes of online advertising.  And I’m sure there are some sophisticated publishers who will read this who are saying “Well, I can do better, so I’m glad I have control”  – to which I reply, great, good for you, your website isn’t attracting tens of billions of dollars in performance marketing spend that would keep our entire ecosystem alive.  We need a solution that works at scale, not just one or two pubs who think they’re savvy.

 Solution

There should be a company that generates optimization-stable and performance insightful GPIDs (Global Placement Identifier, thank you Prebid and Trade Desk for at least conceptualizing this, though I give some old Magnite buddies credit for PBAdslot which was supposed to be the same thing) on behalf of publishers as a service, looking at things like page geometry and page content.  This company needs to exist.  Every publisher who uses prebid should be able to work with them, and this company should provide a consistent, predictable, and standardized method to categorize the inventory on publisher’s pages.  And no, super sophisticated fancy publisher, I’m not talking to you.  You clearly aren’t moving the needle for saving our industry.  I’m talking to the 95% of publishers who don’t have the time or resources to invest in this.

2. Fix Inventory Itself

Problem

Buckle up folks, this one is a little radical.  Let’s say we’ve fixed inventory categorization, and my magical vendor that works all kinds of magic with GPIDs exists.  That vendor still has to work relatively hard – there are so many bazillions of ways that publishers mess up their websites with inconsistencies, outstream video players (lol), and all kinds of on page madness, that buyer predictability still suffers.

Facebook does not suffer from this.  Every part of facebook is very very very consistent.  They know precisely what it is, and there’s very little guesswork.

Solution

What if we rebuilt the internet to take the guesswork out of it?  What if there were an IAB/OpenRTB defined “centralized site layout” that maximizes performance and standardizes things like ad-density?  Could be on Wordpress, or any CMS really, but there should be an advertising industry sanctioned open source website template!  This layout should be available for free, and it should be available to all publishers who want to make as much money from their ads as possible.  It should lead to automatic approvals from Ad Exchanges and DSPs, and it will help the independent internet grow.  This is a pipe dream, and probably won’t happen, but if TTD wants to build it maybe we could get a bunch of mid tail publishers to use it, and we could have an ecosystem emerge around customizing it without meaningfully changing the guts of it (and if it's open source, we can have substantive conversations about what would be involved in maintaining its core goals while making modifications).  I expect it to function a little bit like native advertising in reverse, which should also be completely open source.

The next few that I’d like to propose are not so much inventory specific, rather programmatic ecosystem specific.  These are changes that I’d like to see in Prebid, or in the programmatic ecosystem at large.

3. Demand Filtering : 

Problem

One thing we regularly discussed at Prebid was demand filtering – aka – brand safety controls.  

Most publishers with any form of direct sales, or publishers with brands they’d like to protect somehow, have advertiser and category restrictions on what kind of ads can run on their sites.  The problem is that these are configured ad exchange by ad exchange – and every ad exchange has its own taxonomy!  This means that an advertiser can be blocked in one exchange, not in another, or restricted in a third – or even just completely overlooked if a publisher adops person forgot to log in to one of their 10 ad exchanges.  And maintaining this taxonomy, and this mapping of advertiser URLs to brands to categories, is a huge pain in the ass for the ad exchanges themselves, and something they all invest money into.

Solution

We need an open source company that maintains a centralized repository of all brands running in online advertising.  In order to be eligible to run a campaign, you need to register your brand with all of your landing page URLs with a centralized service that will provide you with a brand identifier, that can then be centrally managed in an advertiser quality module on top of Prebid.  This will be open source so everyone can see everything, there are no disagreements, and there’s no asymmetry. It will also provide publishers data on which advertisers are running on their sites – something Publishers deserve – or at the very least allow them to say they don’t care.  And Google, I’m looking at you, with your “various adwords publishers” categorization.

4. First and Third Party data communication

Problem

Ad Exchanges and OpenRTB are a messy hodgepodge of data communication structures.  We have EIDs, we have FPD and PPID, we have PMPs which act as proxies for data providers in lots of ad exchanges, and I’m sure there are a bunch of other acronyms that I don’t know that are used in moving data back and forth.  This shit is crazy.  Know who doesn’t have 123091231029 ways to communicate data, and therefore optimize against that data?  Facebook.

Solution

We need one, centrally managed data insertion point – and Prebid is the perfect candidate once again.  Publishers need to hold single prebid auctions for all of the ad units on their site (per my other articles, please don’t screw this up by loading a bunch of different prebid instances and fragmenting your demand stack), and within that prebid auction we need a central taxonomy adding any relevant data providers to the impression opportunity and then blasting any of the relevant data out to all of the exchanges in a standard OpenRTB field.  The notion that there are so many decentralized ways to dress inventory up, and that exchanges are often doing this as a value prop, is ludicrous and we need better solutions.  A lot of the infrastructure already exists for this – but things are still done piecemeal because ad sales people be ad salesin.  Stop it.

I promise, The Trade Desk has lots of people working in their machine learning department.  If on every impression, from every exchange, they had 55 uniformly communicated but discrete segments that provided varying levels of utility, they’d be thrilled to filter the signal from the noise and optimize their campaigns using them.  Let’s just actually give our sophisticated bidding platforms a chance, and get them de-duplicated centralized information about impressions that isn’t asymmetrical on every exchange they buy from.  And now, when every exchange is sending the exact same data for the exact same impressions (and the same number of them, if you’re bid jamming in this scenario you’re very easy to catch), we get to have a real race to the bottom on fees and figure out why ad exchanges exist.

I’ll also take this opportunity to indict the modern day ad network – companies that buy average inventory, apply some kind of data to it, and then mark it up for resale with non transparent margins.  Yes, that business makes lots of money.  No, that business is not good for independent ad tech.  Do you want Publishers to keep dying?  Because this is how we kill publishers.

5. Refocus on Performance

Buy-side, this article is not letting you escape unscathed.  The foundation of Facebook advertising is performance buyers.  These are people who are looking for Direct Response, and not just Direct Response, real post-click sales of goods and services.

Problem

If Programmatic doesn’t work for DR sales for mom and pops, which right now it really doesn’t, it doesn’t have a bright future.  The CPMs we see on facebook can only be generated by actual value generated for advertisers – not by vanity loosely attributed metrics, but by sales.  Because then a mom and pop shop, or a D2C startup, can pump money into it and have a company - but spending $10,000 on “lots of high-attention ads” and “high probability of walk-ins having seen a digital ad” doesn’t help them.

Programmatic’s backbone is agency bulldinkie with loose performance goals that are easily manipulated.  This is also why it’s so vulnerable to ad fraud.

Solution

Non-Retargeting Programmatic has to refocus on CPA, and even more specifically, has to get good at post-click CPA.  Yes, I know this can be gamed.  No, I don’t think it’s easier to game this than just about anything else.  If you refocus your whole business on post-click CPA, I promise, you can stop paying fraud providers and stop paying middle men data providers who protect you from “THE BOTS.”  Because bots don’t click and then buy, or if they do, they don’t do it en masse.  Therefore you will stop buying traffic from them if you’re optimizing properly and if we re-do the internet with performance in mind.

Conclusion and Disclaimer

I’m bullish on banners.  I really am.  I think as time passes, as long as we don’t let privacy infrastructure lead to the entire internet becoming a walled garden, we really have a chance here.   But it requires some creative thinking, it requires some cooperation, and it requires some disruption. And also, if we let any self-interested players roll out infrastructure that obviates the need for and breaks all of the things that we as an industry have worked so hard to build, all of these ideas could be rearranging deck chairs on the titanic.

Thank you for coming to my Ted talk, I fully expect zero of this to happen, but hey, a guy can dream.

I know most of you reading this will react with indignation – “well, what do you mean work!?” “Of course it works look at all these companies!” blah blah.  But the fact of the matter is that the meaningful online advertising businesses are the walled gardens.  There’s a reason market growth is primarily going to a few places – Google, Facebook, TikTok, etc – and we aren’t seeing anywhere near the organic growth in independent ad tech.

My theory why is simple – independent ad tech doesn’t convert.  Ask any affiliate who has been working for more than 5 years – DSP banners don’t back out.  And I've witnessed this personally, onboarding performance buyers onto various DSPs that I’ve worked at only to see them fail.  Another barometer for this – how many D2C companies do you see built on the back of DSPs?  I know the CEOs of a few big ones and I can tell you programmatic wasn’t even a consideration until they were spending decent 8 figures per year already on the primary channels (FB, Google) and for many of them still isn’t.  This is because it doesn’t convert for them – you can’t build a sales focused business on DSP based buying.  That’s not to say that no D2C brands buy programmatic, I’m sure some do, but it’s not the “go to” channel to scale up your business with your Series A. It’s just how the world is right now.

In order to diagnose why this is, I’d like to look into the history of Programmatic, namely, the history of DSPs and SSPs.

Definitions – 

DSP - Demand Side Platform – a platform built to run campaigns for advertisers (either with an advertiser or agency logging in themselves, called “self-serve,” or with employees of the DSP doing it for them, called “managed service.”  Apart from the functional definition, many people will say DSPs exist because in the transaction they represent the “buy-side” interest

SSP - Supply Side Platform – a platform built to “make publishers as much money as possible” by selling to DSPs or as many companies as they can.  What SSPs do has actually shifted significantly over the years, and that’s one of the first things I want to discuss.  But always their marketing has been built on “we represent the interest of the publisher.”

WTF does an SSP even do anymore

The original SSPs were not in the business of selling inventory to DSPs.  In fact, that’s a different business model – using OpenRTB to sell inventory to DSPs is being an “Ad Exchange.”  But we’ve used them interchangeably for so long that we’ve lost sight of the original purpose of an SSP.  The original purpose of an SSP was to manage publisher integrations with Ad Networks and Ad Exchanges, not to hold auctions with DSPs.

Cue the Time Machine

Why does programmatic exist?  The original programmatic platform, the Right Media Exchange (Sorry Jeff and Bill and AdECN, I did work at AppNexus after all), was actually a technology integrated marketplace for Ad Networks to buy and sell from one another.  I’m going to make a really crappy graphic to illustrate the evolution that brought modern programmatic into being : 

Phase 1 - Basic Internet Pub

Publisher speaks to advertiser, like coca cola, advertiser agrees to pay publisher 13 goats for 1 month of putting their jpeg on publisher website.  Publisher agrees, much goat wealth was had.

Phase 2 - Ad Serving

Publisher has more advertisers that want to work with it.  Also, publisher realizes that some months have lots of traffic, and publisher should get paid more money in those months.  Enter publisher adserver – two primary functions, change the ads that show on the site without having to re-code jpegs, also count the things so you can charge based on counts not on time (maybe some basic targeting too).  New way to sell, many many more goats for publisher.  Advertiser decides to trust but verify, builds advertiser adserver to count and swap things for them to make sure their goats are getting their money’s worth.

Phase 3 - Ad Networks

Companies realize that publishers aren’t the best at getting in touch with lots of advertisers to get as many goats as possible for their ads – and that they don’t need to be a publisher to represent publisher inventory.  Ad Networks are born – people who live to do nothing but sell ad space and don’t have to worry about pesky things like writing or creating things of tangible value to your average human.

Phase 4 - Hell

Phase 4 is known as waterfalling – where ad networks realize it’s way easier and faster to scale up their business if they deal with each other instead of dealing with those annoying advertisers and publishers.  This is primarily driven by a lack of internal liquidity – it’s difficult to scale up supply and demand at the same time within a single network, and easy to extend your liquidity crisis to another ad network (who has lots of scale) instead of trying to keep adding more advertisers and publishers.  This is where every network starts working with every other network using ad tags, and this is the environment in which programmatic was born.

Phase 5 - The birth of RTB, RMX

You’re welcome for the worst graphic in history – but you get my point.  Right Media Exchange was an environment where instead of pinging all your network partners in a line, as depicted in waterfall hell, you pinged all of them at once and everyone cleared in this lovely unified auction inside of Right Media.

Does this sound familiar?  Because it should.

You see, history started with disaggregated demand and supply integrated in a shitty way (waterfalling), except the people doing the buying and selling were ad networks.  So Right Media created a way for all of the ad networks to sell simultaneously to one another.

We have entered this phase, precisely, once again – except instead of the ad networks being the ones who want to transact simultaneously, it’s the publishers, exchanges, and DSPs.  And instead of RMX, it’s prebid.js.

Welcome to the brunt of my article.  In the interest of brevity, here are the next steps : 

1. After RMX was born, Ad Networks became a commodity.  They needed to figure out a pitch.

2. They realized that they could sell to advertisers or publishers more effectively by convincing them “I’m on your side!  It’s my job to make YOU as much money as possible.”  

3. This pitch works.  This pitch works because these people were trading CPC agency budgets, and performance was a commodity (read : didn’t exist, this was garbage).  This is where ad tech starts to die, and while this is a small bullet point in this article, it’s a really really important one.

4. SSPs realize it’s much easier to sell directly to this new type of ad network, DSPs, on a publisher’s behalf than it is to actually manage a publisher’s relationship with their own ad networks.  They all deprecate their publisher ad network management products. Also DSPs don’t want to bother with pub stuff.

5. The terms SSP and Exchange become interchangeable.  We enter a gross phase of ad tech middle men where both SSP and DSP are taking as much in fees as possible.  Remember RocketFuels 50% margins?  Ever seen Pubmatic’s 30%+ margins on S2S sourced inventory?  And they did like, no work, other than stand up a few OpenRTB integrations

6. Prebid comes along to help publishers manage their relationships with Ad Exchanges – the dominoes begin to fall

And this is where we are today.  Ad Tech is like a wave function that has peaks and troughs.  At the peaks, you have a bunch of different companies doing things and managing one another.  At the trough, a new disruptive technology emerges that commoditizes and flattens ad tech and you don’t have companies managing companies anymore, you fewer players and they work with each other simultaneously instead of synchronously/one at a time.

We are experiencing a flattening right now.  And this is probably the most necessary flattening that’s ever happened in independent ad tech – because walled garden ecosystems have emerged that simply perform better than this monstrosity that I’ve been describing for a bazillion paragraphs.  

Why do walled gardens perform better than this mess?

1. Improved Optimization – when a system is ingesting the publisher inventory from the client, interpreting the signals itself, the optimization works better

2. No Asymmetry of Information – walled gardens are aware of how well a piece of inventory is performing for a campaign and what the goals of that campaign are.  By knowing the goals of the campaign, they know what price the advertiser would be willing to pay, and therefore the “maximum expected value yield” (MEXVY) which they can play with to pay themselves.  They also know, generally, how much their inventory costs, because they’re aware of the market for it (it’s their inventory!)

3. Fee Flexibility – as a result of #1 and #2, walled gardens can take dynamic revshares.  Within any modern bidding systems, there are two competing metrics – MEXVY (maximum expected value yield, or the expected performance of a campaign backed into a bid) and the ECP (expected clear price, or how much they think the inventory will cost based on historical market conditions).  The delta between these two numbers represents the maximum margin a buyer can take.  Sometimes this number could be gigantic, like 50%+, and sometimes this number could be tiny, like 1% – but being able to dynamically deliver it is the foundation of a high margin successful advertising business.

Point number 3 is by far the most important.  It’s the most important because in the modern DSP SSP ecosystem it is impossible – Prebid/GAM has the best ECP data, SSPs have “okay” ECP data, and DSPs have all the MEXVY data, and therefore the margin optimization that needs to occur for a system to be competitive in the auction cannot.  Because of this, advertisers miss out on inventory that would perform for their campaigns, publishers don’t get money they could be getting, and nothing scales and nothing works as well as walled gardens.  And don’t get me started on DSPs and SSPs trying to do their own dynamic margin management in this mess, which they do, and everyone loses except them. This is why modern ad tech is “dying.”

But this is all changing now.  This is changing because the separate layers that were born in inefficient post-RMX ad tech land are being rendered unnecessary by Prebid.  Prebid.js is the new Right Media – providing a central location for all buyers to bid on and assess the performance of their inventory.  But it’s RMX on Steroids – because Prebid obviates the need for a management layer (the SSP), by taking something that was a siloed tech platform problem (buyer-of-my-stuff management) and turns it into an open source function.  Now, this also would’ve been solved over time on Right Media, like it was on AppNexus, if RMX and AppNexus started working with publishers and running their own DSP, which is the case now on Xandr and was always a technology capability in both platforms. It’s also why I think buy microsoft stock?

The Rebirth of the Ad Network

What’s going to happen next, or should happen next, is SSP fees will be ground into the dirt (maybe a flat CPM based fee?  I pitched this at RTK, nobody wanted it, but maybe i was just early) or SSPs will go away.  DSPs will begin buying directly from publishers or publisher proxies like prebid management companies, because there’s basically zero friction for them to do the integrations now that Prebid is so standardized, especially if PBS catches on, and DSPs will become ad networks once again with the sell side managed by open source software.

This would be a huge, huge win for everyone.  Now DSPs get to be Ad Networks like the walled gardens, they have direct access to publisher inventory and can build more meaningful performance signals and do proper ECP MEXVY margin management, and they should be able to capture more market share from Google and FB.  Please god like this happen as soon as possible.

In conclusion, TLDR

1. Prebid needs to usher in an era of Advertiser and Publisher collaboration.  What I mean by this is performance feedback loops – publishers need to know if their inventory is performing for campaigns or not, and bidders need to be ad networks with dynamic margin control.

2. Performance needs to be the determinant of pricing – nothing else.  A free market where high performing inventory is properly valued is the best possible outcome for all publishers.  We know that CPMs in this environment are high, as facebook CPMs in general trounce the programmatic CPMs attained by any publisher right now.

3. Advertisers need to start optimizing to actual performance so they stop getting taken advantage of by middle men.  Stop doing “branding” shit.  This is another hill I will die on.

4. If all this comes to pass, we will have an efficient prebid facilitated auction where bidding platforms buy directly from publishers.

5. Please, don’t talk about SSPs and DSPs needing to “protect the interests” of their clients.  That kind of thinking is what screwed us in the first place, along with dumb agencies allowing their money to get stolen by middle men.

6. Prebid Management Companies are for all intents and purposes publishers.  SSPs are going to buck and try to hurt them, but they’re good horses to bet on long term.

7. SSPs will either go away or will fundamentally change – and frankly, are probably unnecessary with Prebid Management companies aggregating small publishers and prebid itself providing all the relevant and necessary functionality.

8. I also love ad networks, lol.

A Few Caveats before you @ me :

1. SSPs could gather MinBidToWin (MBTW) data from GAM and Prebid and pass this along to DSPs. This would help a lot. But like…DSPs could also easily retrieve this information themselves. This is still in the works.

2. I’m not trying to hate on anyone — but there are too many companies doing what can be done by one with relative ease in the value chain right now. Especially when Prebid can centralize just about everything we could possibly need it to, and more, and could probably even build a superior auction environment to Facebook and Google.

Disclaimer : this article is very pub-focused, and relatively technical (but I try to make it easy to understand).  Might be a skipper if you’re interested in “industry wide stuff” – but still worth knowing what publishers care about for the curious.  It’s also not written for engineers, and I probably get some of the tech stuff mildly wrong – but hopefully it sparks debate and gives folks language to make better choices in their monetization.

Latency is the biggest boogie man in pub side ad tech.  Seriously.  Bigger than identity, bigger than SPO, bigger than MFA.  But latency – oh boy – if you tell a publisher you’re adding significant latency to the page, it doesn’t matter how much you’re willing to pay them – you’re getting the boot.

And this is understandable. In publishing, the core product is still a consumable media that has to be ostensibly consumed by an ostensible human.  Interrupting that, or interfering with that, is highly problematic – publishers don’t want the users they have left coming to their sites to leave and go back to indulging their unhealthy tik tok habit.

But where there is fear there is opportunity – and a number of pundits, influencers, and less-than-responsible sales tactics have hijacked the latency boogeyman to their own end to sell products and shore up their moats.  The goal of today’s article is to create a framework for publishers to understand latency, ask the right questions, and get a little bit closer to something remotely resembling actionable strategies for dealing with the relationship between your tech and your page.

New Latency Definitions

Let’s get started with definitions – what is latency?  Dictionaries are worthless with web slang, so I’ll give it a shot – Latency means time.  High latency, or large amounts of latency, imply that something is taking a long time – low amounts of latency, or low latency, means that things happen quickly.  Google kind of corroborates this – the top search result is 

Computer and OS latency is the combined delay between an input or command and the desired output. Contributors to increased computer latency include insufficient data buffers and mismatches in data speed between the microprocessor and input/output (I/O) devices.

This sounds about right – like when a video game has bad “lag” between input and action, or when a user asks a website to load (!) and it takes time for it to actually come up.

In the interest of the topic at hand, this computing concept actually applies in two ways to ad tech and publishing.  These ways are not independent, but still need to be distinct : 

1. Website Load Latency – how much time between when a user requests the website to do some kind of thing on there and the website actually loads/does the thing

2. Ad Load Latency – how much time between when the ads on a website start to load and when the ads render on the page

These two ways, for most people, are collapsed into one.  If you look up latency moaning on linkedin, 9 times out of 10 they’re not distinguishing between the two, when they really, really, really should be.  This is because their impacts are very different.  One impacts user experience, and all the things that are downstream of that, like SEO and time on site, and the other impacts ad delivery, and all of the things that are downstream of that, like ad revenue and ad performance.

Website Load Latency

Website load latency is the primary topic in most articles about latency.  The genesis for people caring about it is twofold – 1. There’s a supposition that there’s a relationship between user experience and revenue (ha ha ha), and 2. Organic search depends on sites loading quickly.  We’re going to park #1 and not address it in this article, as I will address it in the future, but #2 really deserves examination.

The notion that your website’s latency affects your Search Engine Optimization isn’t conjecture – it comes straight from Google.  Google even provides a framework by which they assess how well websites load, something called Core Web Vitals, documented at length here 

https://support.google.com/webmasters/answer/9205520?hl=en

In their description, they make CWV sound like a panacea – with case studies like this : 

• When a site meets the Core Web Vitals thresholds, research showed that users were 24% less likely to abandon page load.

• With each 100ms reduction in Largest Contentful Paint (LCP), web conversion rate for Farfetch increased by 1.3%.

• Reducing Cumulative Layout Shift (CLS) by 0.2 led Yahoo! JAPAN to a 15% increase in page views per session, 13% longer session durations, and a 1.72 percentage point decrease in bounce rate.

• Netzwelt improved Core Web Vitals and saw advertising revenues increase by 18% and page views by 27%.

• Reducing CLS from 1.65 to 0 significantly uplifted domain rankings globally for redBus.

What publisher wouldn’t read these things and these google invented acronyms and make CWV their Bible?  And there’s definitely something to it – a website that doesn’t load creates an unpleasant experience for users, and that should increase bounce rate.

The components of CWV (at the time of writing this article) are as follows : 

• Largest Contentful Paint (LCP): measures loading performance. To provide a good user experience, LCP should occur within 2.5 seconds of when the page first starts loading.  Layman’s terms : How fast do all of the parts of your site load?

• First Input Delay (FID): measures interactivity. To provide a good user experience, pages should have a FID of 100 milliseconds or less. Layman’s terms : If I click a button, does stuff happen?

• Cumulative Layout Shift (CLS): measures visual stability. To provide a good user experience, pages should maintain a CLS of 0.1. or less.  Layman’s terms : How much does your site wiggle around after the initial load?

 And another one called INP on the way that we don’t need to cover.

LCP / FID / CLS

So let’s start with layman’s terms for LCP – “how fast does your website load” – and things that break this will also probably affect FID and INP.  This metric is built around content elements, so with this metric google is really concerned with the content of your website itself.  This means that any advertising technology, or any technology in general, that slows down the rendering of your content will hurt your LCP score.  For me, in my experience with websites, a few things jump to mind that are ad-tech relevant here : 

1. Slow Delivery of stuff – don’t work with crappy hosting companies, compress whatever you can, your vendors that your website lives in are not a place to cheap out

2. Blocking javascript (this also impacts FID) – in the world of Ad Tech, all javascript from ad tech providers should be async.  Nobody should be loading synchronous javascript on your page.  What are sync and Async you ask? See below

3. Too much crappy javascript – a lot of crappy javascript will block up the main thread, which you can think of as the “brain” that’s making all the code on the page execute.  This is what you get when you pay php developers on fiverr to add “more stuff” to your website.

4. Loading too much stuff – this is when you try to load too many things at once, or from too many different people.

All of these things are considerations when it comes to optimizing for LCP, FID, and INP as well.  These are all “high level things” that will slow down your website in general.  However, it is important to note, that all technologies contribute to these issues, not just ad tech.  You can absolutely tank your CWV without any ad tech on the page at all by committing any of the above 4 sins in infinite creative ways.

Sync vs Async in Gareth’s horribly butchered non technical lingo

Synchronous javascript is javascript that has to execute in a particular synchronous order – ie. sync JS has to execute in order for things after it to execute.  If it breaks, it breaks all the things after it.  Old school header bidding was sync FYI

Async javascript can execute simultaneously, if it breaks it will not break the things loading after it.

As I prepare my inbox for a barrage of hate mail from developers, let’s try to finish this section.

CLS

Cumulative layout shift is a relatively self explanatory one – how much is stuff shifting around in your layouts when users are interacting with the page.  I don’t need to dive too deep into this one – feel free to read in crazy detail – https://web.dev/articles/cls – but TLDR, don’t let your engineers make your pages wiggle when humans click buttons or let your ad tech vendors move your shit.  Wiggling is bad, it makes people click things accidentally and get angry.  

Ad Tech Questions

Now that we have an incredibly cursory understanding of CWV, let’s talk about it in the context of Ad Tech.

In summary, we don’t want vendors to unnecessarily damage our CWV score.  Some damage is inevitable, as they are additional code on the page, but you can absolutely ask them questions that should minimize the damage or at least make you aware of it.   Please find my question guide below : 

1. Can I load you asynchronously / iframed / do you do anything that could block the page executing?  They should let you run them on a friendly iframe, or they should guarantee they’re not going to block the page.  That or they’ll just lie to you, but then you can get angry and ask for money.  CWV bullet points 1+2

2. How many network requests do you expect to require?  Tech providers need to call out to their servers or others to do stuff – but how many is an important question.  Expect that the answer will sometimes/almost always be “we don’t know.”

   1. Tech Providers – ie. utilities – should be able to tell you this on a per session basis.  This will help you gauge the impact on CWV bullet point 4

   2. Ad Tech Monetizers are going to need to call lots of people, with some of them outside of their control.  They should however be able to tell you how many people they’re directly invoking.  Is it 2?  Is it 50?  They need to be able to give you an answer, even though the direct invoking number will inevitably be lower than the final number as all of the programmatic hullabaloo kicks in. I could write an article about this one alone, maybe I will.

3. Are you going to add things to my page and move stuff around? This is for CLS – we can’t have people expanding ads up and down, popping things in and out of the page (*ahem* outstreamvideo providers), and ads moving divs up or down (I’m looking at you all “rich” media).  These things will all hurt your CLS score, and make your pages look crummy.  Don’t let people do this.

All of this being said, at the end of the day, how much you actually care about these things is up to you.  Are you super dependent on organic traffic from Google specifically?  A 10 point hit to your CWV score might materially damage your revenue.  Are you a website with tons of direct and subscription traffic that doesn’t so badly depend on Google?  Maybe you don’t care, and you reach out and grab that beautiful outstream butterfly.  

This is not a one size fits all thing.  Whether or not you care about page latency I actually don’t believe is an absolute – because there is one truth that’s hard to fight, and that’s that additional ad tech providers both add latency and make more money.  The question is how much money is enough to justify a bunch of additional code and page-load – and that’s up to each publisher to answer.

What is Ad Stack Latency?

The second type of latency (and the one that I frankly know better) is ad stack latency.  Ad stack latency, for the purposes of this article, refers to the speed with which ads render to the page.  This used to be wildly variable – in the time of synchronous javascript tag waterfalls, it was anyone’s guess how fast ads would get to the page.  Now, with the advent of Prebid.js, we have a wonderful centralized path to control our ad render speed (as long as you don’t add a bunch of stupid outside of prebid widgets to the page that wreak havoc, which I know you will).

A quick conceptual recap – Prebid is a javascript auctioning system for ads.  Prebid commits every cardinal sin mentioned in the above web site latency list of warnings – well at least most of them, it’s not blocking javascript anymore (but it used to be!) – it has to load a giant resource file from a maybe external place, it issues a bunch of calls to third party servers, it has so much javascript in it that it requires a meaningful team of people just to wrangle old code – but goshdarnit, it’s the best thing we have!

And frankly, the alternative would be much worse.  Having a single application to police, a single application to integrate with our existing site experience javascript and GAM, is actually wonderful compared to the alternative where we’re managing tons of different providers on the page.  Prebid actually has rules about what third parties are about to load – and it polices them.  

And this means that as long as all of our partners are running through prebid, or at least substantially all of them, we have a way to control the critical path by which ads get to our page.  We can control the speed by controlling the participants in prebid and how they participate.  We can also control the network requests on the page by adding players individually and observing their behavior.

Does Ad Stack Latency Matter?

That being said, I’m dubious that anyone is really doing a good job of measuring whether or not ad stack latency matters.  People freak out when their “timeouts are set too high” or when their “auctions are too slow” – but why?  A slow prebid auction, and bad ad stack latency really has two primary negative ramifications I can think of : 

1. Breakage – ad units that don’t have time to render before the user navigates away.  This leads to lost revenue due to fewer impressions

2. Decreased Ad Performance – whatever metric you want, if ads are slower to get there it’s probably going to be worse.  Viewability, CTR, etc.

But that's it.  And guess what – we can measure both of these things pretty easily by A/B testing our setups, and in my experience, you have to make major modifications and really fly off the handle to meaningfully affect these things with your ad stack.  9 times out of 10, these metrics are far more materially affected by bad Web Page Latency than they are by the ad stack itself.  So as we walk through the different components of ad stack latency, specifically with regards to Prebid, let’s keep in mind that making a lot of these settings too long might not actually make that meaningful of a difference in your ad stack latency, while making timeouts too short can absolutely negatively affect monetization.

Ad Speed Latency Part 1 - Timeouts

Prebid gives its users lots of controls over the timing of things.  Off hand, I remember at least 3 different timeouts : 

1. Failsafe Timeout  – hardcode JS timeout for the total auction, after this amount of time call GAM.  Killswitch.

2. Real-Time Data Module (my favorite module) Timeout – how long are real time data providers allowed to respond.  Real time data providers are companies that inject data into the auction, like user IDs or data providers.

3. Prebid/PBS Auction Timeout – how long are bidders in your auction allowed to respond

The sum of the latter 2 timeouts is your controlled “auction” timeout – these are the method by which you control how quickly ads get to your page.  Make your timeouts short and auction participants will not get their data into the auction – more often than not, this means you make less money.  Make your timeouts too long, and supposedly ad KPIs and ad impression counts will suffer.

I strongly, strongly, strongly recommend testing higher timeouts here.  One of the top mistakes I see people make is arbitrarily selecting their timeout and then not A/B testing it or at least _trying_ a higher timeout and seeing if they make more money (you really should be A/B testing everything without hurting other ad KPIs (viewability, breakage).  You might need different timeouts by device or geo – this is due to breakage on slow devices – but still test it, sometimes a slick website loads everywhere pretty well.

Ad Speed Latency Part 2 - Network Requests and Partner Count

In addition to your timeouts, which are the mechanics of your auction, publishers also have control over the number of participants in the auction.  This is an important one – this ties directly back to website page speed, and it actually affects both web latency and auction latency at the same time.  The number of participants maps not-so-cleanly at all to the number of network requests issued from the browser.

What is a network request?  In layman’s terms, it's when your browser reaches out to a server to get some kind of information back.  It could be data, it could be bids, it could be an image, it could be another website – but your browser is talking to someone.  To put it in perspective, at the time of writing this article, on a random page load, cnn.com had over 600 requests in the first 10 seconds of the page loading, and they’re still comin’.

Web pages load a _ton_ of network requests.  But people freak out about them – and the only explanation I can come up with is that they don’t understand how latency in auctions and on pages actually works.

To add an additional prebid participant, you are typically adding a single additional network request (now that everyone SHOULD be on single request architecture.  Ask your bidders this.  If they’re not running SRA, they should be, and they’re demons for not doing so).  Ask your data providers this – how many requests will you add to my auctions?  How fast do you respond on average?

If someone is adding 1 request to your ad stack, and they respond in under 500ms, the likelihood that they’re going to have a material negative impact on anything is incredibly low.  Cnn.com, that blue chip fancy website, is now over 1500 network requests in the time between when I wrote the last set of requests and writing this sentence now.  Adding a bidder who adds 1 request per prebid auction (and you should only have 1 global prebid auction per set of ads, more on this in a minute) is just not going to break the bank.  If they’re a slow responder they might consume more of your timeout and slow down your ads by a few hundred ms – but modern tech players shouldn’t do this.

Ad Speed Latency Part 3 - Widgets and Third Parties

Everything I’ve written above is true for Prebid.  You will get your ads to the page quickly, you have total control, and life is peachy when prebid is in charge of the ads on your page.  

But there’s a pesky problem in publishing – not everything runs through prebid.  This presents us with an issue.  By hardcoding tags on the page, you have tech on your pages that is no longer subject to the code review by the prebid team, and my goodness do people do horrible things.

Wrap up their own instance of prebid and run another auction on your site?  Yup, almost everyone does it.

Add blocking javascript to the page and kill CWV?  Yup, it’s happening, and like, everywhere.

Issue a metric bazillion network requests in the course of trying to hold their auctions? You betcha.

Third party ad widgets that run outside of Prebid are the number one thing that I see break publisher pages.  These always have some kind of shtick, they’re sometimes direct sold, and more often than not when something is direct sold it’s doing something invasive.  So I would ask, if you’re a publisher, ask all of the questions above of every third party widget on the page.  Who are you calling?  How many calls?  Was your code written by a high school junior in Jakarta?  And what exactly do you do to monetize your widget.

Conclusion

Web Page latency is complex, difficult to understand, difficult to measure (Google’s CWV measurement tool hates Google’s own ad library gpt.js and says it’s slowing pages down meaningfully, lol).  

• It is possible to have a fast website that runs big ol’ ad stacks.  More than possible.  

• Your prebid timeouts are probably shorter than they could be, and you could probably be making more money.

• You could probably work with more partners with minimal negative latency impact, as long as you’re intelligent and ask them the right questions.

• How your ad stack is integrated with your page is probably as important as the ad stack itself – don’t be sloppy with your JS.

• Third party widget vendors might be wreaking havoc on :gestures widely:.  Scrutinize them, make sure they’re adding real value and lots of money if they’re going to harm your pages.

• Unless you’re worried about search traffic, it’s easy to measure the impact of additional latency on Viewability and Revenue – just a/b test and scope it out.  Maybe your site can be a little slower than you think.

Middle men do indeed exist, this is ad tech.

But for all of the bellyaching, this doesn’t seem to be a situation that’s meaningfully changing outside of TTD’s OpenPath.  I’d like to attempt to explain why – and also to explore how I deeply believe that blame is often misplaced here (stop blaming publishers you monsters) and how the current situation actually isn’t as dire as it’s made out to be.  So strap on in, and let’s talk a walk down memory lane to explore how we ended up where we are.

The Before Times - Waterfalling Display

One argument I’d like to make is that the Display market is as close to a perfect market as has existed in online advertising for some time.  A unified, open-source facilitated first price auction might not be perfect, but gosh darnit it’s better than the alternatives. What alternatives you say?  Well, let’s discuss what Display used to be (and what much of video and mobile app still is, or at least was a year or two ago when I was building products for it).

Let’s start with some definitions : 

Demand - someone who represents (eventually) advertisers and wants to buy ads on a website

Supply - a publisher’s website, they have ad inventory they want to sell, or someone reselling that inventory

Ad Exchange - a system that aggregates integrations with buying tools via OpenRTB (do proprietary bidders still exist?  I guess maybe…)

DSP - a system that works with advertisers and agencies to place ads on websites by integrating with Ad Exchanges or with Publishers to algorithmically bid on that inventory

Now, in the before times, life was simple.  Publishers, the supply, would work with Ad Exchanges (basically never with DSPs, but Criteo was early here and then took steps backwards, and AppNexus kind of invented the game so they don’t really count as a pure DSP) synchronously.  Synchronously means that the ability to work with an exchange could only happen one at a time in a set order – supply would use a javascript ad tag to call a single exchange, that exchange would call their DSPs/Bidders, and an auction would occur.  If that exchange didn’t buy it, normally with an arbitrary price floor band set up (again, this was a miserable, necessary use of price floors, and I think plays some of the role in people misapprehending the “necessity” of price floors in modern ad tech), you would call the next exchange.

This was known as the waterfall – as you called an exchange, if they didn’t buy the advertising opportunity, you would call the next one.  DSPs basically _never_ received a request for the same impression at the same time because of the synchronous nature of waterfalling – impressions could only be really sold in one place at a time.

This was great for DSPs because of fragmentation.  They only had to compete with the other DSPs within a given exchange at once, and inventory was cheap.  Combine that with second price auctions, and I think this was the heyday of DSP cheap inventory buying, to be counteracted only by the massive amount of real ad fraud that existed via unpoliced domain spoofing (literally, you could type &domain=google.com at the end of an ad tag and exchanges would pass this in.  I am dead serious).  

My second company, RTK.io, was working on building a dynamic waterfalling adserver that monitored what inventory a given exchange was most likely to buy at a given price floor, and dynamically rearrange the order of the waterfall impression by impression.  The idea was so good that video adservers all started doing it, and basically still do it today, because VAST and VPAID are kind of awful and for various reasons can’t escape this waterfall. Btw — this is the original purpose of an SSP, a term now bastardized and synonymous with ad exchange — check out liftDNA or the original rubicon product, they existed to optimize and manage waterfalls.

Header Bidding Murders My Product and Frees Publishers

In the 2014s ish there were whispers about people placing javascript in the header of publisher’s pages and blocking the pageload until the auction completed.  My reaction at the time was “Yeah, Right, Like any publisher will ever let you do this.  They have to risk their entire pageload for programmatic ads? No way.”

Jokes on me, and I was running a header bidding company 8 months later.  The reason this technology was so absurdly effective was because it achieved the most important thing a publisher could do to maximize their yield – flattening a fragmented auction.  By having all exchanges compete at the same time, in the same place, you plugged a bunch of holes in what was a leaky waterfall bucket.  This is because buyers are smart – they realized if they wait to buy a given impression on a given exchange, even if it was lower down in the waterfall, they got inventory that performed just as well cheaper.  Optimizing exchange vs exchange was an incredibly important part of buy-side optimization.  But now, in a header bidding context, because a low bid had to compete with higher bids from other systems, this waiting game opportunity ended.

This brings us to the crux of our article.  Why does simultaneously sending a bid request to many different exchanges, ergo many different DSPs, for the same impression at the same time yield more revenue?  I’d like to posit a few explanations, all of which I think are potentially legitimate, and all of which justify the existence of resellers and of tons of players in the supply chain, at least from a publisher’s perspective, for as long as the buy side doesn’t want to build OpenPath (and maybe even for some time afterward)?

Why Does Duplication Work?

In a header bidding auction, other nuances of the auction dynamic transition away from waterfalls aside, a single DSP can receive many bid requests for the exact same impression (even more if exchanges are engaging in bid jamming, which some do, which is artificial bid request duplication intra-exchange as opposed to inter-exchange.  Yes this is real, and yes it works.  Literally, an exchange will make fake additional bid requests for a single real one).  And somehow this makes publishers more money.  I’ve worked with literally hundreds of publishers, building their ad stacks, building header bidding management platforms, and I can tell you that up to a point adding additional bidders with access to real DSPs will almost always make you additional money (with diminishing returns, but those kick in at a pretty high number of exchanges).  I’m sure some of you will want to fight me on this, but all you need to do is look at the ad stacks of some of the biggest managed services out there to know that they’ve a/b tested themselves into 10-15 client side bidders and even more server side.  And no, I don’t care that you removed a bidder and didn’t seem to make less money.  If you didn’t properly do it with server side session persistent A/B testing, it doesn’t count.  I will die on this hill.

Possible Explanation #1 - Distributed Computing

My first explanation for why duplication works is rooted in the nature of what a DSP is.  You see DSPs aren’t single advertisers – DSPs are massive agglomerations of hundreds of advertisers and thousands of campaigns and strategies.  This means that at any given moment, for a given impression, within a single DSP, there are probably hundreds of eligible campaigns.

All of these campaigns have lives of their own.  They have bid valuations associated with how many outcomes a given piece of inventory at a given time on a given user has driven for them, they have pacing considerations, they have all of the massively intricate workings of a modern advertising campaign.  Each of these campaigns is not particularly aware of other campaigns in the same DSP.

In addition to this, the existence, the concurrent spend, and the state of each of these campaigns has to be dynamically maintained across servers all over the world, in datacenters all over the place.  Oh and DSPs are expected to respond in under 100ms.  Not easy.

Given that this is the case, is it such a surprise that if I hit a DSP twice, almost simultaneously, with an identical bid request, two different campaigns might come back as the winner?  It could be pacing algorithms turning campaigns on and off.  It could be different servers with different priorities.  It could be any number of myriad things – but to expect consistency here is to misunderstand how DSPs actually work and the task they have laid before them.  I’m sure some DSPs have tried to architect for this, and I’d love to hear from some DSP product leads about solving this problem in the comments below – but this explanation is rooted in the notion that IF a DSP responds differently, even for an identical impression almost simultaneously, you have now created an optimization vector for publishers.  I can pound the DSP until it gives me the highest number.  This works, has always worked, and we shouldn’t be surprised.

TLDR – DSPs will never respond with one campaign over and over.  The people responsible for this are DSPs.  I can get you to pay me more money by hitting you more.

Possible Explanation #2 - Asymmetrical Filtering

The DSP and SSP world has become awash with “optimization” solutions dedicated to traffic shaping.  With all of the duplication we see in the marketplace, every major player out there has some kind of traffic shaping trying to optimize cost – often interfering with one another.  Allow me to explain.

Phase 1 - DSPs try to control cost

One of the most expensive things in the world if you use AWS is Egress.  AWS charges out the wazoo.  Egress is the cost of transmitting information between servers (engineers please don’t @me i know this is a gross simplification) – or a volume based cost for sending out packets of data.  It can be mitigated via tech stack, or by owning other parts of the pipes, but lots of people like to use AWS cuz it’s easy and lot’s of people know it.  This means that DSPs hate listening and responding to traffic that they’re probably not going to buy.  It literally costs them money, and if there’s almost no chance they’re going to buy it, they don’t want to listen to it.  Cue solution #1 – filtering – where DSPs will literally just “not listen” to a bunch of traffic that comes from SSPs.

Phase 2 - Traffic Shaping

DSPs stopping listening to traffic means they spend less money, and SSPs don’t like that.  So, in their infinite wisdom, SSPs come up with their own solution – we’re going to write machine learning that tries to predict which impressions a DSP wants to buy.  And they do.  And it (kind of?) works.  I hate this because of the functional nature of it – SSPs are by proxy building an optimization algorithm that finds traffic that converts for advertisers, without actually knowing what they’re optimizing to, which creates a lot of opportunity for noise and interference.  But this is a thing.  Everyone does it.

Phase 3 - Current State

Every SSP <> DSP interaction is shaped by a convoluted complex mess of filtering and traffic shaping, and the likelihood that a given impression slips through consistently becomes very low.  This means that different exchanges are going to “surface” different impressions to different DSPs, and there’s going to be highly mediocre rhyme or reason to it.  

In Phase 3, the solution is simple – SEND MORE!  By sending more, through as many connections as possible, you maximize the likelihood that your impressions make it through the filter/traffic shape ringer to the promised land of biddable impressions for a DSP.  

TLDR - Because DSPs control cost, they don’t listen to some of my inventory opportunities’ bid requests.  If I send more, through more places, I hit the DSP more, and I make more money.

Possible Explanation #3 - “Bid Salad Dressing”

My third explanation for why duplication works in header bidding is related to the nature of OpenRTB itself.  In a given openRTB auction, a bid request for a display ad unit looks like this (found by googling openRTB bid request) : 

{
    "id": "df472a5ca259ef79fec1567f17160ff545a80fbe",
    "at": 2,
    "tmax": 129,
    "imp": [{
        "id": "1",
        "tagid": "70821",
        "banner": {
            "w": 728,
            "h": 90,
            "battr": [9, 1, 2, 14018, 14014, 3, 5, 14, 13, 14005, 10, 14015, 8, 14019],
            "api": []
        },
        "iframebuster": []
    }],
    "site": {
        "id": "15756",
        "domain": "http://zoopla.co.uk",
        "name": "Zoopla",
        "cat": ["IAB21"],
        "page": "http://www.zoopla.co.uk/for-sale/details/31611794?search_identifier=e12d33a11efeb568768a845e59d51dfc",
        "publisher": {
            "id": "9208"
        }
    },
    "device": {
        "ua": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.3; .NET4.0C; .NET4.0E; MS-RTC LM 8)",
        "ip": "192.168.1.1",
        "language": "en",
        "devicetype": 2,
        "js": 1,
        "geo": {
            "country": "GBR",
            "region": "HNS"
        }
    },
    "user": {
        "id": "64c017a3d756e2566596cc1499294d1b602c88a3",
        "buyeruid": "7be2ed3d-a245-4045-af05-f15c9771a73e",
        "ext": {
            "sessiondepth": 5
        }
    }
}

As you can see there are a bunch of fields.  And the exchange could add more if they wanted to.  This is significant because the population of these fields is entirely up to the exchanges – it depends on the information they pull out of the browser, on the information they have about the user, on their third party services that map their user agents and geos, on their servers and their hosting, on all kinds of random stuff that will be different from exchange to exchange.  This means that for a given impression, you will have tremendous inconsistency in how that advertising impression is presented to DSPs – and from the inconsistency, you have asymmetry in DSP optimization nodes.  Now, I will say, Prebid has the opportunity to fix all of this – but until prebid is passing OpenRTB bid requests, and SSPs become dumb proxies that pass those along, we’re going to have this asymmetry, and we might still have it after that.  This means that working with more exchanges will make you more money – higher variability in optimization nodes will lead to increased outcomes.

Conclusion - RESELLERS AND DUPLICATION AREN’T A BIG DEAL AND I WANT ALL OF YOU TO STOP COMPLAINING ABOUT IT

I want to double click (lol, I hate me) on that concluding thought because it’s a significant thought that underpins how I feel about all of these things.  At the end of the day, solutions will generate more revenue for publishers that enable advertisers to optimize more effectively.  One might read this article and think that advertisers are being injured here.  I would actually argue the exact opposite.  If a tool makes a publisher more money, it’s because either that inventory is being surfaced to buyers it would not have been surfaced to before, or it is because the additional information being surfaced to buyers is leading to improved outcomes for those buyers.  This is a good thing.  We want this.  The ultimate problem for third party ad tech is how do we make it work better, not necessarily how do we make it more “efficient” from an infrastructure perspective.  

Let’s not forget, as well, that DSPs are in the driver’s seat here.  If a DSP wanted to listen to all of a publisher’s inventory in one place with no duplication, it’s incredibly easy to reach out to that publisher and agree on a time and place.  So, so, so, easy.  A college kid could easily do it with 500 publishers in 1 year.  Here’s a sample email that this theoretical college kid can use : 

“Hello publisher A, I work at DSP B.  We’d like to only buy you through one exchange.  We currently see your inventory through these exchanges - C,D,E.  Which one would you like for us to buy it through?

Thanks!

SPO done by 22 year old”

So please, please, stop blaming publishers for all this.  Stop demonizing resellers. And Advertisers/Agencies, for pete’s sake, stop buying products from people who tell you that they’re going to make things more “efficient” for you unless the thing they’re making more efficient is your Cost Per Action – because any solution that doesn’t make your media dollars drive more conversions is more wasted than the money given to any ad tech middle man. And as an aside, given modern browsing infrastructure, this isn’t even that slow anymore! It’s like 20 additional page requests — go on CNN, there are literally hundreds there already, header bidding is costing you nothing!

Also, don’t come to me with “greenwashing tech,” because I don’t buy it.  If you want campaigns to be greener, send everyone to Google, because there’s nothing cleaner than dv360 buying from GAM all hosted on GCP.  Targeting away from publishers because they have “dirty supply chains” is not only probably technologically unsound and nonsensical, but it’s inequitable – you’re literally targeting the poorest companies in the supply chain, telling them they probably have to make less money in order to be eligible for campaigns that might or might not buy their traffic anyway.  It’s messed up, and I’m not here for it.

Selective outrage has ensued. People have accused TTD of working against publishers, of shoring up its power and of trying to exert control. The Adexchanger article continued a few high level allusions to “floor prices not representing market value” and some other milquetoast quotes, but lots and lots of people see this as an “anti-publishing” move by TTD. To an extent – this is true! But not for the reasons you might suspect.

To begin with, let’s explore a brief history of floor prices in the online advertising industry.

Floor Prices 1.0 – Gaming the Vickrey Auction, not protecting value

I first encountered floor prices, and a demand for floor prices from the publishing industry, over 10 years ago. Back then the goal was simple – maximizing value for publishers by creating artificial additional demand in your second price auctions.

The logic here was in a Vickrey auction, or a second price auction, the top bidder paid the price of the next highest bid (or the price+1) :

By implementing a floor, most exchanges would allow publishers to augment the dynamic of the second price auction – and publishers (and some vendors) spend a bunch of money using ML to derive dynamic flooring systems that they didn’t think DSPs would detect (which, more often than not, they didn’t)

This was a delicate balance – floor too aggressively, and DSPs would detect your sell-side shenanigans. In fact, Criteo did – I remember some very angry emails from Criteo, who at the time were bidding SUPER aggressively to find the users they really wanted and thought would convert (in fact, a big part of their business was finding users undervalued by the market – which in my opinion is actually a huge value add for a DSP), but these floors prices screwed that strategy up.

This is the genesis of floor prices as a yield management tool in programmatic— gaming Vickrey auctions.

This use case is simple – but it’s tremendously at odds with how most people view floor prices. The uninitiated will often allude to “value,” “protecting publisher value,” or some other vague concept of publishers needing to exert some kind of subjective control over their pricing. This instinct, while intuitive, is very limited in its application, and is a remnant of a prior era where pricing was determined by vibes and not by algorithms.

In simpler terms — publisher’s used to dictate what their inventory was worth. This was because there a. weren’t very many places to buy ads, and b. measuring their efficacy was very loose (or non existent). This world doesn’t exist anymore. In digital advertising, buyers now dictate what publisher inventory is worth using complex pricing algorithms that they correlate to outcomes. This has been tremendous for the global economy, but traditional publishers haven’t intellectually caught up with the core pricing mechanism — attributed value.

And that brings us to the crux of the floor pricing debate — anachronistic publisher pricing methodologies wrongfully applied as an explanation for the efficacy of an auction gaming technique.

There are a few reasons the publisher value thesis doesn’t hold up.

1. Floor Prices to preserve value assume that when bidders don’t win, they will bid more to get your inventory. More often than not, this simply isn’t the case. There’s too much inventory out there. Ever seen a DSP delivery report? They will not notice that they stop buying on a single publisher.

2. Floor Prices to preserve value, by definition, cannot be A/B tested, and therefore cannot be proven to work. In order for anything in RTB to be statistically assessed for its value, you need to run a test and control framework — x% of impressions get the floor, x% of impressions don’t. But with a “value preserving” floor price – you can’t do that! The section of traffic you don’t run it on will make the inventory available cheaper, invalidating the value protection mechanism.

The Death of the Vickrey Auction and Floor Prices 2.0

With the advent of header bidding (and having sold one of the earliest header bidding companies), I was avowedly anti price floors for years. This is because the original use case was gone – in an environment where there are no second price auctions, there was no delta to capitalize on between the first and second price bids, therefore the insertion of any floor price could only hurt your yield as a publisher.

In principle, I still believe this, that in an intelligent world this is the case. However, there are a few use cases for floors that I overlooked, where they can provide value - DISCLAIMER - I DISLIKE ALL OF THESE, BUT DAMMIT THEY WORK

Why do floor prices make publishers more money, and why is it kind of awful

So, why do floor prices still exist?

1. Floor Prices can help publishers to extract value in situations where a bidder is converting from a CPC pricing model to a CPM pricing model.

DISCLAIMER : THIS SECTION IS WHAT’S KNOWN IN AD TECH AS “CONJECTURE” - THIS COULD BE COMPLETE WITCHCRAFT, BUT HEY IT SOUNDS GOOD

There’s one major use case for this – Google. GDN makes up a tremendous proportion of the demand on the internet, and it’s all built on performance buyers buying clicks and often backing them out into a CPA.

One of Google’s many, many clevernesses is that they have come to realize that the concept of performance translation is a place where they can realize additional revenue – depending on their expected probability of a click, the CPM that they derive can allow them to spend more or less money on inventory (and capture more upside). The way that this works is as follows :

Google Expected Click Probability – 20%

Google CPC - $5.00

Google Expected CPM – $1.00

In this scenario, Google thinks it’s willing to bid a $1.00 CPM for this inventory. However, because these determinations are entirely internal, they don’t have to bid this $1.00 CPM. More often than not, if they think they can get the inventory for $.20 – that’s what they’ll bid! And in this scenario, they’re still going to get paid their $5.00 per click, but they’re getting the inventory significantly cheaper than their expected value. This spells profits, especially in an ad network that sometimes takes 60% + margins.

In this scenario, the publisher can set price floors on Adsense and realize additional value by manipulating the translation between CPC and CPM.

2. Floor Prices are signals that can manipulate the bid stream

Okay, so I’m all for making more money from Google, that’s good and great, but surely with the migration to first price auctions floor prices are useless when it comes to adexchanges, right?

Wrong!

Floor Prices can be used in another semi-shady way to inform bid behavior. For this, I want to bring out a few floor pricing concepts, that TTD seems to be slowly getting to the bottom of but this article will lay out straight.

1. Floor Prices without signaling with enforcement

2. Floor Prices with signaling with enforcement

3. Floor Pricing with signaling without enforcement

4. Floor Price without signaling without enforcement (aka no floors)

We’ll only discuss the first three because the fourth is nonsensical.

Floor Prices without signaling with enforcement

In the world of “value preservation,” these actually make sense. I put a floor on my inventory, bidders see they don’t win, so they bid more until they start winning. Yay! Except there are literally _trillions_ of bid requests in RTB. Trillions. The likelihood that a given bidder, in a distributed computing environment, processing millions of queries per second, even thinks twice about bidding on your inventory again is infinitesimally low. It just isn’t going to happen. Please, if you’re a publisher, don’t do this.

Floor Prices with signaling with enforcement

This is a slightly better version of the above, but still not great. It’s not great for all of the value preservation reasons I outlined earlier – if you’re doing this to get more value from your inventory, you’re going to have a bad time. If it seems like it’s working, it’s not working for the reasons you think it is. Please don’t do this either.

Floor Prices with signaling without enforcement

Here is the crux of controversy. Why would a publisher ever do this? The answer is simple the perception of a floor, for some bidders, alters bidding patterns. Whether or not that floor actually exists doesn’t matter – in fact, it’s better if it doesn’t exist, because if it does the publisher will make less money by rejecting bids. Now we have the modern iteration of dynamic flooring systems – “how can I manipulate my signals in the bid request to get advertisers to pay more money for the inventory.”

This road that we’re on is a crappy one, because you can see what it has lead to – different floor prices flowing from different exchanges, or the really potentially sketchy one, different floor prices for different DSPs or seats in the same auction!

Dorothy we aren’t in Kansas anymore. Floor prices in these circumstances are so far past the concept of “value preservation” that I can’t even see it with binoculars. These floor prices are pure signal manipulations to try to get DSPs to bid more, full stop. It is what it is. Some DSPs will stop listening. Some DSPs will publisher virtue signal and keep listening. I don’t really care either way, but let’s call this what it is – an “asking price” as opposed to a “floor price” that bidders can choose to listen to or not listen to based on their data science around bid filtering and compute management.

Conclusion

Floors have nothing to do with market value, but they can work because they can manipulate bidders into bidding more. Publishers, you should use these to make more money while you can, even if I don’t like them. They’ll probably stop working one day, but might not. The one thing that matters here is that the best way to increase the value of your inventory as a publisher is to think advertiser first – how can I make sure my inventory satisfies the performance metrics of the advertiser. Because at the end of the day, that’s all that matters, not some “perception” of value.

There’s one use case that I’ve excluded from this article, which is placing an advertiser or brand specific floor because you have a PMP or Deal in place with that advertiser. I haven’t mentioned it because I hate it – if you’ve convinced a brand or an advertiser to pay more money for shit they could buy OMP, I don’t think you’ve created more value on the internet, and in fact you’re probably just extracting it from a naive buyer. Power to you, but I’m not going to pat you on the back.