The Future of ROAS Measurement

Looking back, I feel that the report is so full of fascinating data, methodological information, and quotes that I’m not sure that the main message really came across: the need to combine all ROAS methods. Hence this column which drills down directly into this recommendation, the whys and wherefores, and the importance of not “electing one method as truth and ignoring the other methods”.

The main reason to not accept MMM as the singular truth standard is that MMM, in 99% of cases, does not control for the creative, frequency, target segments, context effects, synergies, competitive efforts, and other immensely powerful variables. The co-incidence of sales going up or down in geographic areas with different media mixes, promotions, prices is the sole algorithm. Often the media data are attributed to geographic areas based on noisy data. Often the estimated geographic media weight by type is not that different for different media types, in other words, several types may have the same geographic pattern, which makes it difficult to separate out their effects from each other.

Ignoring some of the most powerful variables in advertising and imputing ROAS based only on geographic deviances in GRP or ad dollars per capita is an approach which is unquestionably oversimplified. It is amazing that it works as well as it does. However, part of the appearance of working well is manufactured by the practice of Bayesian Priors, i.e. smoothing trends to agree with past MMM findings.

Singlesource data is not without its flaws, the major flaw being that certain sell side players continue to get away with monadic designs in which only their own media are being correlated with sales, so that only their own media can get credit for producing sales. A booster effect to that trick is to target IDs known to be brand purchasers already, so that predictable repeat sales can be made to seem like incremental sales produced by the media platform using these tricks. Many people on the buy side who ought to know better continue to be taken in by these tricks, and so the trickery continues.

At the tops of advertiser organizations, CEOs and CFOs are falling back on oversimplified approaches to correcting these mistakes, mainly by cutting media budgets. However, the budgets being cut tend to be for non-digital media. The digital media continue to grow budget, and this is not necessarily the result of scientific proof of higher ROAS; it is apparently more reflective of the lower CPMs and faster data feedback, which creates a comforting illusion of being a sophisticated appraiser of media. The ten-year FOX BHC MMM study – perhaps the largest MMM ever conducted in the world – shows that during the great dollar pilgrimage from television and other traditional media to digital, fewer than one in five national advertisers studied, increased market share. This ought to have been a wake-up call but it was not. The rest of the C suite has not yet realized that marketing can be a science, and perhaps is daunted by the need to bring a lot more science to bear in marketing.

To demonstrate how strong are the variables being left out in MMM, let’s look at just one of them, one of the subtlest: context effects. Imagine that there are two geographic markets adjacent to one another and quite similar in many ways. Imagine that the sales go up in market A but stay flat in market B. Through the MMM lens, the spurt in sales in market A must be due to the fact that media type X was at a higher GRP level there than in market B.

But what if you also had singlesource data for those markets. You might see that the households in which sales increased were barely reached at all by media type X. And you might see that by accident the psychological similarity between ads and contexts were higher in market A, and that this was the real reason why the sales went up there.

A Nielsen NCS study of RMT(Research Measurement Technologies) showed that context effects can lift incremental sales effect by +36%. The Wharton Neuroscience study presented at two ARF conferences recently by FOX and Bill Harvey Consulting shows the same thing using brainwaves: the same ads have very different effects depending upon the content the audience members see just prior to the ad. The following graphs would show all the lines superimposed over each other if there were very weak context effects; there would seem to be only a single line/curve in each of the two graphs below.

Or, there could be more of a campaign audience skew to the moveable middle in market A than in market B; that can also have strong effects: TRA showed an average increase of +28% in ROAS when the buy was skewed to Heavy Swing Purchasers (same thing as moveable middle).

Or, there could be more of a campaign audience skew to people who have the same motivations as are subconsciously communicated in the ad, in market A; a Neustar study of RMT found that this increased incremental sales ROAS +95%, and for new to brand, +115%.

These examples show that individually, the variables which are left out of MMM, can add huge amounts of variance to the data, causing misattribution of the causes to media type effects.

MMM should be combined with singlesource and RCT and branding measures. The means of integrating them is to rely upon RCT as the thing closest to a truth standard, and AI for the continuous deep learning of how much weight to give to each datum from each method. The degree of fit to the granular empirical sales data is the key signal for training the forecasting model enabling optimization before, during, and after each flight.

Debunking the Hoax of Lost Signal

The walled gardens use privacy as the rationale for the walling of their gardens. However, they are ignoring the availability of cleanroom technology by doing so. Yet the industry is not calling them on it.

The real reason for maintaining walled gardens and extending their walls is for competitive advantage. Thus, we see the two biggest walled gardens talking the advertisers into the idea of virtual IDs, even though the use of VIDs ignores the effect of campaign dispersion on reach, reducing all reach predictions to GRP level. Thus, we see third-party cookie deprecation, good for garnering brownie points for protecting privacy and maximizing advantage to the walled gardens.

In the old days, advertisers would blackball media unwilling to submit to third-party audience measurement.

Now there is street wisdom which says that loss of signal is coming and we must retreat to content targeting and MMM, but this is actually not true, for two reasons.

First, the signal we had before deprecation began was never as accurate as we thought. Truthset shows that the average of 15 major ID graphs is about half accurate. An audited accredited player, Nielsen, is creating its mega ID graph using 5-10 ID graph sources plus so far more than 20 spine-to-spine full integrations with major platforms, to overcome the inaccuracy of individual sources.

Second, although third-party cookies are going away, retail media networks and other giant players with very accurate first-party identity data are ballooning; the age of singlesource is not ending, it is just beginning, with better identity data than ever before on average.

ROAS when done right is the most powerful tool for maximizing marketing productivity in all time frames.

Using old-fashioned methods is a guarantee of leaving money on the table for competitors.

Posted at MediaVillage through the Thought Leadership self-publishing platform.

Click the social buttons to share this story with colleagues and friends.
The opinions expressed here are the author's views and do not necessarily represent the views of