When Nielsen's numbers for Q3 2020 came under public scrutiny, I suggested to Jack Myers and Jeff Minsky at MediaVillage that, as a service to the industry, we begin a series of "infographics" to compare the results of all the great measurers of the television audience. Jack and Jeff agreed, and we reached out to ten companies, nine of which responded positively, and three of which have already supplied data you shall see below. (Editor's note: Jeff Minsky contributed to this report.)
Insiders in media research have a saying that the industry in any country will support 1.5 television audience measurement services. In such an environment as that, which persisted until quite recently, there will inevitably be a dominant supplier in each country, a near-monopoly if not a true monopoly. The Joint Industry Committee (JIC) approach used in most industrialized nations other than the U.S. works very well to allow the boss TV rating supplier to be turned over potentially every five years. In the U.S. of course Nielsen has been the sustained ratings source leader, its numbers given the honorific term "currency."
There have been periods of time in the past when Nielsen numbers have become temporarily implausible to a plurality of its customers, and Nielsen has handily always come back from these. We observe, however, that the most recent such incident appears to have really shaken up a number of high-ranking network television executives, who are now speaking openly about having non-Nielsen alternatives at the ready from now on.
Not that anyone is eager to lose the sturdy bulwark of Nielsen, which may be with us forever, and deserves it for the work they have done in the past. And some of the tribulations Nielsen faces in adding streaming measurement, maintaining its panel during periods of pandemic or civil unrest or other crises, maintaining its quality and cohesiveness across multiple methodologies and the dilution of fusion, are capable of solution.
But how lucky we are now as an industry when we have so many great companies investing millions of dollars each in building audience measurement services based on big data? Where we are now with all these measurers is that we can look across their findings seeking consilience, the reinforcement of confidence in certain conclusions found by multiple researchers.
In the '90s the term "coopetition" began to be used to signal the opportunity for competitors to sometimes join together, and Sir Martin Sorrell may have coined the term "frenemies" to mean the same thing. His research holding company Kantar a few years ago stood up at an ARF conference to announce a willingness to work with everyone to put together data for maximum value to marketers.
It's in that spirit that we present the following comparisons of results found by Nielsen and three other companies -- 605, DISH and Samba.
The big issue with Nielsen's numbers in Q3 2020 was the drop in viewing year over year (YOY). In particular, after reading many articles about the increase in streaming, it seemed implausible to many of us that even 18-34s would be reducing their streaming in Q3 2020. MRC came out with an unprecedented announcement that Nielsen numbers were probably about 6% understated as result of the inability to follow its own panel maintenance procedures because of COVID.
What did 605, DISH and Samba find? None of them show a non-trivial reduction in viewing levels in Q3 2020, supporting the common hunch that Nielsen's numbers had been compromised by COVID and would take a little time for fixing and then return to normal.
Although DISH shows about a half of one percentage point decline, both 605 and Samba show double digit increases YOY in Q3 2020.
The picture is similar for 18-34:
Artie Bulgrin's Project Blueprint, and the WFA ANA ISBA blueprint for future audience measurement across media, both recognize the strengths and weaknesses of panels and of big data, and so recommend the use of both. As Bill has written here before, he is skeptical of the notion of conforming big data to the panel, as if the panel is truth. Big data has three advantages over peoplemeter panels:
- Sample size -- In maximizing reach, for example, panel data on duplication between programs is unstable, now that the average program has a rating under 0.1
- Passive -- No need to push buttons, which has according to Nielsen double digit errors in households with more members
- No Nonresponse Error – Although a big data set may be biased in other ways, the mechanism of opt-out results in less than 1% nonresponse, whereas the nonresponse is 70% in Nielsen according to MRC. And people who are not research cooperators are psychologically different from research cooperators, meaning that demographic reweighting doesn't solve that problem.
So, what we are potentially looking at going forward is buyers and sellers having a Nielsen and at least one non-Nielsen upon which to base all decisions.
There may be quite a few in the non-Nielsen race until market forces (supply and demand) coalesce that set of companies into a smaller set. Part of what will distinguish the winners will be the believability of their findings. Plus, of course, the methodology they report using, how it is critiqued by research buyers, how fast they respond to the need for improvements and other down-to-Earth factors.
For the users of research, it is a time to take advantage of, to learn, to study, to become better at appraising audience data and methodologies, because we may never again have as many companies trying to do the same thing.
Watch this column for updates on this series, which we call #DataUpdate.
Click the social buttons above or below to share this content with your friends and colleagues.