On a special edition of “The Needle,” Derek Futterman talks to industry professionals, including a Nielsen executive, about the measurement company’s shift to a new methodology — and what it means for the industry.
It has become routine to see networks tout record viewership figures this fall. CBS on Thanksgiving delivered the largest NFL audience ever recorded by Nielsen Media Research, as 57.23 million viewers tuned in to the Kansas City Chiefs’ Thanksgiving Day clash with the Dallas Cowboys. It broke the record set mere hours earlier for the contest between the Green Bay Packers and Detroit Lions, which drew 47.67 million on Fox.
There is, however, a caveat. This is the first season of a new Nielsen methodology, “Big Data + Panel,” that combines the traditional estimates obtained from PPMs with viewership data from smart TVs and set-top boxes in more than 45 million U.S. homes. Nielsen had been experimenting with the new system for several years before it received accreditation in January, and it became the official currency in September. Jon Stainer, global GM of Nielsen Sports, told Sports Media Watch that the change allows the company to get a better picture of viewership for major sporting events.
“We’re getting data from connected televisions, which we weren’t before,” Stainer said. “That is being used to calibrate with the panel so we get the representation of the audience. That’s still a significant component of what we provide, and that’s giving us more granularity around specific demographics for the viewing and consuming content.”
Hans Schroeder, who serves as the EVP of media distribution for the NFL, has credited out-of-home measurement as being a key driver for viewership gains on the holiday. The metric was first included in television ratings starting in September 2020 to evaluate the number of people watching outside of their primary residences.
Nielsen’s out-of-home measurement started covering 100% of the country’s contiguous television population just six days before Super Bowl LIX, which subsequently averaged a record 127.71 million viewers across Fox, Tubi, Telemundo, Fox Deportes and NFL digital platforms. But the household rating — which by definition does not include out-of-home viewing — was down 4%.
Schroeder believes Nielsen had not previously had the mechanism in place to capture the full scope of viewership in the market. As the league looks for new ways to grow its fanbase both domestically and around the globe, it has been focused on ensuring that Nielsen’s calculations are more accurate.
“The NFL can bring — as does sports, but the NFL in particular — people together in a unique way, and that’s certainly the case on Thanksgiving,” Schroeder told reporters the day the viewership estimates were announced. “You see that on Super Bowl, you see that across the playoffs where that dynamic of people coming together to watch some of the biggest windows and biggest games has been there for a long time, and so I think we strongly believe when you layer that with the power of broadcast and the reach of broadcast, ‘How do you now have the ability to capture that viewership in a more accurate way?’ We think that’s what we’re seeing here and that viewership, we think, will continue to grow.”
Criticism before the opening kickoff
NFL chief data and analytics officer Paul Ballew called out Nielsen before the season began, claiming that the firm had been “systematically undercounting” viewership. Ballew cited inaccuracies in co-viewing and a lack of first-party data from league rightsholders while also remarking on the progress of competitor VideoAmp. Ed Desser, who is the president of the Desser Sports Media consulting firm, found Ballew’s comments to be “unremarkable” and questioned the downside.
“This is a business where there’s so much of a black box to it,” Desser said. “Nobody really knows how it all works, and that gives rise to kind of a natural level of cynicism, and I think that’s what those comments reflect, but I’m reluctant to read too much into it.”
The NFL feeling on Nielsen seems to have shifted amid progress in the methodology and stellar results throughout the regular-season slate. Schroeder called the OOH expansion “a big step forward” while acknowledging that the league thinks the audience can grow more as Big Data is refined. Fox Sports president of insights and analytics Mike Mulvihill, an occasional critic of Nielsen in recent years, has been conciliatory in his recent comments.
“Nielsen takes a lot of criticism in this business, but you have to give them credit for the fact that through their rollout of out-of-home measurement, the scorekeeping in this business has finally caught up to the reality,” Mulvihill told reporters following the Thanksgiving games. “That power that sports does have to bring us together and facilitate shared experience, and the numbers finally reflect the reality that’s been in place for many, many years, and it’s a welcome change.”
Mulvihill restated his confidence at a recent Nielsen event ahead of the FIFA World Cup 26, saying that OOH measurement in particular — and Big Data to a lesser extent — will be impactful in next year’s tournament. In addition, he expressed comfort with the place at which measurement has reached, comments that Stainer referenced in addressing the league’s public rebuke earlier this fall.
“[Fox] is very pleased with the advances that the business has made, and you’d have seen some of the reports over the last few weeks from the NFL and many others around the coverage that Nielsen is providing,” Stainer said. “So we feel like we’re on the right path and delighted with the work that we’ve been doing on getting accurate measurement and representation of the video viewing audience.”
Keeping the context in mind
John Kosner, the president of the Kosner Media consulting firm and a former ESPN executive, underscores the importance of context when some of his friends who view college football get excited that their school out-rated another. The principle rings true in analyzing the college basketball contests on CBS and Fox this past Thanksgiving, both of which were scheduled with the benefit of record-setting NFL lead-ins and delivered the two largest regular season college basketball audiences since 1993.
“That tactic of following NFL singleheader games is something — Ed and I are students of that — but without that context, you could think that that’s a sport that’s even more successful than it is, so the context of these different things matter,” Kosner said. “In general, if you feel you have good news, especially in the social media age, you can’t wait to share it. If you don’t have good news, you’re not sharing it, but you’re sending nasty grams to Nielsen about it.”
Since it is official Nielsen policy to compare the new viewership figures from the Big Data + Panel to the panel-only numbers from last year, outlining a genuine YoY comparison is not always possible with the available data. Because media companies are reporting the Big Data + Panel numbers, the panel-only figures are oftentimes never publicized despite still being tracked. Amazon’s Prime Video did provide Big Data numbers in its “Thursday Night Football” ratings last season, although the official figures were still panel only.
“[U]nless they were going to go back and recompute last year based upon the new methodology, I don’t know what alternative there is,” Desser said. “Now maybe there should be an asterisk because — surprise, surprise — we’re seeing most sports seeing significant increases year over year, and it’s hard to imagine that everything has gotten more popular than it was just a year ago.”
Though viewership is generally up this fall, there are some noted exceptions. Nexstar-owned CW issued a statement last month expressing “deep frustration” with how Nielsen has handled its rollout of Big Data, calling the metric “fundamentally flawed.” According to Brandon Thurston of Wrestlenomics, the Big Data viewership for the four major wrestling series is not only down from the prior panel-only averages, but also lower than the current panel-only equivalents.
As an example, “AEW Dynamite” viewership for episodes between Wednesday, Sept. 24 and Saturday, Nov. 15 declined 9% from the panel-only numbers, per Thurston. While those results may be outliers, it is possible for “Big Data + Panel” to result in lower viewership estimates than under the previous format, or a minuscule uptick.
“A lot of that could be [due] to external factors — it’s scheduling, it’s programming, it’s timing, it’s competing events and activities that are going on,” Stainer said. “I think that’s the main driver of change there of why things may go down or even may go up.”
“It is not an unreasonable conclusion that certain kinds of content that are less likely to be on, say, in a bar or restaurant, might not fare as well,” Desser added. “The notion that everything just goes up at the same rate as a result of change doesn’t seem particularly intuitive, but I can’t say that I’ve looked at what The CW beef is and come to any useful conclusion of it.”
While Big Data provides a larger sample of the audience, not all of its partners have agreed to share their first-party data. Select streaming services, including Amazon’s Prime Video, have their internal metrics combined with the Nielsen measurements to provide what is thought to be a more accurate picture. The linear networks have not yet taken the step to share their internal data, and NBCUniversal does not use Nielsen at all for any of its streaming viewership, which is tracked by Adobe Analytics. The inconsistency means that the numbers are not necessarily wholly comparable across networks.
Behind the measurement
The relationship between Nielsen and its clients has been choppy over the years, with networks expressing occasional frustration and flirting with alternatives. Nielsen faced an onslaught of criticism about its estimates four years ago amidst COVID-related changes in protocol that impacted the quality of its panel. The company had to revise more than a year of viewership figures after undercounting out-of-home viewing.
During that conflict, Nielsen lost accreditation from the Media Ratings Council for its television audience measurement (TAM) service, but it was reinstated 19 months later after the company enacted issues to correct the problems. The MRC is reportedly considering pulling accreditation of the Big Data + Panel product amid processing delays, sample representation and other concerns, according to Jasmine Sheena of Marketing Brew.
Earlier this week, the Video Advertising Bureau claimed that the new Nielsen Big Data + Panel estimates have “severe variability” when compared to panel-only numbers, and VAB president/CEO Sean Cunningham said that the change “has already gone to worst-case scenario on early returns.” In particular, he cited “deep instability on the big-volume trading demos of [ages] 25-54, 18-49 and 18-34.”
“I just wonder if the MRC stuff is just like ‘Inside Baseball,’ and it matters to people who follow this and track ratings and think about it, but it doesn’t appear to interfere with how advertisers and networks are buying and selling and valuing inventory,” Kosner said. “I’m sure Nielsen takes it seriously and they will look to address it, but I haven’t witnessed many MRC issues that actually changed the way the business operates.”
In the early years of Desser’s tenure at the National Basketball Association, he noticed that Nielsen classified the league as a syndicator and that the company was more focused on entertainment and news than sports. Combined with the fact that the NBA did not receive market-by-market data, Desser chose to move on from Nielsen.
But Nielsen has ramped up its attention towards sports programming — now the most-valuable programming in all of television — creating a sports division and curated metrics for the needs of its clients.
“I do think that they are movable, but only in sort of an extreme circumstance, and that was sort of absolutely an extreme one, but it ultimately got to a better place for sports,” Desser said. “It didn’t change any of their methodology. All it did was change what the products were and how they made them available, and it ultimately gave a lot more data to sports organizations that was actually relevant.”
Nielsen can seem like an enigma in terms of understanding its measurement and data accumulation processes, but the company is starting to shift toward more transparency with the general public. The firm puts out a monthly ‘Gauge Report’ that tracks total-day shares of television usage.
“At the end of the day, we’re the neutral, independent party here, so we consider all views from the marketplace, but we do work in partnership with our clients in trying to understand what their needs are, what their biggest priorities are [and] how they see the future of measurement,” Stainer said. “We use them as part of our design process around the product, and that’s really exciting, and I think we look to continue to do that into the future.”
As Nielsen holds a virtual monopoly on television ratings, it has filed several lawsuits against potential competitors such as TVision, HyphaMetrics, Comscore and, most recently, VideoAmp. Conversely, TVision filed an antitrust counterclaim against Nielsen, alleging that the company locks customers into contracts and files lawsuits “to intimidate, harass, deter, and drain the resources of new competitors.” TVision estimated Nielsen maintains about 90% of market share in television audience measurement services.
“There really hasn’t been anyone that had a substitutable system to Nielsen in television,” Desser said. “There have been various sources for set-top data that have been available, but they were basically a form of a sample too because you were limited to the universe that they were in.”
Moving forward, Nielsen says that is looking to continue improving its measurement program. While there is ambiguity around what that could look like, possibilities include faster turnaround times and eliminating discrepancies in streaming measurement.
“We’re focused on following the pack in terms of where the media industry is going, where the audiences are going to get their content,” Stainer said, “and so I see continued evolution of Nielsen’s measurement program in the future across various screens and platforms.”
With the upcoming Christmas and New Year’s holidays plus football playoffs, the Winter Olympics and FIFA World Cup 26, it is likely that viewership will continue to rise across the board. Just how long it takes to reach the proverbial summit is anyone’s best guess, let alone what numbers it could represent on a given platform.
Yet the box score that has impacted media rights, scheduling, distribution, production, advertising and programming strategy clearly has a defined scorekeeper with no signs of slowing down.
“We live in an environment where we’re dealing with data overload on an ongoing basis, and that requires either an ability to ignore a lot of it or to curate it in some fashion to make it useful, and people who care about it and are impacted by it will take the time to do that curation,” Desser explained, “and I think for other people, they’re just sort of awash with statistics, and it’s a matter of — it’s almost like click bait.”
Even for Fox Sports’ Mulvihill, it might be getting out of hand. “Honestly, as the guy here who maybe is always eager to talk about the ratings and the data, I feel like maybe we’re talking too much about the ratings and the data,” Mulvihill said to reporters after the NFL Thanksgiving Day success. “What we’re really celebrating here is the game and the holiday and the unique interplay of the NFL with a day of national celebration, and maybe we’re too focused on the scorekeeping when we’re not celebrating the scorekeeping.”
