The Fall television season is beginning. Both networks and fans will anxiously scan Nielsen ratings to see which shows will survive. But how do these ratings work? And could we develop a system that measures how we really watch TV?

We decided to find out the answers to these questions. We talked to Nielsen execs, a former Nielsen household member, and a bunch of consultants who watch both the TV advertising market and the next-generation methods of TV viewing such as Hulu, DVRs and new Internet-enabled televisions. Here's what we found out.


How the Nielsen system works

Nielsen Media Research puts boxes into about 25,000 households, and these boxes record the viewing habits of every member of the household.

"It's a very select few," explains Jon Gibs, Nielsen's senior vice president for analytics and insights. "The reason these people are picked is because they are representative of a broader population." The company goes to great lengths to make sure that each household chosen to be among the 25,000 Nielsen households stands for a large number of other people. "We make sure that our TV households maintain what we'd consider sort of normal behavior," and people are warned against changing their viewing habits just to make a particular show popular.


So even if you don't have a Nielsen box in your house, you should rest assured that you're being represented in the statistical sample, Gibs says.

Science fiction writer Nick Mamatas was part of a Nielsen household at one point. He lived for a year with a roommate who had a Nielsen box. And even though the Nielsen box was under the housemate's name, Mamatas was still one of the people who had to record his viewing habits since he was part of the household. Mamatas describes the fabled box:

A small black box with eight main buttons-a few buttons are assigned to either the residents or frequent guests; the others are controlled with a few other buttons to input the age and gender of other people who might be over to watch, say, the Oscars.


Nielsen households are sworn to secrecy about their status so their friends and coworkers can't hassle them into watching a particular show. Gibs adds, "We're actually audited by a third party athat make sure we obey a very, very specific set of rules, that make sure we're not monkeying around with the data."

Says Mamatas:

We weren't to tell people or accept gifts or otherwise be persuaded to watch a certain show... We did make sure to watch Buffy and Angel in syndicated repeats, and made a special effort to watch the first season of the Venture Bros, which did need just another household or two to put it over the top. And it worked.


So now you know who to thank for the Venture Bros.!

So why doesn't Nielsen just collect data from all of the millions of set-top cable TV boxes around the country? Shouldn't it be possible to measure exactly who is watching a TV program at any given time, thanks to the huge proliferation of cable boxes?


Gibs says that the cable box data wouldn't provide enough details or demographic information:

You know, I think overall, there's some very interesting stuff that's come out of set-top box data. [But] there's some very difficult challenges with using that datset. You don't know how many people are watching the TV show at a given time, [just] if the set-top box is on or off. It's difficult to know if someone is still watching the program or not. We still appreciate the granularity of the data, but accuracy is important. [Ratings data] is the measure of accounting for the TV industry... This is what dollars and cents are being traded upon.

And that's what you come back to again and again, when you talk about TV ratings — because this data is used to sell advertising, it has to be as accurate as possible. And it has to contain as much demographic information as possible.


What are the problems with the Nielsen data?

Nielsen definitely has its critics — probably including every fan of a TV show that was ever cancelled due to low ratings.


A fairly eloquent critique comes from analyst Arash Amel, research director for digital media with Screen Digest. Amel tells us:

There has never been a proven method of actually measuring how many people are watching, for how long and what [they're watching]. It's all a shorthand that was created from a panel of several thousand homes, who were supposed to be indicative of 100 million television households, and to create some kind of a common ground so you could sell advertising... It's just an immense amount of guesswork, that's carried out from a very small percentage, no matter how accurately you can say that each panel member is indicitative of a certain amount of a demographic, no matter how closely you can vet that panel, you still don't know... This is just [designed] for linear broadcasting, when you used to have three channels. Now take that, multiply it by a thousand — or how many cable channels you receive — then throw in a DVR... All of this guesswork, over the past few decades, has basically been standardized as fact. But it's not.


And then there's the DVR issue. Gibs claims that Nielsen collects data about what shows you're watching on your DVR the same way other television-viewing is recorded. But Amel insists that actually, Nielsen has no way of knowing what you're watching on a DVR. "All they can say is, [people] are watching 'the DVR channel.'" The data on DVR viewing is all extrapolated. And Nielsen assumes that people mostly watch DVR-ed programs within a week, which isn't necessarily true.

And that gets to the heart of the most important critique of Nielsen — the company isn't moving fast enough to keep up with the way in which most of us actually do watch television. Not just DVRs, but also video on demand, and viewing online via services like Hulu, iTunes and Netflix instant. Gibs admits that Nielsen isn't moving as quickly as some broadcasters and advertisers would like to take account of these trends.

One of the criticisms we frequently get is, we're perhaps not as fast as the industry would like us to be. What I think a lot of people have to realize is: We're the currency for television. While fast is good, right is better. And we're very much dedicated to doing this properly. We are the way the largest media is bought and sold. We can't move quickly for the sake of moving quickly. We know DVR is important, we know streaming is important. We are working with agencies, and we're working with Hulu quite a bit.


Gibs says that Nielsen has already made huge strides to incorporate internet viewing into its ratings. Nielsen has something called the "extended screen initiative," which he says will expand the way the company looks at DVR and streaming viewership.

Nielsen has been trying a lot of experiments, too — recently, Nielsen partnered with a company called IMI to equip a bunch of people with a small device, like a pager, that would record a few seconds of audio every now and then. That way, if the wearer was out at a bar or restaurant where a television set was on, the pager could record a sample of audio which Nielsen's systems could analyze to figure out what TV program was on in the background. But Gibs says that idea was abandoned due to "technical issues."


Will the current ad-break-supported television model even be around in 20 years?

I can't remember the last time I watched a television program and sat through all the ads. Thanks to my DVR, the idea of watching an obnoxious ad for cereal or detergent in the middle of a TV program seems just ridiculous. When I watch a program on Hulu and an ad comes on, I mute the sound and click over to another window until the ad is over. I have a feeling I'm not alone in these habits, and I suspect that a whole generation is growing up with the idea that watching commercials is for suckers.

So the real question for the future isn't, "How do we make TV ratings account for DVR and online viewers?" Rather, it's, "How do we keep monetizing television viewers?" Or really, "How can the television industry continue to convince soap companies to pay for television programs?"


I talked to a ton of industry analysts to try and make sense of these questions, and here's what I gleaned:

1) Nobody thinks the current model of broadcast television is going away anytime soon. Online video is still tiny compared to broadcast TV, and most people still watch TV live rather than via DVR.


"Just to give a rough idea, the TV broadcast advertising market last year was $60 billion. Online video advertising was half a billion [dollars], depending on who you ask," says Dan Rayburn, EVP for and a principal analyst with Frost & Sullivan. Most people still want to watch television in their living rooms on their TV sets, not on their iPads or computer screens.

According to Amel, Hulu made about $100 million in revenue last year, and it's on track to make about $210 million in revenue this year, of which about 70 percent gets kicked back to the content owners. The "big five" broadcast networks will have served about 11 billion television episodes via their own players this year, while AOL, MSN, Yahoo, YouTube and Hulu will have served another 19 billion TV episodes in 2010.

(Amel says Hulu is getting ready to roll out a subscription service called Hulu-Plus, which will give you access to the full archives of all the networks' past TV episodes, for about $10 a month. It's in beta right now.)


Still, in comparison to broadcast TV, online viewing is still tiny — but consider the fact that none of this stuff existed five years ago, Amel points out. And most of it isn't monetized that well, in comparison to broadcast TV, which brings us to the second point.

2) When you watch an episode via Hulu or other streaming sites, you're probably not making as much money for the studios as a live broadcast viewer would.


Chances are, you're not a Nielsen household. So the only way your viewership of a TV episode can even count is if you watch it online. But unfortunately, your viewership online doesn't generate as much revenue as you would if you were watching it via live TV and sitting through all the ads. (If advertisers knew you were watching, which they don't.)

Apart from anything else, online streaming video usually only comes with a few minutes of ads, versus around 17 minutes of ads on broadcast TV. But the ad rates may also be a fair bit lower.

Data about the revenue from online video advertising is hard to come by. Rayburn says YouTube gets paid about $20 CPM (per thousand impressions). CNN's website probably gets a lower rate, because it's a huge audience but not at all targeted. Meanwhile, Hulu might get a bit more, because its audience is more targeted and Hulu actually offers people a choice of ads sometimes. And the more data about exactly who you are and what demographic you belong to, the more valuable your eyeballs become — as opposed to Nielsen's huge categories like "Males aged 18-49."


But it gets a lot more complicated than that. Hulu probably pays a cut to whoever serves the ad. And in some cases, Hulu leases the content directly from the content-owner and keeps the ad revenue for itself. There may be different deals for different levels of content.

Bottom line: Nobody's going to be able to pay for an expensive television show based on online advertising revenue any time soon. The industry is still totally dependent on Nielsen numbers, because that's what the huge advertising market understands.


3) Eventually, there will be widespread adoption of internet-enabled television, which could change the way you watch TV completely and could also make Nielsen obsolete once and for all. But it's going to be slow going.

Amel says you're going to start seeing more of what he calls "over the top television," or television delivered via the Internet directly to your TV screen. Apple is already selling Apple TV for just $99, and Google is developing Google TV. Samsung is already selling Samsung Internet@TV as well. The ability to stream online programming directly to your big-screen TV is already here, and it's going to be growing. (But Apple won't be a big player in this market, Rayburn predicts — they won't sell very many of these Apple TV products.)

And in a world where the television networks can know exactly who is watching their programs and when, because the content is streaming online, then you won't need Nielsen ratings as much, says Amel.


"Broadcast TV still has a strong position," insists analyst Michael Inouye with ABI Research. Not just for live events like sporting events, but also for anything else that people are going to want to watch at the same time as everybody else for fear of Twitter spoilers.

And the biggest problem with these Internet-enabled televisions is that there are so many platforms right now. Explains Inouye, "From a developer standpoint, if you want to write an application, you have to write it for each of these individual platforms." It's possible Google TV will provide a common standard that everyone can rally behind, but it's still just on a trial basis, and it'll take some time. For the next few years, don't expect to see as many apps for internet TVs as for the iPhone or the Android, Inouye cautions.


Plus televisions have a long life-cycle, and people won't be quick to throw away a five-year-old television just to install a few new apps. Newer TVs may come with multi-core Intel processors, letting you play the newest games and install the fanciest apps — but most people won't want to throw out a TV that's just a few years old. So developers will have to take a "lowest common denominator" approach to developing for internet TVs, which in turn could slow adoption.

4) In the long run, though, everybody knows this is a huge problem.

Everybody, including the Nielsen people, know that the Nielsen ratings will have to start accounting for all the ways in which people watch television, or they'll need to be replaced.


"The consumer is clearly demonstrating that he/she wants to consume content on the go, via mobile platforms, and in the home on the TV screen, but with Internet types of functionality. So in order to stay relevant, measurement systems must keep up with these entertainment consumption trends," says analyst Laura Martin with Needham & Co.

The dream, for advertisers, is a world where the ads you see on your television set are carefully targeted to you individually, and the broadcasters know exactly who you are and what you're watching, every second of the day. But that's probably a few years away. For now, we're stuck with a system where 25,000 anonymous people decide which television programs you get to see, by pressing buttons on a plain black box.