Deb Roy stands before what is now a familiar image of a “social graph,” representing the ways people are connected to each other through online social networks such as Twitter. But as he stands there, a more complicated graph appears. Roy calls it the “content graph.” It represents how pieces of content are connected to each other on TV—for example, which commercial aired with which show, or how viewership of one episode compares with the audience for the next installment. Then he puts in the final piece. A thick network of connections grows between the people and the content. The human eye can’t make out any of the complexities at this point; the image looks like an enormous cocoon. But by hunting for important signals in that mass of interconnected information, Roy hopes to find the future of television.

Roy is CEO and founder of a three-year-old startup called Bluefin, an MIT spinoff based in Cambridge, Massachusetts, that has never revealed much about its technology and what it can do. But Bluefin has given Technology Review an exclusive early look at how it is trying to glean insights about the way people respond to what’s on television. Though it has not yet disclosed its customers, Bluefin hopes to sell its findings to TV networks designing programming and to companies looking to maximize their advertising spending.

The company gathers two main kinds of data. First, Bluefin has to know exactly what’s on television—not just what show is scheduled for when, but also which advertisements air and where. There’s no master list for this information, Roy says. Instead, Bluefin gets broadcast and cable television feeds from all over the United States and analyzes them frame by frame. This lets the company break shows down into smaller components (for example, individual plays in sports events) and identify the commercials aired with popular shows. In April, Bluefin analyzed more than two million minutes of TV broadcast video.

Second, Bluefin scours social networks. In April, it gathered more than three billion pieces of social-media commentary, analyzing their meaning to determine which might be references to television content. From that sea of data, the company was able to identify about 4.5 million unique authors of commentary and link 13.7 million pieces of commentary to specific TV shows and commercials. From there, the company also looked at trends in what people were saying—for example, which characters were talked about more, or which words were most commonly used to describe a show.

The magic, Roy says, is in that last bit—linking commentary to actual moments on TV. This is what allows Bluefin to determine the degree to which people were engaged by a show or a particular airing of a commercial. It’s one thing, he says, to analyze social networks for a general sense of the sentiment around a well-known brand. It’s quite another to say how people responded to a specific commercial that aired on a specific network at a certain time—and then to compare that to all 1,200 other showings of that ad. “We can tell you, quite literally, how remarkable content is,” Roy says. In other words, he argues that Bluefin can help brands figure out where their ad dollars are best spent.

For example, Bluefin analyzed recent airings of Geico commercials. The company found that the ads aired most frequently on NBC, Spike TV, and TNT, in that order. Geico itself knows that. But Bluefin went further and determined that the best responses from social media came when the ads aired on Fox, Cartoon Network, and CNBC, in that order. Breaking down audience by demographic can also yield intriguing insights. Bluefin’s analysis determined that commercials for Axe body spray—racy ads that seem designed purely to appeal to men—were more likely to be discussed online by women, and that these discussions were generally positive.

Entertainment companies and advertisers are increasingly interested in analyzing the information available on social networks. “The real-time Web is an incredibly valuable indicator of what people are watching on TV and how engaged they are with it,” says Alex Iskold, CEO and founder of Adaptive Blue, the company that makes the entertainment-oriented social network GetGlue. There’s a huge volume of this information, too. In April, his company’s users posted millions of comments, replies, check-ins, and votes and shared more than 50,000 of them with Facebook and Twitter every day. This new source of information “needs to be taken into consideration,” he says, when allocating ad dollars and assessing the popularity of shows.

Bluefin may face competition from Nielsen, the traditional giant of TV ratings. Radha Subramanyam, Nielsen’s senior vice president for media and advertising insights and analytics, says the company sees social media as “a significant area of upheaval and importance.” The key is in connecting information about television with information about social media, she adds: “It’s not that easy, and it’s not that simple. It takes a lot of sophistication.” Nielsen is working on bringing its various data sources together to go beyond simple buzz measurements and sentiment analysis, though it has not yet announced its new strategies.

Bluefin is hoping to offer value partly through scale. “What’s easy to do is take 10 or 20 very popular shows,” Roy says. What’s much harder, he adds, is to get and process information about every show on every channel. Even harder is to add data on the advertisements.

In April, the company ranked Fox’s broadcast network as the most engaging network, when judged by audience response per show airing. The rest of the top 10: ABC, MTV, CBS, TNT, BET, NBC, CW, Cartoon Network, and Bravo.