Yahoo Gives the iPad the Power to Understand TV

A new iPad app from Yahoo can recognize any TV show by listening to the audio, and automatically serve up related Web content, such as news stories related to a news broadcast or play-by-play stats for a ballgame.

The app, called IntoNow, is a refreshed and improved version of technology that Yahoo acquired by buying a startup with the same name. When the company was swallowed by Yahoo in May, it was only able to share information over Facebook and Twitter about what show a user was watching. By using Yahoo technology that can recognize the meaning of text, the IntoNow version released today is much more capable, says Adam Cahan, who founded IntoNow and is now a Yahoo executive.

“This technology actually understands the ‘aboutness’ of the show,” Cahan said at a Yahoo press event. He showed the app automatically pulling up a list of Yahoo news content related to a story airing on CNN about an airplane making an emergency landing in Warsaw, Poland. The app also uses its knowledge of the show to find relevant Twitter conversations.

IntoNow identifies TV shows by comparing an audio “fingerprint” of a person’s viewing with a vast library of TV audio fingerprints stored on cloud servers. The closed captions for that moment can then be retrieved and fed into Yahoo’s technology to determine the meaning of what is currently on screen.

Cahan said people often use a mobile device while watching TV. One Yahoo study found that 30 percent of iPad use happened while the device’s owner was in front of the TV—the most popular place to use the tablet. IntoNow is intended to do automatically what people are currently doing manually—searching out and consuming extra information related to what they’re viewing.

The company’s ultimate goal is to create a new advertising revenue stream by targeting ad content through the app that is relevant to a show onscreen and a person’s viewing habits. Cahan says IntoNow did a trial promotion in which the app responded to a Pepsi commercial on TV by offering up a coupon for a free drink. “That takes you from a passive TV ad to a direct response,” he says.

Yahoo’s strategy for extracting ad revenue from TV is less confrontational than that of its competitor, Google, which last year launched computer-like devices that bring Web content to TV. Google TV has been hamstrung by the hostile response of broadcast networks worried about jeopardizing their lucrative advertising revenue. Yahoo’s approach with IntoNow sidesteps such problems by working entirely on a tablet, a device that’s growing rapidly in popularity and is beyond the control of the TV industry.

The new IntoNow app was launched at Yahoo’s Silicon Valley headquarters alongside three other new products, and against a background of questions about the company’s ability to stay relevant as Google, Facebook, and others redefine how people use the Web. Yahoo’s CEO, Carol Bartz, was fired in September; her interim replacement, Tim Morse, was in attendance but did not take the stage.

Yahoo also launched a personalized magazine for the iPad, called Livestand, that can combine content from many different sources; an improved e-mail app for tablets; and a new weather app for Android phones.

Livestand is a competitor to the popular Flipboard iPad app, which kick-started the personalized magazine category. Yahoo’s chief product officer, Blake Irving, emphasized that Livestand provides a new platform for magazine-style ads by letting them include interactivity and video. Toyota and Disney are already signed up to buy ads in this new format, while the Livestand app already includes content from over 100 media titles.