Skip to Content

All the News That’s Fit for You

Microsoft’s video clipping service aims to ease your information overload.
November 16, 2001

Now that we’ve all become news junkies, we’re struck again by how much junk news is out there-especially on TV, where you may have to watch hours of car chases to find the five minutes that interest you. Microsoft has turned its heavy guns on the problem, with technology it hopes will bring order to video chaos.

At its tenth anniversary this fall, Microsoft Research unveiled prototype software that analyzes and indexes news footage. Q-Video, as Microsoft calls it, lets viewers find a CNN story as easily as a New York Times article.

“The objective is to provide an intelligent and interactive viewing experience to users, and technologies for next generation TV,” says HongJiang Zhang, who heads the Q-video effort at Microsoft’s Beijing, China research center.

Combine and Conquer

Q-video works through the artful blend of a variety of technologies, from face recognition to image analysis to natural language processing. Although researchers have developed a number of ways to search video (see Upstream: Video Searching), Zhang’s group may be the first to combine so many different technologies-an approach that promises to categorize clips more accurately.

First, the software tries to break down a video into distinct stories, using both visual and audio cues. Face and voice recognition identify news anchors and reporters. Music and so-called “keyframes”-a particular view of the studio, for example-signal story transitions. Q-video compares this information to identify a story’s beginning and end.

Next, Q-video attempts to categorize the story, by analyzing the closed caption text. If the story isn’t captioned, Q-video relies on speech recognition. The software also uses optical character recognition to extract words displayed onscreen, such as “Live at Camp David.”

Users could type-or, one day, say-“Osama bin Laden,” into a set-top box and select from a list of relevant stories. In addition, filters would gather the news a viewer cares most about: investors could monitor their portfolio companies, parents the local schools or commuters the traffic report.

Zhang cautions that Q-video is “far from being a product.” However, he says that Microsoft may soon integrate some of its features into enhanced TV offerings such as MSN TV-a Web-enabled TV service-and UltimateTV, a combination of Web-enabled satellite TV and a Tivo-like personal video recorder.

He Shoots, He Scores!

Researchers have been tackling the problem of video search for more than two decades, says Walter Bender, executive director of MIT’s Media Lab and a pioneer of video-search technology.

In the early 1980s, soon after closed captioning was introduced, Bender and colleagues at MIT built a keyword-based system to analyze the captioning of TV news stories, automatically categorize them and augment them with additional information.

At Columbia University’s Center for Telecommunications Research, Professor Shih-Fu Chang leads several projects that seek to extract and categorize highlights from sports video. One project uses scene detection and structure analysis to find highlights of tennis and baseball games. Another distinguishes wide, narrow and close-up shots in soccer game footage by analyzing how much of the frame is occupied by grass.

One problem still to be solved, Bender says, is personalization. “I don’t work or live alone,” he says. “The communities in which I interact have an impact on me.” To decide what is important to a particular user at a specific time, a personalized search may one day take into account such information as entries in an appointment calendar or the identities of recent callers and e-mail correspondents.

Even then, many people may still prefer to rely on the serendipitous discovery of new information as much as on automated news systems. After all, what once seemed an obscure event on the far side of the globe may suddenly become crucial information.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

Google DeepMind’s new generative model makes Super Mario–like games from scratch

Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.