Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

Video Search Solves the News Segmentation Problem

The problem of indexing video news content has been solved by computer scientists at an innovative French search engine

  • November 30, 2011

Video search is one of the most difficult challenges facing search engines. Given a particular query, the task is to select from all stored videos the ones that the searcher most wants to see.

Most video search services do this without any reference to the video content. Instead, they crunch the metadata associated with each clip: the key words, popularity and dates that describe the video and must be added separately.

That works reasonably well if you’re looking for “Jesus Christ Fenton” or some other viral content. But it’s not so good if for news.

News coverage provides more or less continuous content that is divided up into sections that often have little or no connection. Search news videos for “Barack Obama in China” and you could easily be served up a video that mentions the president in one segment and China in the following one.

Now Julien Lawto at Exalead, a search engine company based in Paris, and a few pals have developed a technology that could change all this.

For some time now, these guys have been playing with a video search engine called Voxalead. The novelty of this approach is that Voxalead processes the audio content of news videos to produce an automated transcript of what’s being said. This is then searchable for key words in a straightforward way.

What’s new, however, is that they’ve found a way to automatically divide up news broadcasts into standalone passages on specific topics.

The team do this using software that analyses the transcript looking for obvious transitions form one segment to another. “The general idea of this method is to search for the best possible segmentation among all the possible ones,” they say.

In practice, this means looking for repetitions of nouns, adjectives and certain verbs and then testing the fitness of various patterns of segmentation to see which seems the best (as measured by what the company mysteriously calls ‘thematic cohesion probability’).

That produces a number of advantages. First up is the accuracy of the results. If a user searchers for “Barack Obama AND Hu Jintao”, the engine will only return passages of news that contain references to both individuals.

It also allows users to search for events, such as footage of the British embassy sacking in Tehran.

Finally, this approach allows the engine to display the results in various innovative ways. For example, it shows a timeline giving how the number of mentions of the topic has changed in the past, it also shows a geographical mashup indicating where the events have taken place or been referred too. There is also a tag cloud showing trending topics.

These guys have also taken care to ensure that the technology is scalable. “The system processes more than 150 new video and audio items each day, amounting to roughly 3.5Gb or 15 hours of new daily content, on a single 6 core server,” they say. That gives it the potential to process about 100 hours of videos per day.

There are obviously potential problems with this approach. Not least of these is the accuracy of the voice recognition software. That can be a particular problem with news coverage which often refers to rare and sometimes entirely new words. They say the design is highly scalable.

Lawto and co have ambitious plans for the future. For example, they want to augment the search results with the results from facial recognition software, presumably to be more certain about who is talking.

Clearly video search is becoming more powerful. Worth a look if you have a few spare moments.

Voxalead is available now in demonstrator form at the Exalead labs. You can find it here: http://voxaleadnews.labs.exalead.com/ However, it isn’t clear from the paper whether the new technology is running now.

Ref: arxiv.org/abs/1111.6265: A Scalable Video Search Engine Based on Audio Content Indexing and Topic Segmentation

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today
Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Print Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.