Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Startup’s Neural Network Can Understand Video

Software that understands what it sees in video could lead to new forms of advertising, or make video editing easier.

Enabling computers to understand video content could unlock a vast new store of information.

In recent years, researchers at companies including Google and Facebook have made impressive breakthroughs in training software to understand what’s going on in images, thanks to a technique known as deep learning. Now a startup called Clarifai is offering a service that uses deep learning to understand video.

Clarifai’s video processing software can identify objects and scenes in video and provides a timeline so you can jump to the place a certain element appears.

The company says its software can rapidly analyze video clips to recognize 10,000 different objects or types of scene. In a demo given last week at a conference on deep learning, Clarifai’s cofounder and CEO Matthew Zeiler uploaded a clip that included footage of a varied alpine landscape. The software created a timeline with graph lines summarizing when different objects or types of scene were detected. It showed exactly when “snow” and “mountains” occurred individually and together. The software can analyze video faster than a human could watch it; in the demonstration, the 3.5 minute clip was processed in just 10 seconds.

Clarifai is offering the technology as a service and expects it to be used for things like matching ads to content in online videos or developing new ways to organize video collections and edit footage.

Deep learning involves processing data through a network of simple simulated “neurons” that have been trained using example data. Clarifai’s technology originated in research at New York University, and in 2013 the company took the top five spots at the leading annual contest for software that identifies the content of images.

Most research in the field has so far focused on still images, not video. Clarifai launched a still-image recognition service last year. Another startup that uses deep learning, Dextro, launched its own video-processing service at the end of December.

Anyone can put Clarifai’s new video system to the test. On its website, you can upload a video clip of up to 10 megabytes in size for analysis. The different objects or types of scene detected might include cars, trees, or people. The software can also apply more abstract descriptive concepts, such as “fun” or “togetherness.”

Zeiler believes his technology could lead to a new approach to serving up advertising alongside videos online. “The software can identify where in the video is the optimum place for an ad to be placed,” he said.

Companies can already pay to position their ads next to videos of a particular type or ones on a particular subject. Zeiler believes that being able to automatically match ads with particular moments in videos should be even more attractive to advertisers. Clarifai’s technology could make it possible, for example, for Starbucks to make its ads appear anytime coffee was being consumed in footage.

Zeiler says he is already working with companies interested in making use of the video analysis technology. The company is also working on having the software do more. Two features under development will let the software automatically summarize what happens in a video and recognize when a particular activity has taken place, Zeiler says. 

Hear more about deep learning at EmTech MIT 2017.

Register now

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look: exclusive early access to important stories, before they’re available to anyone else

    Insider Conversations: listen in on in-depth calls between our editors and today’s thought leaders

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.