Skip to Content

Where’d You Get That Cool Shirt? This Software Knows

The products that appear in online images and video could soon be automatically recognized and offered for sale.
October 8, 2012

In the arms race to deliver ever more relevant and effective online advertising, a startup called Graymatics has a formidable new weapon.

The company has developed software that can automatically identify specific products in visual media—a pair of Ray-Ban sunglasses in a music video, say, or the Banana Republic shirt your friend is wearing in a holiday photo. As a way to drive online purchases, it could be a revenue booster for both content publishers and content platforms like Facebook and YouTube, which are serving up an exploding number of images and video online.

Online videos, especially, haven’t lived up to their potential for driving revenues, even with those annoying ads before a clip. Improving video advertisements is a niche that several young startups, including Graymatics, are looking to fill.

On stage at the Demo startup conference in Santa Clara, California, last week, the company’s executives showed off software that can quickly identify items in videos and photos and then match these with the same or a similar product for sale through various online retailers and marketplaces.

As one example, the software matched the sunglasses worn by Brad Pitt and Angelina Jolie in an image accompanying a news article about the movie stars to similar pairs available on Amazon. A reader online who hovered a cursor over the object would see the tagged link for the product.

At least two other companies offer similar product-recognition services, but Graymatics’ business development director, Michael Scolari, says they require humans to input some data and can handle only still images, not video. His company’s software, which relies on computer-vision and machine-learning techniques developed by researchers in Singapore, is the first to be fully automated, he says.

The software learns by scanning product images available on the Web. It then recognizes objects in images or videos and uses algorithms to break them down into almost two dozen attributes like color, shape, and even texture. Finally, it finds the closest matches in a database of products, such as a feed of Amazon or eBay’s stock, or on specific retailers’ sites.

David Hagan, CEO of Boingo Wireless, who was on a judging panel at the conference, was impressed by the technology’s potential. But he felt its success would hinge on the accuracy of its identifications and matches.

There is reason to be optimistic. J.K. Aggarwal, a computer vision researcher at the University of Texas at Austin, says that the scope of the problem the Graymatics software is tackling is similar to the problem of facial recognition, for which there is already technology in wide use, and that product recognition is, if anything, easier, not harder, to accomplish with current technology.

Founded in 2010, Graymatics is now pitching its software to major online media companies, advertising networks, and platforms for user content, says Scolari, and it is in some preliminary trials. So far, it has signed one deal with the site Metatube, a primarily Spanish-language alternative to YouTube, the results of which will go live soon, he says.

The company isn’t focused solely on advertising. It is also developing options to help companies automatically screen for objectionable content or copyrighted material uploaded by users, among several other applications.

So far, Graymatics has raised seed investments from the office of Singapore’s prime minister and the company Citrix Systems, through its Silicon Valley accelerator program.

Keep Reading

Most Popular

still from Embodied Intelligence video
still from Embodied Intelligence video

These weird virtual creatures evolve their bodies to solve problems

They show how intelligence and body plans are closely linked—and could unlock AI for robots.

conceptual illustration showing various women's faces being scanned
conceptual illustration showing various women's faces being scanned

A horrifying new AI app swaps women into porn videos with a click

Deepfake researchers have long feared the day this would arrive.

protein structures
protein structures

DeepMind says it will release the structure of every protein known to science

The company has already used its protein-folding AI, AlphaFold, to generate structures for the human proteome, as well as yeast, fruit flies, mice, and more.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.