Skip to Content

Where’d You Get That Cool Shirt? This Software Knows

The products that appear in online images and video could soon be automatically recognized and offered for sale.
October 8, 2012

In the arms race to deliver ever more relevant and effective online advertising, a startup called Graymatics has a formidable new weapon.

The company has developed software that can automatically identify specific products in visual media—a pair of Ray-Ban sunglasses in a music video, say, or the Banana Republic shirt your friend is wearing in a holiday photo. As a way to drive online purchases, it could be a revenue booster for both content publishers and content platforms like Facebook and YouTube, which are serving up an exploding number of images and video online.

Online videos, especially, haven’t lived up to their potential for driving revenues, even with those annoying ads before a clip. Improving video advertisements is a niche that several young startups, including Graymatics, are looking to fill.

On stage at the Demo startup conference in Santa Clara, California, last week, the company’s executives showed off software that can quickly identify items in videos and photos and then match these with the same or a similar product for sale through various online retailers and marketplaces.

As one example, the software matched the sunglasses worn by Brad Pitt and Angelina Jolie in an image accompanying a news article about the movie stars to similar pairs available on Amazon. A reader online who hovered a cursor over the object would see the tagged link for the product.

At least two other companies offer similar product-recognition services, but Graymatics’ business development director, Michael Scolari, says they require humans to input some data and can handle only still images, not video. His company’s software, which relies on computer-vision and machine-learning techniques developed by researchers in Singapore, is the first to be fully automated, he says.

The software learns by scanning product images available on the Web. It then recognizes objects in images or videos and uses algorithms to break them down into almost two dozen attributes like color, shape, and even texture. Finally, it finds the closest matches in a database of products, such as a feed of Amazon or eBay’s stock, or on specific retailers’ sites.

David Hagan, CEO of Boingo Wireless, who was on a judging panel at the conference, was impressed by the technology’s potential. But he felt its success would hinge on the accuracy of its identifications and matches.

There is reason to be optimistic. J.K. Aggarwal, a computer vision researcher at the University of Texas at Austin, says that the scope of the problem the Graymatics software is tackling is similar to the problem of facial recognition, for which there is already technology in wide use, and that product recognition is, if anything, easier, not harder, to accomplish with current technology.

Founded in 2010, Graymatics is now pitching its software to major online media companies, advertising networks, and platforms for user content, says Scolari, and it is in some preliminary trials. So far, it has signed one deal with the site Metatube, a primarily Spanish-language alternative to YouTube, the results of which will go live soon, he says.

The company isn’t focused solely on advertising. It is also developing options to help companies automatically screen for objectionable content or copyrighted material uploaded by users, among several other applications.

So far, Graymatics has raised seed investments from the office of Singapore’s prime minister and the company Citrix Systems, through its Silicon Valley accelerator program.

Keep Reading

Most Popular

This startup wants to copy you into an embryo for organ harvesting

With plans to create realistic synthetic embryos, grown in jars, Renewal Bio is on a journey to the horizon of science and ethics.

VR is as good as psychedelics at helping people reach transcendence

On key metrics, a VR experience elicited a response indistinguishable from subjects who took medium doses of LSD or magic mushrooms.

This nanoparticle could be the key to a universal covid vaccine

Ending the covid pandemic might well require a vaccine that protects against any new strains. Researchers may have found a strategy that will work.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.