Skip to Content

Where’d You Get That Cool Shirt? This Software Knows

The products that appear in online images and video could soon be automatically recognized and offered for sale.
October 8, 2012

In the arms race to deliver ever more relevant and effective online advertising, a startup called Graymatics has a formidable new weapon.

The company has developed software that can automatically identify specific products in visual media—a pair of Ray-Ban sunglasses in a music video, say, or the Banana Republic shirt your friend is wearing in a holiday photo. As a way to drive online purchases, it could be a revenue booster for both content publishers and content platforms like Facebook and YouTube, which are serving up an exploding number of images and video online.

Online videos, especially, haven’t lived up to their potential for driving revenues, even with those annoying ads before a clip. Improving video advertisements is a niche that several young startups, including Graymatics, are looking to fill.

On stage at the Demo startup conference in Santa Clara, California, last week, the company’s executives showed off software that can quickly identify items in videos and photos and then match these with the same or a similar product for sale through various online retailers and marketplaces.

As one example, the software matched the sunglasses worn by Brad Pitt and Angelina Jolie in an image accompanying a news article about the movie stars to similar pairs available on Amazon. A reader online who hovered a cursor over the object would see the tagged link for the product.

At least two other companies offer similar product-recognition services, but Graymatics’ business development director, Michael Scolari, says they require humans to input some data and can handle only still images, not video. His company’s software, which relies on computer-vision and machine-learning techniques developed by researchers in Singapore, is the first to be fully automated, he says.

The software learns by scanning product images available on the Web. It then recognizes objects in images or videos and uses algorithms to break them down into almost two dozen attributes like color, shape, and even texture. Finally, it finds the closest matches in a database of products, such as a feed of Amazon or eBay’s stock, or on specific retailers’ sites.

David Hagan, CEO of Boingo Wireless, who was on a judging panel at the conference, was impressed by the technology’s potential. But he felt its success would hinge on the accuracy of its identifications and matches.

There is reason to be optimistic. J.K. Aggarwal, a computer vision researcher at the University of Texas at Austin, says that the scope of the problem the Graymatics software is tackling is similar to the problem of facial recognition, for which there is already technology in wide use, and that product recognition is, if anything, easier, not harder, to accomplish with current technology.

Founded in 2010, Graymatics is now pitching its software to major online media companies, advertising networks, and platforms for user content, says Scolari, and it is in some preliminary trials. So far, it has signed one deal with the site Metatube, a primarily Spanish-language alternative to YouTube, the results of which will go live soon, he says.

The company isn’t focused solely on advertising. It is also developing options to help companies automatically screen for objectionable content or copyrighted material uploaded by users, among several other applications.

So far, Graymatics has raised seed investments from the office of Singapore’s prime minister and the company Citrix Systems, through its Silicon Valley accelerator program.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

Google DeepMind’s new generative model makes Super Mario–like games from scratch

Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.

How scientists traced a mysterious covid case back to six toilets

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.