We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

Beauty Now in the Eye of the Algorithm

New image recognition technology judges photographic aesthetics.

New technology from Xerox can sort photos not just by their content but also according to their aesthetic qualities, such as which portraits are close-in and well-lit, or which wildlife shots are least cluttered.

Pleasing portrait: The Xerox algorithm deemed the portrait on the left high quality in part because of its lighting and simple black background, while judging the one on the right low quality because of its washed-out lighting.

Still in the prototype stage, the technology could eventually help with tasks like choosing which of hundreds of digital photos taken on a family vacation should appear in a photo album. It could help stock agencies sort photos by their characteristics, and it could be deployed inside a camera to help people delete lower-quality scenes quickly, saving on storage space and hassle.

“What they show is that now you don’t need a human to select images that are going to be judged beautiful,” says Aude Oliva, an associate professor of brain and cognitive sciences at MIT, who also works on image recognition. “You can run the algorithm, and it will give a good estimate.”

The technology—developed at the Xerox Research Center Europe in Grenoble, France—is slated for beta testing with Xerox corporate partners next year, says Craig Saunders, manager of the computer vision research group there. These partners include graphic design firms, online photo-book companies, and stock agencies, all of which might want new ways to sort and find photos.

The Xerox system learns about quality photography by studying photos that had previously been chosen for public display in online photo albums, such as public ones shown on Facebook, or photos tagged as high quality on Flickr. Then it notes common characteristics of these photos.

Not surprisingly, these characteristics often correspond to what experts already understand about good photographs. The best portraits of people, for example, have indirect lighting and blurry or monochromatic backgrounds that help keep the focus on the person. Good beach photos often include silky-looking waves, a trick achieved through slow shutter speeds. And many kinds of photos are appealing because they follow the “rule of threes,” with subjects divided among three zones in the photo. “We try to learn what it is about these features that makes photos ‘good,’” says Saunders. (Examples and demonstrations can be found here.)

Facing the algorithm: Other portraits on which the Xerox system passed judgment include the high-quality ones on the left and the low-quality ones on the right.

The technique builds on a larger body of research, conducted at Xerox and many other labs, that strives to improve image recognition by breaking down photos into what researchers call a visual vocabulary—corners and edges that might define buildings, round shapes that might be wheels, regions of green that might indicate landscapes, and many more such elements (and combinations thereof). The resulting technologies build up knowledge about what pieces correspond to certain types of images by examining Internet-based photos that are already tagged with text identifying what’s in them.

Many research groups, including the one at Xerox, are working on improving not only the accuracy of these methods but also their computational efficiency. For example, Xerox announced recently that it has developed a system capable of finding images that have similar characteristics. It can sort through five million images in less than a second.

Xerox plans to launch this tool next year as a cloud-based service that could be used to refine searches in large image repositories like stock photo agencies. The company also released a related Facebook app, called Catepix, that examines your Facebook photos, categorizes them (portrait, landscape, etc.), and tells you what they say about your personality.

Unfortunately, I have posted only three pictures on Facebook, so the app failed to tell me much of anything. But it did put up a post under my name declaring that I was a portrait kind of guy.

Keep up with the latest in intelligent machines at EmTech Digital.

The Countdown has begun.
March 25-26, 2019
San Francisco, CA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.