Skip to Content

Computational Aesthetics Algorithm Spots Beauty That Humans Overlook

Beautiful images are not always popular ones, which is where the CrowdBeauty algorithm can help, say computer scientists.

One of the depressing truths about social media is that the popularity of an image is not necessarily an indication of its quality. It’s easy to find hugely popular content of dubious quality. But it’s much harder to find unpopular content of high quality.

That’s largely because popularity is governed by a power law: a small proportion of content receives a large proportion of attention while the vast majority of content shares the rest. Take the picture-sharing website Flickr, which hosts some 200 million pictures. Of these, 166 million have five favorites or less.

That’s a large number of unpopular pictures! It’s easy to imagine that there must be many photographic gems hidden within this long tail of unpopularity. But how to reveal it?

Today, we get and answer thanks to the work of Rossano Schifanella at the University of Turin in Italy and Miriam Redi and Luca Maria Aiello at Yahoo Labs in Barcelona. These guys have taught a machine vision algorithm to recognize beauty and then allowed it to trawl through the long tail of unpopular Flickr images looking for gems that nobody has noticed. And the results are impressive.

Schifanella and co begin by crowdsourcing human opinion on the aesthetic quality of some 10,000 pictures from the Flickr database. These include both popular and unpopular images in four categories depending on whether they depict people, nature, animals or urban subjects.

Each image is rated by at least five humans according to five aesthetic categories ranging from extremely poor to exceptional quality. This process produces a ground truth data set of images that correspond to each aesthetic category.

Next, the team uses this data set to train a machine vision system called CrowdBeauty to recognize images that fall into each category. To do this, the algorithm analyzes each picture using criteria such as its contrast, brightness, color patterns, the arrangement of features within the image, and so on.

The algorithm works how these criteria relate to the eventual aesthetic rating. Schifanella and co then test the CrowdBeauty algorithm by asking it to predict the rating given to a number of photographs it hasn’t yet seen.  This it can do with surprising accuracy, particularly for animal pictures and urban pictures.

Finally, Schifanella and co let CrowdBeauty loose on a database of nine million images from Flickr that have fewer than five favorites. Its goal is to pick out beautiful images that have yet to become popular.

The results are impressive with CrowdBeauty highlighting numerous beautiful pictures.  The team compares these pictures to ones that are merely popular by crowdsourcing opinion about them. “Our method retrieves photos whose median perceived beauty score is equal to the most popular ones, and whose average is lower by only 1.5%,” they say.

(In the image above, unpopular pictures are in the left hand column, popular pictures appear in the middle column, and pictures selected by CrowdBeauty appear in the right hand column.)

That immediately raises the possibility of various applications. One idea is to use CrowdBeauty to find beautiful images that have not yet become popular. 

“As a proof-of-concept, we envision a new Flickr Beauty Explorer page that surfaces the most beautiful yet unpopular photos of the month to complement the classic Flickr Explorer that contains photos with very high social feedback,” say Schifanella and co.

In other words, the CrowdBeauty algorithm has the potential to democratize photo sharing platforms by highlighting talented individual who are otherwise under appreciated.

So there’s hope yet for all you unappreciated photographers out there.

Ref: arxiv.org/abs/1505.03358 : An Image Is Worth More Than a Thousand Favorites: Surfacing the Hidden Beauty of Flickr Pictures

Keep Reading

Most Popular

Europe's AI Act concept
Europe's AI Act concept

A quick guide to the most important AI law you’ve never heard of

The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

Uber Autonomous Vehicles parked in a lot
Uber Autonomous Vehicles parked in a lot

It will soon be easy for self-driving cars to hide in plain sight. We shouldn’t let them.

If they ever hit our roads for real, other drivers need to know exactly what they are.

supermassive black hole at center of Milky Way
supermassive black hole at center of Milky Way

This is the first image of the black hole at the center of our galaxy

The stunning image was made possible by linking eight existing radio observatories across the globe.

transplant surgery
transplant surgery

The gene-edited pig heart given to a dying patient was infected with a pig virus

The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.