Skip to Content

Computational Aesthetics Algorithm Spots Beauty That Humans Overlook

Beautiful images are not always popular ones, which is where the CrowdBeauty algorithm can help, say computer scientists.

One of the depressing truths about social media is that the popularity of an image is not necessarily an indication of its quality. It’s easy to find hugely popular content of dubious quality. But it’s much harder to find unpopular content of high quality.

That’s largely because popularity is governed by a power law: a small proportion of content receives a large proportion of attention while the vast majority of content shares the rest. Take the picture-sharing website Flickr, which hosts some 200 million pictures. Of these, 166 million have five favorites or less.

That’s a large number of unpopular pictures! It’s easy to imagine that there must be many photographic gems hidden within this long tail of unpopularity. But how to reveal it?

Today, we get and answer thanks to the work of Rossano Schifanella at the University of Turin in Italy and Miriam Redi and Luca Maria Aiello at Yahoo Labs in Barcelona. These guys have taught a machine vision algorithm to recognize beauty and then allowed it to trawl through the long tail of unpopular Flickr images looking for gems that nobody has noticed. And the results are impressive.

Schifanella and co begin by crowdsourcing human opinion on the aesthetic quality of some 10,000 pictures from the Flickr database. These include both popular and unpopular images in four categories depending on whether they depict people, nature, animals or urban subjects.

Each image is rated by at least five humans according to five aesthetic categories ranging from extremely poor to exceptional quality. This process produces a ground truth data set of images that correspond to each aesthetic category.

Next, the team uses this data set to train a machine vision system called CrowdBeauty to recognize images that fall into each category. To do this, the algorithm analyzes each picture using criteria such as its contrast, brightness, color patterns, the arrangement of features within the image, and so on.

The algorithm works how these criteria relate to the eventual aesthetic rating. Schifanella and co then test the CrowdBeauty algorithm by asking it to predict the rating given to a number of photographs it hasn’t yet seen.  This it can do with surprising accuracy, particularly for animal pictures and urban pictures.

Finally, Schifanella and co let CrowdBeauty loose on a database of nine million images from Flickr that have fewer than five favorites. Its goal is to pick out beautiful images that have yet to become popular.

The results are impressive with CrowdBeauty highlighting numerous beautiful pictures.  The team compares these pictures to ones that are merely popular by crowdsourcing opinion about them. “Our method retrieves photos whose median perceived beauty score is equal to the most popular ones, and whose average is lower by only 1.5%,” they say.

(In the image above, unpopular pictures are in the left hand column, popular pictures appear in the middle column, and pictures selected by CrowdBeauty appear in the right hand column.)

That immediately raises the possibility of various applications. One idea is to use CrowdBeauty to find beautiful images that have not yet become popular. 

“As a proof-of-concept, we envision a new Flickr Beauty Explorer page that surfaces the most beautiful yet unpopular photos of the month to complement the classic Flickr Explorer that contains photos with very high social feedback,” say Schifanella and co.

In other words, the CrowdBeauty algorithm has the potential to democratize photo sharing platforms by highlighting talented individual who are otherwise under appreciated.

So there’s hope yet for all you unappreciated photographers out there.

Ref: arxiv.org/abs/1505.03358 : An Image Is Worth More Than a Thousand Favorites: Surfacing the Hidden Beauty of Flickr Pictures

Keep Reading

Most Popular

conceptual illustration of a heart with an arrow going in on one side and a cursor coming out on the other
conceptual illustration of a heart with an arrow going in on one side and a cursor coming out on the other

Forget dating apps: Here’s how the net’s newest matchmakers help you find love

Fed up with apps, people looking for romance are finding inspiration on Twitter, TikTok—and even email newsletters.

digital twins concept
digital twins concept

How AI could solve supply chain shortages and save Christmas

Just-in-time shipping is dead. Long live supply chains stress-tested with AI digital twins.

still from Embodied Intelligence video
still from Embodied Intelligence video

These weird virtual creatures evolve their bodies to solve problems

They show how intelligence and body plans are closely linked—and could unlock AI for robots.

computation concept
computation concept

How AI is reinventing what computers are

Three key ways artificial intelligence is changing what it means to compute.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.