A View from Emerging Technology from the arXiv
Computational Aesthetics Algorithm Spots Beauty That Humans Overlook
Beautiful images are not always popular ones, which is where the CrowdBeauty algorithm can help, say computer scientists.
One of the depressing truths about social media is that the popularity of an image is not necessarily an indication of its quality. It’s easy to find hugely popular content of dubious quality. But it’s much harder to find unpopular content of high quality.
That’s largely because popularity is governed by a power law: a small proportion of content receives a large proportion of attention while the vast majority of content shares the rest. Take the picture-sharing website Flickr, which hosts some 200 million pictures. Of these, 166 million have five favorites or less.
That’s a large number of unpopular pictures! It’s easy to imagine that there must be many photographic gems hidden within this long tail of unpopularity. But how to reveal it?
Today, we get and answer thanks to the work of Rossano Schifanella at the University of Turin in Italy and Miriam Redi and Luca Maria Aiello at Yahoo Labs in Barcelona. These guys have taught a machine vision algorithm to recognize beauty and then allowed it to trawl through the long tail of unpopular Flickr images looking for gems that nobody has noticed. And the results are impressive.
Schifanella and co begin by crowdsourcing human opinion on the aesthetic quality of some 10,000 pictures from the Flickr database. These include both popular and unpopular images in four categories depending on whether they depict people, nature, animals or urban subjects.
Each image is rated by at least five humans according to five aesthetic categories ranging from extremely poor to exceptional quality. This process produces a ground truth data set of images that correspond to each aesthetic category.
Next, the team uses this data set to train a machine vision system called CrowdBeauty to recognize images that fall into each category. To do this, the algorithm analyzes each picture using criteria such as its contrast, brightness, color patterns, the arrangement of features within the image, and so on.
The algorithm works how these criteria relate to the eventual aesthetic rating. Schifanella and co then test the CrowdBeauty algorithm by asking it to predict the rating given to a number of photographs it hasn’t yet seen. This it can do with surprising accuracy, particularly for animal pictures and urban pictures.
Finally, Schifanella and co let CrowdBeauty loose on a database of nine million images from Flickr that have fewer than five favorites. Its goal is to pick out beautiful images that have yet to become popular.
The results are impressive with CrowdBeauty highlighting numerous beautiful pictures. The team compares these pictures to ones that are merely popular by crowdsourcing opinion about them. “Our method retrieves photos whose median perceived beauty score is equal to the most popular ones, and whose average is lower by only 1.5%,” they say.
(In the image above, unpopular pictures are in the left hand column, popular pictures appear in the middle column, and pictures selected by CrowdBeauty appear in the right hand column.)
That immediately raises the possibility of various applications. One idea is to use CrowdBeauty to find beautiful images that have not yet become popular.
“As a proof-of-concept, we envision a new Flickr Beauty Explorer page that surfaces the most beautiful yet unpopular photos of the month to complement the classic Flickr Explorer that contains photos with very high social feedback,” say Schifanella and co.
In other words, the CrowdBeauty algorithm has the potential to democratize photo sharing platforms by highlighting talented individual who are otherwise under appreciated.
So there’s hope yet for all you unappreciated photographers out there.
Ref: arxiv.org/abs/1505.03358 : An Image Is Worth More Than a Thousand Favorites: Surfacing the Hidden Beauty of Flickr Pictures
AI is here. Will you lead or follow?
Join us at EmTech Digital 2019.