Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

How an Algorithm Learned to Identify Depressed Individuals by Studying Their Instagram Photos

Your mental health is reflected in the images you choose to post on social media, say researchers who have trained a machine to spot depression on Instagram.

One of the curious things about color is that we associate it with emotions. Intuitively, we tend to link darker, grayer colors with negative moods and brighter, lighter colors with positive ones. Indeed, researchers have found that people suffering from depression prefer darker colors.

That raises the fascinating possibility that it might be possible to diagnose depression  en masse by analyzing the photos people post to social-media sites such as Instagram. But how reliable could such an approach ever be?

Today, we get an answer thanks to the work of Andrew Reece at Harvard University in Cambridge, Massachusetts, and Chris Danforth at the University of Vermont in Burlington, who have found significant correlations between the colors in photos posted to Instagram and an individual’s mental health. The link is so strong that the pair suggest that it could be used for early detection of mental illness.

The researchers began by sourcing some 500 workers from Amazon’s Mechanical Turk service who also had Instagram accounts. They asked these Turkers to complete a series of questionnaires, including a standard clinical depression survey. They then invited the Turkers to share their Instagram posts for the study.

Some 170 Turkers agreed, of whom around 70 were clinically depressed. The survey asked various additional questions about their condition, such as the original date of their diagnosis.

The Instagram downloads resulted in a database of over 40,000 photographs which the team then analyzed, again using crowdsourcing with a different set of Turkers. For each healthy user, the researchers chose the 100 most recent photographs to be rated. For depressed individuals, the researchers chose the 100 photographs posted before their diagnosis.

These raters were asked to judge how interesting, likeable, happy, and sad each photo seemed on a scale of 0 to 5.

The researchers also evaluated the photographs using objective measures such as the average hue, color saturation, contrast, and so on. This shows how vivid a picture is, for example, and whether it appears gray or  faded.

They also counted the number of faces in each image using face detection software, on the assumption that faces are a proxy for an individual’s level of social activity.  And they assessed the Instagram community’s reaction to each image by counting the number of likes and comments.

Armed with this data, the researchers used a machine-learning algorithm to spot correlations between depression and image properties. 

The researchers found that depressed individuals tend to post images that are bluer, grayer, and darker, and receive fewer likes, than those posted by heathy individuals.

Instagram offers a wide range of filters that give images a certain character and atmosphere. Depressed individuals had a clear favorite among these. “When depressed participants did employ filters, they most disproportionately favored the ‘Inkwell’ filter, which converts color photographs to black-­and-­white images,” say Reece and Danforth. By contrast, healthy individuals preferred a filter called Valencia, which lightens photographs.

The data analysis threw up some curious observations, too. Photographs that humans rated as happy or sad were only weakly correlated with depression. Depressed individuals were also more likely to post photos with faces, but these photos tended to have fewer faces per photo.

Just why this happens isn’t clear, but Reece and Danforth have an idea. They point out that depressed people tend to use more self-focused language, and that this may extend to images as well. “If so, it may be that the abundance of low-­face­-count photos posted by depressed users are, in fact, self­-portraits,” say the researchers, although they add that this “sad selfie” hypothesis remains untested.

An interesting question is how well the algorithm can identify depressed individuals using the images they post on Instagram. So the researchers set it loose on the images posted by 100 individuals and found that the algorithm correctly identified 70 percent of those who were depressed. That’s significantly better than GPs manage when asked to identify depressed individuals.

“These findings support the notion that major changes in individual psychology are transmitted in social-media use, and can be identified via computational methods,” say Reece and Danforth.  

And that will provide hope that mental illness can be accurately detected earlier, allowing for more effective intervention.

Ref: arxiv.org/abs/1608.03282 : Instagram Photos Reveal Predictive Markers of Depression

Cut off? Read unlimited articles today.

Become an Insider
Already an Insider? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Premium.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look: exclusive early access to important stories, before they’re available to anyone else

    Insider Conversations: listen in on in-depth calls between our editors and today’s thought leaders

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.