Data-Mining 100 Million Instagram Photos Reveals Global Clothing Patterns
The millions of photos uploaded to social media are a massive untapped resource for studying humanity. But machine learning is beginning to tap this mother lode.
“Imagine a future anthropologist with access to trillions of photos of people—taken over centuries and across the world—and equipped with effective tools for analyzing these photos to derive insights. What kinds of new questions can be answered?”
This is the dream that has inspired Kevin Matzen, Kavita Bala, and Noah Snavely at Cornell University in Ithaca, New York.
Advertisement
Their thinking is that the millions of photos uploaded each day to social media provide a fascinating window into the cultural, social, and economic factors that shape societies around the world. With powerful enough machine intelligence, they say, it ought to be possible to mine this mother lode of data for deep insights into our civilization.
This story is only available to subscribers.
Don’t settle for half the story.
Get paywall-free access to technology news for the here and now.
The algorithm reveals how clothing styles change over time
As luck would have it, this kind of machine intelligence is currently emerging at breakneck speed. And Matzen and co have put it to work studying 100 million photos posted on Instagram.
The question these guys specifically want to answer was how clothing styles vary around the world, a cultural phenomenon that is otherwise difficult to study on this scale.
For example, their approach can tackle questions such as how the frequency of scarf use in the U.S. is changing over time, what styles are most specific to particular regions or cities, and, conversely, which styles are popular across the world.
To find out, Matzen and co turned to Instagram, which allowed them to download images within five kilometers of a specific location and within five days of a specific date.
The team then identified 44 cities to study and downloaded a total of 100 million images from these locations in five-day windows between June 2013 and June 2016.
They used a standard face recognition program to filter out all the pictures that did not contain a face, and they also filtered for a visible torso, leaving a set of 15 million photos of people showing the upper half of their body, along with their location and the date.
Next, they trained a machine-learning algorithm to recognize various types of clothing and accessories in images. For example, the algorithm learned to recognize whether people were wearing a jacket, a scarf, a necktie, glasses, a hat, and so on. The algorithm could also recognize colors, neckline styles, and sleeve length; clothing categories such as T-shirt, dress, or tank top; and clothing patterns, such as solid, striped, plaid, and so on.
Advertisement
Finally, they let the machine lose on the 15 million photos in their data set and then used another algorithm to search for clusters of images with similar visual themes and track how these varied across time and from one location to another.
The results make for interesting reading. The clustering algorithm found some 400 different visual themes, such as people wearing white T-shirts and glasses, or wearing red V-neck tops or black dresses, or not wearing tops at all!
Matzen and co can then study how these visual themes vary by time and place. They found, for example, that certain colors vary periodically, with black and brown being more common in winter and white and blue more common in summer.
Other colors show different patterns. For example, the popularity of red is dropping. And although it is much less periodic than black or white, it does become suddenly popular from time to time. Matzen and co point to small spikes in popularity near the end of October and December: in other words, at Halloween and Christmas. “What stood out were a large assortment of Santa hats as well as an unexpected assortment of red Halloween costumes with red hats or hoods,” they say.
They also found a sudden increase in popularity of yellow shirts in Colombia and Brazil during the June/July 2014 football World Cup—both countries’ football teams wear yellow.
They also noted various geographical trends. “Countries further north tend to feature more jackets,” they say, presumably because it is colder.
Hat wearing is also more popular in colder countries. But curiously, Oman in the Middle East turns out to be one of the hat-wearing capitals of the world. “In particular, the kuma and massar are popular in Oman, as they are an important element of the men’s national dress,” say Matzen and co.
Some clothes are unique to particular places: the gele, a Nigerian head-tie, is very distinctive of Lagos. And other styles are common around the world and throughout the year, including blue collared shirts, plaid shirts, and black T-shirts.
Advertisement
That’s interesting work that reveals the potential for machine learning to tease apart the cultural fabric of our society.
Of course, this approach is not perfect. The algorithm did not learn to distinguish between sunglasses and prescription glasses, which play different roles in society. The images are unlikely to be representative of society as a whole, since Instagram users are heavily skewed toward a younger demographic. And the technique only looks at the upper body, since the legs are often cropped in online images.
But there is significant potential to correct these shortcomings in future work and to go further. An ongoing challenge in machine vision is to work out whether people are standing or sitting or what they are doing in general. It would also be possible to combine this data set with others, such as weather and temperature data.
As Matzen and co conclude: “The combination of big data, machine learning, computer vision, and automated analysis algorithms would make for a very powerful analysis tool more broadly in visual discovery of fashion and many other areas.”
Clearly, we don’t need to wait for the anthropologists of the future.
Ref: arxiv.org/abs/1706.01869 :StreetStyle: Exploring World-Wide Clothing Styles from Millions of Photos
