Biased by Design
A few months ago, my boyfriend convinced me to use Snapchat. He was taking goofy pictures and wanted me to join him. But when I tried, the app would map my face, and—nothing. We changed positions. We tried improving the lighting. Nothing worked. I started to get that familiar sinking feeling in my stomach, the flush of heat to my face. Growing up in Texas, I received many negative messages about my dark skin. Now here I was, 20 years later, too black for Snapchat.
Today, we recognize that the tech industry has a problem with bias—conscious and unconscious. The business case for diversity has been established—in 2015, McKinsey found that diverse teams perform better over the long term (see Innovator Under 35 Stephanie Lampkin). But we’re also finding the immediate benefits of diversity. Businesses need people who can identify a company’s blind spots—and the more homogenous the team, the bigger the blind spot.
Companies that lack diversity risk building products that exclude their customers—my experience with Snapchat being one example. Sometimes the exclusion is more blatant, as when Amazon was discovered in April to be excluding black neighborhoods from its next-day delivery service.
Exclusion isn’t the only way disenfranchised groups feel pushed out. Facebook has come under fire for suppressing posts supporting Black Lives Matter or conservative politics. People who send death threats via networks like Twitter often face only delayed or underwhelming repercussions.
Biases can become embedded in a product during any period of the development process. If the people making the products happen to come from a group that rarely experiences discrimination, those people will have a harder time predicting how bias will manifest itself. As an example, a recent study found that Airbnb guests with African-American names were 16 percent less likely to find lodging. Airbnb itself may have been shocked by that, if only because most Airbnb employees don’t encounter that kind of bias in their own lives.
Unconscious biases woven into the DNA of a firm can be passed down from team to product to user. And these blind spots have consequences. They can lead to low levels of adoption, premature market saturation, and broken products.
Simply put, diversity increases the likelihood of a tech company’s survival. Nearly half of all new businesses survive their first five years, but only 10 percent of tech startups survive past the first 18 months. An estimated 42 percent of tech-startup failures can be attributed in part to a fundamental misunderstanding of the market. A company with a diverse team that’s capable of recognizing gaps in organizational thinking is less likely to have a crippling blind spot than one with a homogenous team. An organization that sees, understands, and is able to respond to its market is more likely to thrive than one that can’t.
Y-Vonne Hutchinson is the founder of ReadySet and a founding member of the advocacy group Project Include.
The inside story of how ChatGPT was built from the people who made it
Exclusive conversations that take us behind the scenes of a cultural phenomenon.
ChatGPT is about to revolutionize the economy. We need to decide what that looks like.
New large language models will transform many jobs. Whether they will lead to widespread prosperity or not is up to us.
Sam Altman invested $180 million into a company trying to delay death
Can anti-aging breakthroughs add 10 healthy years to the human life span? The CEO of OpenAI is paying to find out.
GPT-4 is bigger and better than ChatGPT—but OpenAI won’t say why
We got a first look at the much-anticipated big new language model from OpenAI. But this time how it works is even more deeply under wraps.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.