Skip to Content

Biased by Design

Exclusion hurts tech companies more than they know.

A few months ago, my boyfriend convinced me to use Snapchat. He was taking goofy pictures and wanted me to join him. But when I tried, the app would map my face, and—nothing. We changed positions. We tried improving the lighting. Nothing worked. I started to get that familiar sinking feeling in my stomach, the flush of heat to my face. Growing up in Texas, I received many negative messages about my dark skin. Now here I was, 20 years later, too black for Snapchat.

Today, we recognize that the tech industry has a problem with bias—conscious and unconscious. The business case for diversity has been established—in 2015, McKinsey found that diverse teams perform better over the long term (see Innovator Under 35 Stephanie Lampkin). But we’re also finding the immediate benefits of diversity. Businesses need people who can identify a company’s blind spots—and the more homogenous the team, the bigger the blind spot.

Companies that lack diversity risk building products that exclude their customers—my experience with Snapchat being one example. Sometimes the exclusion is more blatant, as when Amazon was discovered in April to be excluding black neighborhoods from its next-day delivery service.

Exclusion isn’t the only way disenfranchised groups feel pushed out. Facebook has come under fire for suppressing posts supporting Black Lives Matter or conservative politics. People who send death threats via networks like Twitter often face only delayed or underwhelming repercussions.

Biases can become embedded in a product during any period of the development process. If the people making the products happen to come from a group that rarely experiences discrimination, those people will have a harder time predicting how bias will manifest itself. As an example, a recent study found that Airbnb guests with African-American names were 16 percent less likely to find lodging. Airbnb itself may have been shocked by that, if only because most Airbnb employees don’t encounter that kind of bias in their own lives.

Unconscious biases woven into the DNA of a firm can be passed down from team to product to user. And these blind spots have consequences. They can lead to low levels of adoption, premature market saturation, and broken products.

Simply put, diversity increases the likelihood of a tech company’s survival. Nearly half of all new businesses survive their first five years, but only 10 percent of tech startups survive past the first 18 months. An estimated 42 percent of tech-startup failures can be attributed in part to a fundamental misunderstanding of the market. A company with a diverse team that’s capable of recognizing gaps in organizational thinking is less likely to have a crippling blind spot than one with a homogenous team. An organization that sees, understands, and is able to respond to its market is more likely to thrive than one that can’t.

Y-Vonne Hutchinson is the founder of ReadySet and a founding member of the advocacy group Project Include.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.