We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

AI Programs Are Learning to Exclude Some African-American Voices

Voice interfaces, chatbots, and other systems are discriminating against certain minority dialects.

All too often people make snap judgments based on how you speak. Some AI systems are also learning to be prejudiced against some dialects. And as language-based AI systems become ever more common, some minorities may automatically be discriminated against by machines, warn researchers studying the issue.

Anyone with a strong or unusual accent may know what it’s like to have trouble being understood by Siri or Alexa. This is because voice-recognition systems use natural-language technology to parse the contents of speech, and it often relies on algorithms that have been trained with example data. If there aren’t enough examples of a particular accent or vernacular, then these systems may simply fail to understand you (see “AI’s Language Problem”).

The problem may be more widespread and pernicious than most people realize. Natural-language technology now powers automated interactions with customers, through automated phone systems or chatbots. It’s used to mine public opinion on the Web and social networks, and to comb through written documents for useful information. This means that services and products built on top of language systems may already be unfairly discriminating against certain groups.

Sign up for The Download
Your daily dose of what's up in emerging technology

Brendan O’Connor, an assistant professor at the University of Massachusetts, Amherst, and one of his graduate students, Su Lin Blodgett, looked at the use of language on Twitter. Using demographic filtering, the researchers collected 59.2 million tweets with a high probability of containing African-American slang or vernacular. They then tested several natural-language processing tools on this data set to see how they would treat these statements. They found that one popular tool classified these posts as Danish with a high level of confidence.

“If you analyze Twitter for people’s opinions on a politician and you’re not even considering what African-Americans are saying or young adults are saying, that seems problematic,” O’Connor says.

The pair also tested several popular machine-learning-based APIs that analyze text for meaning and sentiment, and they found that these systems struggled, too. “If you purchase a sentiment analyzer from some company, you don’t even know what biases it has in it,” O’Connor says. “We don’t have a lot of auditing or knowledge about these things.”

He says the problem extends to any system that uses language, including search engines.

The issue of unfairness emerging from the use of AI algorithms is gaining attention in some quarters as these algorithms are used more widely. One controversial example of possible bias is a proprietary algorithm called Compass, which is used to decide whether prison inmates should be granted parole. The workings of the algorithm are unknown, but research suggests it is biased against black inmates.

Some experts, however, say that the problem may be more serious than many people know, affecting a growing number of decisions in finance, health care, and education (see “Biased Algorithms are Everywhere and No One Seems to Care”).

The UMass researchers presented their work at a workshop dedicated to exploring the issue of bias in AI. The event, Fairness and Transparency in Machine Learning, was part of a larger data-science conference this year, but it will become a stand-alone conference itself in 2018. Solon Barocas, an assistant professor at Cornell and a cofounder of the event, says the field is growing, with more and more researchers exploring the issue of bias in AI systems.

Shared Goel, an assistant professor at Stanford University who studies algorithmic fairness and public policy, says the issue is not always straightforward. He notes that it can be overly simplistic to call algorithms biased, in that they may work entirely as intended, making predictions that are accurate, and simply reflect broader social biases. “It’s better to describe what an algorithm is doing, the reason it’s doing it, and then to decide if that’s what we want it to do,” Goel says.

Learn from the humans leading the way in intelligent machines at EmTech Next. Register Today!
June 11-12, 2019
Cambridge, MA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to All Access Digital.
  • All Access Digital {! insider.prices.digital !}*

    {! insider.display.menuOptionsLabel !}

    The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    Digital magazine (6 bi-monthly issues)

    Access to entire PDF magazine archive dating back to 1899

    The Download: newsletter delivery each weekday to your inbox

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.