Amazon ditched AI recruitment software because it was biased against women
The data on which the artificial-intelligence algorithm was trained created a preference for male candidates.
The news: According to a report by Reuters, Amazon began developing an automated system in 2014 to rank job seekers with one to five stars. But last year, the company scrapped the project after seeing it had developed a preference for male candidates in technical roles.
Why? The AI tool was trained on 10 years’ worth of résumés the company had received. Because tech is a male-dominated industry, the majority of those résumés came from men.
The result: The system was unintentionally trained to choose male candidates over female candidates. It would reportedly penalize résumés containing the word “women’s” or the names of certain all-women colleges. Although Amazon made changes to make these terms neutral, the company lost confidence that the program was indeed gender neutral in all other areas.
Why it matters: We can’t treat artificial intelligence as inherently unbiased. Training the systems on biased data means the algorithms also become biased. If unfair AI hiring programs like this aren’t uncovered before being implemented, they will perpetuate long-standing diversity issues in business rather than solve them.
Want to keep up with how technology is changing the workplace of the future? Sign up for our future of work newsletter, Clocking In.
Keep Reading
Most Popular
Scientists are finding signals of long covid in blood. They could lead to new treatments.
Faults in a certain part of the immune system might be at the root of some long covid cases, new research suggests.
Large language models can do jaw-dropping things. But nobody knows exactly why.
And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.
OpenAI teases an amazing new generative video model called Sora
The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.
Google’s Gemini is now in everything. Here’s how you can try it out.
Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.