Why is it hard for algorithm designers or data scientists to account for bias and unfairness?
Take a work environment that’s hostile to women. Suppose that you define success as the fact that someone holds a job for two to three years and gets a promotion. Then your predictor—based on the historical data—will accurately predict that it’s not a good idea to hire women. What’s interesting here is that we’re not talking about historical hiring decisions. Even if the hiring decisions were totally unbiased, the reality—the real discrimination in the hostile environment—persists. It’s deeper, more structural, more ingrained and harder to overcome.
I believe the great use for machine learning and AI will be in conjunction with really knowledgeable people who know history and sociology and psychology to figure out who should be treated similarly to whom.
I’m not saying computers will never be able to do it, but I don’t see it now. How do you know when you have the right model, and when it’s capturing what really happened in society? You need to have an understanding of what you’re talking about. There’s the famous saying “All models are wrong and some are useful.”
—as told to Will Knight
Keep Reading
Most Popular

Meta has built a massive new language AI—and it’s giving it away for free
Facebook’s parent company is inviting researchers to pore over and pick apart the flaws in its version of GPT-3

The gene-edited pig heart given to a dying patient was infected with a pig virus
The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.

Saudi Arabia plans to spend $1 billion a year discovering treatments to slow aging
The oil kingdom fears that its population is aging at an accelerated rate and hopes to test drugs to reverse the problem. First up might be the diabetes drug metformin.

Yann LeCun has a bold new vision for the future of AI
One of the godfathers of deep learning pulls together old ideas to sketch out a fresh path for AI, but raises as many questions as he answers.
Stay connected

Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.