Why is it hard for algorithm designers or data scientists to account for bias and unfairness?
Take a work environment that’s hostile to women. Suppose that you define success as the fact that someone holds a job for two to three years and gets a promotion. Then your predictor—based on the historical data—will accurately predict that it’s not a good idea to hire women. What’s interesting here is that we’re not talking about historical hiring decisions. Even if the hiring decisions were totally unbiased, the reality—the real discrimination in the hostile environment—persists. It’s deeper, more structural, more ingrained and harder to overcome.
I believe the great use for machine learning and AI will be in conjunction with really knowledgeable people who know history and sociology and psychology to figure out who should be treated similarly to whom.
I’m not saying computers will never be able to do it, but I don’t see it now. How do you know when you have the right model, and when it’s capturing what really happened in society? You need to have an understanding of what you’re talking about. There’s the famous saying “All models are wrong and some are useful.”
—as told to Will Knight
Keep Reading
Most Popular

A quick guide to the most important AI law you’ve never heard of
The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

It will soon be easy for self-driving cars to hide in plain sight. We shouldn’t let them.
If they ever hit our roads for real, other drivers need to know exactly what they are.

This is the first image of the black hole at the center of our galaxy
The stunning image was made possible by linking eight existing radio observatories across the globe.

The gene-edited pig heart given to a dying patient was infected with a pig virus
The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.
Stay connected

Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.