Skip to Content

Man and Machine

Despite many advances, AI still works best when paired with humans.
March 28, 2016

Engineers at Pinterest constantly create new artificial-intelligence algorithms to help its users find what they’re looking for among billions of pictures of food, products, houses, and other items. Matching search queries with relevant images is crucial to keep users coming back. But until last year, it could take days to test the effectiveness of each new algorithm.

To fine-tune its machine learning and provide better search results faster, Pinterest turned to an unexpected source: human intelligence. It hired crowdsourcing companies such as CrowdFlower to marshal people to quickly do “micro-tasks” such as labeling photos and assessing the quality of search results. In an hour, the workers collectively could test hundreds of search terms to see if results matched well enough.

For all the recent advances in AI, human beings remain more adept than machines at distinguishing, say, a tile mosaic from a similar pattern on a blanket. “It will be a long way out before machines will be able to do this,” says Pinterest data scientist Mohammad Shahangian.

Pinterest’s experience reveals a sometimes forgotten truth: AI and machine learning depend on people as much as on math. Google’s search engine and ad system use thousands of human “raters” to assess the quality of its AI-driven search results and help identify scam ads. Facebook’s facial recognition software asks people to label their photos to improve accuracy. Deep learning, a branch of AI responsible for recent breakthroughs in speech recognition, language translation, and image analysis, can require extensive human training on hand-picked data sets.

Like Pinterest, many companies hire CrowdFlower, Amazon’s Mechanical Turk, or other crowdsourcing services to clean up the data that must be fed into most AI systems in order to teach them the concepts and relationships they need to know for particular tasks. Workers perform such tasks as analyzing linguistic sentiment on Twitter and weeding out offensive user-generated photos or videos.

Pinterest’s experience reveals a sometimes forgotten truth: AI and machine learning depend on people as much as on math.

Sometimes companies set up tasks so people perform them without even realizing it. For example, key in the amount of a check you’re depositing that the automated teller machine couldn’t read, and you’re improving the bank’s system.

But even if humans currently can do some of this work more accurately than machines, it seems likely that AI should eventually be smart enough to catch up. “This is a temporary embarrassment,” says neuroscience researcher Jeff Hawkins, cofounder of the machine intelligence firm Numenta—though “temporary” could extend to years or even decades, experts say.

Some AI researchers believe the most useful model will be a hybrid system designed at the outset for machines and humans to work together as more equal partners. The nonprofit Intermountain Healthcare in Salt Lake City, for instance, is running a pilot program to support young diabetic patients starting to live on their own, when they tend to suffer gaps in care. A smartphone app provides personalized advice in real time, thanks to a cloud computing system from Austin-based CognitiveScale. Using data on factors such as a patient’s behavior and diet, it can determine what is most affecting the patient’s blood glucose level at any given moment, suggesting when to eat and even providing reviews of appropriate nearby restaurants.

Others are melding human intelligence and AI in even more intimate ways. Unlike Apple’s Siri, Facebook’s virtual assistant M uses people to help make decisions. After the AI picks three local restaurants, for example, human “trainers” might jump in to ask whether a person wants a certain kind of food or a window seat, then book the table online. The trainers, whose actions are tracked and fed back into the system, help the AI learn to do more on its own.

The ultimate dream of many AI researchers is to create machines that can think as well as people. But today human judgment and creativity remain indispensable. “Even if you have a fancy car,” notes John Giannandrea, vice president of engineering at Google, “you still have to decide where to go.”

Keep Reading

Most Popular

DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.

“This is a profound moment in the history of technology,” says Mustafa Suleyman.

What to know about this autumn’s covid vaccines

New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.

Human-plus-AI solutions mitigate security threats

With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure

Next slide, please: A brief history of the corporate presentation

From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.