Skip to Content

Man and Machine

Despite many advances, AI still works best when paired with humans.

Engineers at Pinterest constantly create new artificial-intelligence algorithms to help its users find what they’re looking for among billions of pictures of food, products, houses, and other items. Matching search queries with relevant images is crucial to keep users coming back. But until last year, it could take days to test the effectiveness of each new algorithm.

To fine-tune its machine learning and provide better search results faster, Pinterest turned to an unexpected source: human intelligence. It hired crowdsourcing companies such as CrowdFlower to marshal people to quickly do “micro-tasks” such as labeling photos and assessing the quality of search results. In an hour, the workers collectively could test hundreds of search terms to see if results matched well enough.

For all the recent advances in AI, human beings remain more adept than machines at distinguishing, say, a tile mosaic from a similar pattern on a blanket. “It will be a long way out before machines will be able to do this,” says Pinterest data scientist Mohammad Shahangian.

Pinterest’s experience reveals a sometimes forgotten truth: AI and machine learning depend on people as much as on math. Google’s search engine and ad system use thousands of human “raters” to assess the quality of its AI-driven search results and help identify scam ads. Facebook’s facial recognition software asks people to label their photos to improve accuracy. Deep learning, a branch of AI responsible for recent breakthroughs in speech recognition, language translation, and image analysis, can require extensive human training on hand-picked data sets.

Like Pinterest, many companies hire CrowdFlower, Amazon’s Mechanical Turk, or other crowdsourcing services to clean up the data that must be fed into most AI systems in order to teach them the concepts and relationships they need to know for particular tasks. Workers perform such tasks as analyzing linguistic sentiment on Twitter and weeding out offensive user-generated photos or videos.

Pinterest’s experience reveals a sometimes forgotten truth: AI and machine learning depend on people as much as on math.

Sometimes companies set up tasks so people perform them without even realizing it. For example, key in the amount of a check you’re depositing that the automated teller machine couldn’t read, and you’re improving the bank’s system.

But even if humans currently can do some of this work more accurately than machines, it seems likely that AI should eventually be smart enough to catch up. “This is a temporary embarrassment,” says neuroscience researcher Jeff Hawkins, cofounder of the machine intelligence firm Numenta—though “temporary” could extend to years or even decades, experts say.

Some AI researchers believe the most useful model will be a hybrid system designed at the outset for machines and humans to work together as more equal partners. The nonprofit Intermountain Healthcare in Salt Lake City, for instance, is running a pilot program to support young diabetic patients starting to live on their own, when they tend to suffer gaps in care. A smartphone app provides personalized advice in real time, thanks to a cloud computing system from Austin-based CognitiveScale. Using data on factors such as a patient’s behavior and diet, it can determine what is most affecting the patient’s blood glucose level at any given moment, suggesting when to eat and even providing reviews of appropriate nearby restaurants.

Others are melding human intelligence and AI in even more intimate ways. Unlike Apple’s Siri, Facebook’s virtual assistant M uses people to help make decisions. After the AI picks three local restaurants, for example, human “trainers” might jump in to ask whether a person wants a certain kind of food or a window seat, then book the table online. The trainers, whose actions are tracked and fed back into the system, help the AI learn to do more on its own.

The ultimate dream of many AI researchers is to create machines that can think as well as people. But today human judgment and creativity remain indispensable. “Even if you have a fancy car,” notes John Giannandrea, vice president of engineering at Google, “you still have to decide where to go.”