Skip to Content
Artificial intelligence

An AI is playing Pictionary to figure out how the world works

Forget Go or StarCraft—guessing the phrase behind a drawing will require machines to gain some understanding of the way concepts fit together in the real world.
February 5, 2019

It might be a frivolous after-dinner game to you, but Pictionary could perhaps give AI programs a deeper understanding of the world.

AI’s lack of common sense is one of the main obstacles to the development of chatbots and voice assistants that are genuinely useful. What’s more, while AI programs can trounce the best human players of many games, including chess, Go, and (more recently) StarCraft, mastering them offers only a narrow measure of artificial intelligence. Learning to play chess, for instance, does nothing to help a computer play Sudoku.

Researchers at the Allen Institute for AI (Ai2) believe that Pictionary could push machine intelligence beyond its current limits. To that end, they have devised an online version of the game that pairs a human player with an AI program.

In case you’ve never played it before, Pictionary involves trying to draw an image that conveys a written word or phrase for your teammates to guess. This tests a person’s drawing skills but also the ability to convey complex meaning using simple concepts. Given the phrase “wedding ring,” for example, a player might try to draw the object itself but also a bride and groom or a wedding ceremony.

“You’ve got to use a lot of sophisticated reasoning,” says Ali Farhadi, the lead research on the Ai2 project. “It actually teaches common sense.”

That makes it the perfect vehicle to help teach machines. The team developed an online version of the game, called Iconary, that pairs a user with an AI bot called AllenAI. Both take turns as the artist and the guesser. Playing as artist, a user is given a phrase and then has to sketch things to convey it. The sketches are first turned into clip-art icons using computer vision; then the computer program tries to guess the phrase using a database of words and concepts and the relationship between them. If the program gets only part of the phrase, it will ask for another image to clarify.

Given the phrase “turning a page,” for instance, a player could try drawing a book, a hand, and a curved arrow. In that particular case, the AllenAI program guesses correctly after just a couple of tries.

The AI program uses a combination of AI techniques to draw and guess. Over time, by playing against enough people, AllenAI should learn from their common-sense understanding of how concepts (like “books” and “pages”) go together in everyday life, Fahadi says. It will also help the researchers explore ways for humans and machines to communicate and collaborate more effectively.

In total there are 1,200 icons, 75,000 possible phrases, and a vocabulary of 20,000 words. And there are two modes: easy and hard. The plan is to create a leaderboard to bring out people’s competitive spirit—and help accelerate the AI's learning.

Fahadi says Iconary is a better measure of AI smarts than Go or chess because it requires much broader intelligence. “Games have been a successful platform for testing AI,” he says, “but intelligence is more than pattern matching.”

Eventually, he says, it could be used as a new kind of Turing test—players could try to guess whether their teammate is a human or a machine.

Deep Dive

Artificial intelligence

chasm concept
chasm concept

Artificial intelligence is creating a new colonial world order

An MIT Technology Review series investigates how AI is enriching a powerful few by dispossessing communities that have been dispossessed before.

open sourcing language models concept
open sourcing language models concept

Meta has built a massive new language AI—and it’s giving it away for free

Facebook’s parent company is inviting researchers to pore over and pick apart the flaws in its version of GPT-3

spaceman on a horse generated by DALL-E
spaceman on a horse generated by DALL-E

This horse-riding astronaut is a milestone in AI’s journey to make sense of the world

OpenAI’s latest picture-making AI is amazing—but raises questions about what we mean by intelligence.

labor exploitation concept
labor exploitation concept

How the AI industry profits from catastrophe

As the demand for data labeling exploded, an economic catastrophe turned Venezuela into ground zero for a new model of labor exploitation.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.