Skip to Content
Artificial intelligence

AI Can Re-create Video Games Just by Watching Them

September 11, 2017

Machines just took aim at video-game development—from the '80s. AIs have been able to learn to play games like Space Invaders  by watching them for a while. But now, Georgia Tech researchers have written a paper describing how AI can actually build the underlying game engine of Super Mario Bros. just by spectating.

The approach, first reported by the Verge, works by analyzing thousands of frames of game play to see what happens as everyone’s favorite mustachioed plumber moves through the game. The AI looks at what changes between one frame and the next, and tries to link cause to effect—what happens when Mario, say, touches a coin, or lands on an evil sentient mushroom (oh, okay, then: a Goomba).

Over time, the researchers say, the AI can build up rules into a rudimentary version of the game engine. The Verge’s James Vincent calls the results “glitchy, but passable” and notes that the tool is limited to simple 2-D platform games like Super Mario Bros. and Mega Man at the moment.

Speaking to the Verge, one of the researchers says that “a future version of this could [analyze] limited domains of reality.” That’s a nice idea, but as we’ve explained before, making sense of the world is one of the biggest challenges facing AI right now—and re-creating Super Mario Bros. is only a very small jump toward cracking it.


Deep Dive

Artificial intelligence

Why Meta’s latest large language model survived only three days online

Galactica was supposed to help scientists. Instead, it mindlessly spat out biased and incorrect nonsense.

DeepMind’s game-playing AI has beaten a 50-year-old record in computer science

The new version of AlphaZero discovered a faster way to do matrix multiplication, a core problem in computing that affects thousands of everyday computer tasks.

A bot that watched 70,000 hours of Minecraft could unlock AI’s next big thing

Online videos are a vast and untapped source of training data—and OpenAI says it has a new way to use it.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.