Intelligent Machines

Minecraft Is a Testing Ground for Human-AI Collaboration

Microsoft turns the popular game into an “AI Olympics.”

The blockish and slightly dorky computer game Minecraft may turn out to be a great place for humans and AI to learn how to work together.

An experimental new version of the game, released by Microsoft researchers this month, can be used to train an AI to perform all sorts of tasks, from crossing bridges to building complex objects. The new platform, called Project Malmo, makes it possible for a learning algorithm to control a Minecraft character that’s normally operated by a human player. But it also provides ways for human players and AI agents to work together, and a chat window through which a person can talk with a nascent AI.

“In the long run I want to work toward AI that can be taught by any user to help them achieve their goals,” says Katja Hofmann, a researcher at Microsoft Cambridge in the U.K. who leads the project.

Hofmann, who gave a demo of the software to AI researchers at an academic conference in New York last week, says that human-AI collaboration is a key goal for the project: “We’ve built in all the capabilities that a researcher would need in order to work toward collaborative AI.”

Malmo is geared toward testing reinforcement-learning algorithms, a way of training a computer to perform a task by providing simulated rewards. It is possible, for instance, to use reinforcement learning to train an AI controlling a Minecraft character to navigate through a room filled with obstacles by providing a reward when it reaches the other side. A human Minecraft player could take part in that process by offering helpful instructions, which the AI would also learn to recognize over time.

A character in Minecraft controlled by an algorithm and another operated by a person.

Increasingly clever machine-learning algorithms have the potential to make people more productive and efficient, an idea that Microsoft’s CEO Satya Nadella has emphasized as especially important for his company. However, relatively little research emphasis has been placed on getting humans and AI to team up.

Hofmann believes the work might eventually feed into the regular game. “I could imagine that in the future you train your AI agent to take over the boring parts,” she says.

She says the development of AI that helps Minecraft players may be the first step toward computer software that augments people’s abilities in everyday settings. “What we really should be developing is something that is genuinely useful and empowers people,” Hofmann says.

AI researchers have turned to computer games as a proving ground for some of the most sophisticated machine-learning approaches around. Early last year, researchers at Google DeepMind made a splash by demonstrating an algorithm that could learn to play various 2-D Atari games to a superhuman level. Their algorithm used a combination of deep learning—a very large neural network trained to respond to an input—together with reinforcement learning. The company’s researchers have since shown video clips of algorithms learning to navigate more complex, 3-D game environments, but they have yet to publish any work on this.

Minecraft, an indie game acquired by Microsoft in 2014, became hugely popular despite its no-fuss graphics and its lack of any obvious objective. Fans of the game have used it to build an incredible array of complex structures and machines (see “The Secret to a Video-Game Phenomenon”).

Although Minecraft is relatively simple, it can provide a useful testing ground for robot algorithms (see “Minecraft Shows Robots How to Stop Dithering”).

Stefanie Tellex, a professor at Brown University who has used Minecraft for this purpose, says Malmo will be very useful because it will make it easier for researchers to compare their approaches. “The idea of a having Minecraft as a kind of AI Olympics is really great,” she says. She adds that the platform will offer an efficient way to gather large amounts of data, for example concerning human-AI interaction, that is needed for modern machine learning.

While Malmo is primarily aimed at those working on artificial intelligence, machine learning, and robotics, anyone with sufficient technical savvy can download it and experiment with an in-game AI. The software comes with several machine-learning packages and example AIs. Indeed, some of those who have so far downloaded the game are hobbyists and software engineers who do not specialize in AI.

Cut off? Read unlimited articles today.

Become an Insider
Already an Insider? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium

$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.

  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look. Exclusive early access to stories.

    Insider Conversations. Join in and ask questions as our editors talk to innovators from around the world.

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.