Skip to Content
Artificial intelligence

An algorithm can transform your doodles into photorealistic images

March 19, 2019

In December of last year, at one of the world’s largest AI research conferences, American chipmaker Nvidia showed off an incredible new concept: using generative adversarial networks, or GANs (remember them?), to turn simple sketches into photorealistic scenes. The idea was the technology could easily render new virtual environments for video games and movies, or for training self-driving cars.

Now the company has turned those same algorithms into a new doodling app called GauGAN, named after post-Impressionist artist Paul Gauguin. It allows anyone to scribble a few lines in an MS Paint–like interface and converts it in real time into beautiful pictures with mountains, oceans, trees, and stone. It does this by associating each of the colors with specific objects, such as brown for “rock” and light blue for “sky.” Once an artist adds a paint stroke in a specific color, a deep-learning model trained on a million images fills in the texture and lighting detail. The tool also comes with different filters for changing the time of day, from sunrise to sunset, or the style of painting, from photorealistic to Impressionist.

While GauGAN currently specializes in nature scenes and is not yet publicly available, the demonstration shows how much fine-tuned control we now have when it comes to creating fake images. As much as this is an impressive (even magical) achievement, it also raises important questions about the potential these algorithms have to spread disinformation and undermine truth in the future. Fortunately, the AI research community is already at work trying to tackle this problem.

This story originally appeared in our AI newsletter The Algorithm. To have it directly delivered to your inbox, sign up here for free.

Deep Dive

Artificial intelligence

The inside story of how ChatGPT was built from the people who made it

Exclusive conversations that take us behind the scenes of a cultural phenomenon.

ChatGPT is about to revolutionize the economy. We need to decide what that looks like.

New large language models will transform many jobs. Whether they will lead to widespread prosperity or not is up to us.

GPT-4 is bigger and better than ChatGPT—but OpenAI won’t say why

We got a first look at the much-anticipated big new language model from OpenAI. But this time how it works is even more deeply under wraps.

Google just launched Bard, its answer to ChatGPT—and it wants you to make it better

Under pressure from its rivals, Google is updating the way we look for information by introducing a sidekick to its search engine.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.