Skip to Content
Artificial intelligence

Memers are making deepfakes, and things are getting weird

The rapidly increasing accessibility of the technology raises new concerns about its abuse.
August 28, 2020
Bottom right: it me.Ms Tech | NeurIPS (training set); Hao (courtesy)

Grace Windheim had heard of deepfakes before. But she had never considered how to make one. It was a viral meme using the technology that led her to research the possibility—and discover that it was super easy and completely free.

Within a day, she had created a step-by-step YouTube tutorial to walk others through the process. “Making one of these deepfakes and overlaying audio is not as complicated as you may think,” she says in the video, published on August 4. It has since been viewed over 360,000 times.

Windheim is part of a new group of online creators who are toying with deepfakes as the technology grows increasingly accessible and seeps into internet culture. The phenomenon is not surprising; media manipulation tools have often gained traction through play and parody. But it also raises fresh concerns about its potential for abuse.

Deepfakes have already been used to harass women by nonconsensually swapping their faces into porn videos. Scholars also worry about their ability to disrupt elections. While the deepfakes created for memes are still obviously fake and relatively harmless, they may not stay that way for long.

“There’s a fine line between using deepfakes for entertainment and memes, and using them for harm,” Windheim says. “In this tutorial, I’m saying, ‘This is how you make this particular deepfake.’ But the scary thing about the script is it can just be applied to make any type of deepfake you want.”

“I’ve Been a Fool”

Windheim, a recent college grad, works as a content creator at the San Francisco–based startup Kapwing. The company, which got its start as a meme maker, offers a free suite of browser-based video-editing software tools. As part of her job, Windheim runs the YouTube channel and produces content marketing videos to show off the products’ capabilities.

In early August, she came across a particularly viral search term on Google Trends. Three of the five top queries were asking about a “Baka Mitai deepfake meme.” “I almost never see a query come up that frequently,” she says.

The meme, as it turns out, was based on a video of a YouTuber lip-synching to a Japanese video-game song called “Baka Mitai” (translation: “I’ve Been a Fool”). Various internet users had used the video to create crappy deepfakes of everyone from Barack Obama to Thanos singing the song. Despite its popularity, however, Windheim found that little had been written about how to actually make it. She saw an opportunity.

The particular deepfake algorithm that people were using comes from a 2019 research paper presented at NeurIPS, the largest annual AI research conference. Unlike other, more complex algorithms, it allows a user to take any video of a person’s face and use it to animate a photo of someone else’s face with only a few lines of code.

Windheim found the open-source algorithm in a YouTube tutorial and ported it into a Google Colab notebook, a free service for running code in the cloud. After a few tries, aided by the skills she’d picked up in the occasional coding class in college, she got the script to spit out a deepfake video. She then synched the song to the video with Kapwing’s tools, creating a new version of the meme.

Since she posted her tutorial on Kapwing’s YouTube channel, a number of other YouTubers have also made tutorials using the same copy-and-pasted algorithm. The difference: many of them are teaching their audience how to make any kind of deepfake meme. One even teaches people how to make them on mobile.

These memes are now appearing everywhere on social media: on Twitter, Instagram, and especially TikTok. The platform’s short videos, which often feature snappy choreography to catchy music, are particularly conducive to being deepfaked to mesmerizing effect. The #deepfake hashtag in the app has already racked up more than 120 million views.

@lemonjezus
@helltaker03

There’s a telltale wonkiness to the faces in the videos made with this algorithm, which makes its handiwork easy to recognize; that is part of the deepfakes’ humor. These imperfections—and the surrealist quality of the memes—will keep them from being confused for reality. At the moment, more hyper-realistic deepfakes are far more technically challenging and computationally expensive to create.

But at the rate that the technology is advancing, easy-to-make deepfakes that are nearly indistinguishable from reality are likely around the corner. Some companies like Chinese tech giant Tencent, owner of WeChat, have publicly announced their intentions to invest more resources into advancing the state of the art for commercial applications.

Deepfakes are not inherently bad. The technology has already been used by artists, educators, and others as a powerful new tool for creative expression. In February, for example, Time magazine used deepfakes to re-create the experience of Martin Luther King Jr. delivering his “I Have a Dream” speech in virtual reality. Ultimately,  regulators need to define what is appropriate use and what could lead to harm.

For now, Windheim is relying on her own judgment to make that call. Before posting her video, she read up on the implications of deepfakes and had a conversation with her colleagues. “We’re never intending our products to help users spread misinformation,” she says, “so we just wanted to sanity-check ourselves.”

In the end, they decided on some ground rules: they would focus their tutorials on making specific memes, never on creating deepfakes outside of that context. As long as it’s entertainment and within meme culture, she says, “we’re in the clear zone.”

Deep Dive

Artificial intelligence

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

Providing the right products at the right time with machine learning

Amid shifting customer needs, CPG enterprises look to machine learning to bolster their data strategy, says global head of MLOps and platforms at Kraft Heinz Company, Jorge Balestra.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.