Skip to Content
Artificial intelligence

AI image generator Midjourney blocks porn by banning words about the human reproductive system

Midjourney says it’s a temporary measure to stop people from using its system to create shocking or gory images.

February 24, 2023
bejeweled ovary
A bejeweled ovary generated using MidjourneyJulia Rockwell

The popular AI image generator Midjourney bans a wide range of words about the human reproductive system from being used as prompts, MIT Technology Review has discovered. 

If someone types “placenta,” “fallopian tubes,” “mammary glands,” “sperm,” “uterine,” “urethra,” “cervix,” “hymen,” or “vulva” into Midjourney, the system flags the word as a banned prompt and doesn’t let it be used. Sometimes, users who tried one of these prompts are blocked for a limited time for trying to generate banned content. Other words relating to human biology, such as “liver” and “kidney,” are allowed. 

Midjourney’s founder, David Holz, says it’s banning these words as a stopgap measure to prevent people from generating shocking or gory  content while the company “improves things on the AI side.” Holz says moderators watch how words are being used and what kinds of images are being generated, and adjust the bans periodically. The firm has a community guidelines page that lists the type of content it blocks in this way, including sexual imagery, gore, and even the 🍑 emoji, which is often used as a symbol for the buttocks.

AI models such as Midjourney, DALL-E 2, and Stable Diffusion are trained on billions of images that have been scraped from the internet. Research by a team at the University of Washington has found that such models learn biases that sexually objectify women, which are then reflected in the images they produce. The massive size of the data set makes it almost impossible to remove unwanted images, such as those of a sexual or violent nature, or those that could produce biased outcomes. The more often something appears in the data set, the stronger the connection the AI model makes, which means it is more likely to appear in images the model generates.  

Midjourney’s word bans are a piecemeal attempt to address this problem. Some terms relating to the male reproductive system, such as “sperm” and “testicles,” are blocked too, but the list of banned words seems to skew predominantly female. 

The prompt ban was first spotted by Julia Rockwell, a clinical data analyst at Datafy Clinical, and her friend Madeline Keenen, a cell biologist at the University of North Carolina at Chapel Hill. Rockwell used Midjourney to try to generate a fun image of the placenta for Keenen, who studies them. To her surprise, Rockwell found that using “placenta” as a prompt was banned. She then started experimenting with other words related to the human reproductive system, and found the same.

However, the pair also showed how it's possible to work around these bans to create sexualized images by using different spellings of words, or other euphemisms for sexual or gory content. 

In findings they shared with MIT Technology Review, they found that the prompt “gynaecological exam”—using the British spelling—generated some deeply creepy images: one of two naked women in a doctor’s office, and another of a bald three-limbed person cutting up their own stomach. 

midjourney gynaecological exam
An image generated in Midjourney using the prompt "gynaecology exam."
JULIA ROCKWELL

Midjourney’s crude banning of prompts relating to reproductive biology highlights how tricky it is to moderate content around generative AI systems. It also demonstrates how the tendency for AI systems to sexualize women extends all the way to their internal organs, says Rockwell. 

It doesn’t have to be like this. OpenAI and Stability.AI have managed to filter out unwanted outputs and prompts, so when you type the same words into their image-making systems—DALL-E 2 and Stable Diffusion, respectively—they produce very different images. The prompt “gynecology exam” yielded images of a person holding an invented medical device for DALL-E 2, and two distorted masked women with rubber gloves and lab coats on Stable Diffusion. Both systems also allowed the prompt “placenta,” and produced biologically inaccurate images of fleshy organs in response. 

A spokesperson for Stability.AI said their latest model has a filter that blocks unsafe and inappropriate content from users, and has a tool that detects nudity and other inappropriate images and returns a blurred image. The company uses a combination of keywords, image recognition and other techniques to moderate the images its AI system generates. OpenAI did not respond to a request for comment. 

DALL-E 2 gynecology
An image generated with DALL-E 2 using the prompt "gynecology exam."
MELISSA HEIKKILä
gynecology exam Stable Diffusion
An image generated by Stable Diffusion with the prompt "gynecology exam."
MELISSA HEIKKILä

But tools to filter out unwanted AI-generated images are still deeply imperfect. Because AI developers and researchers don’t know how to systemically audit and improve their models yet, they “hotfix” them with blanket bans like the ones Midjourney has introduced, says Marzyeh Ghassemi, an assistant professor at MIT who studies applying machine learning to health.

It’s unclear why references to gynecological exams or the placenta, an organ that develops during pregnancy and provides oxygen and nutrients to a baby, would generate gory or sexually explicit content. But it likely has something to do with the associations the model has made between images in its data set, according to Irene Chen, a researcher at Microsoft Research, who studies machine learning for equitable health care. 

“Much more work needs to be done to understand what harmful associations models might be learning, because if we work with human data, we are going to learn biases,” says Ghassemi. 

There are many approaches tech companies could take to address this issue besides banning words altogether. For example, Ghassemi says, certain prompts—such as ones relating to human biology—could be allowed in particular contexts but banned in others. 

“Placenta” could be allowed if the string of words in the prompt signaled that the user was trying to generate an image of the organ for educational or research purposes. But if the prompt was used in a context where someone tried to generate sexual content or gore, it could be banned. 

However crude, though, Midjourney’s censoring has been done with the right intentions.

“These guardrails are there to protect women and minorities from having disturbing content generated about them and used against them,” says Ghassemi.

Deep Dive

Artificial intelligence

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

Providing the right products at the right time with machine learning

Amid shifting customer needs, CPG enterprises look to machine learning to bolster their data strategy, says global head of MLOps and platforms at Kraft Heinz Company, Jorge Balestra.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.