Skip to Content
Artificial intelligence

Humans may be more likely to believe disinformation generated by AI

The way AI models structure text may have something to do with it, according to the study authors.

empty suit runs with laptop and tweet in binary code
Stephanie Arnett/MITTR | Envato

Disinformation generated by AI may be more convincing than disinformation written by humans, a new study suggests. 

The research found that people were 3% less likely to spot false tweets generated by AI than those written by humans.

That credibility gap, while small, is concerning given that the problem of AI-generated disinformation seems poised to grow significantly, says Giovanni Spitale, the researcher at the University of Zurich who led the study, which appeared in Science Advances today. 

“The fact that AI-generated disinformation is not only cheaper and faster, but also more effective, gives me nightmares,” he says. He believes that if the team repeated the study with the latest large language model from OpenAI, GPT-4, the difference would be even bigger, given how much more powerful GPT-4 is. 

To test our susceptibility to different types of text, the researchers chose common disinformation topics, including climate change and covid. Then they asked OpenAI’s large language model GPT-3 to generate 10 true tweets and 10 false ones, and collected a random sample of both true and false tweets from Twitter. 

Next, they recruited 697 people to complete an online quiz judging whether tweets were generated by AI or collected from Twitter, and whether they were accurate or contained disinformation. They found that participants were 3% less likely to believe human-written false tweets than AI-written ones. 

The researchers are unsure why people may be more likely to believe tweets written by AI. But the way in which GPT-3 orders information could have something to do with it, according to Spitale. 

“GPT-3’s text tends to be a bit more structured when compared to organic [human-written] text,” he says. “But it’s also condensed, so it’s easier to process.”

The generative AI boom puts powerful, accessible AI tools in the hands of everyone, including bad actors. Models like GPT-3 can generate incorrect text that appears convincing, which could be used to generate false narratives quickly and cheaply for conspiracy theorists and disinformation campaigns. The weapons to fight the problem—AI text-detection tools—are still in the early stages of development, and many are not entirely accurate. 

OpenAI is aware that its AI tools could be weaponized to produce large-scale disinformation campaigns. Although this violates its policies, it released a report in January warning that it’s “all but impossible to ensure that large language models are never used to generate disinformation.” OpenAI did not immediately respond to a request for comment.

However, the company has also urged caution when it comes to overestimating the impact of disinformation campaigns. Further research is needed to determine the populations at greatest risk from AI-generated inauthentic content, as well as the relationship between AI model size and the overall performance or persuasiveness of its output, the authors of OpenAI’s report say. 

It’s too early to panic, says Jon Roozenbeek, a postdoc researcher who studies misinformation at the department of psychology at the University of Cambridge, who was not involved in the study. 

Although distributing disinformation online may be easier and cheaper with AI than with human-staffed troll farms, moderation on tech platforms and automated detection systems are still obstacles to its spread, he says. 

“Just because AI makes it easier to write a tweet that might be slightly more persuasive than whatever some poor sap in some factory in St. Petersburg came up with, it doesn’t necessarily mean that all of a sudden everyone is ripe to be manipulated,” he adds.

Deep Dive

Artificial intelligence

DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.

“This is a profound moment in the history of technology,” says Mustafa Suleyman.

AI hype is built on high test scores. Those tests are flawed.

With hopes and fears about the technology running wild, it's time to agree on what it can and can't do.

You need to talk to your kid about AI. Here are 6 things you should say.

As children start back at school this week, it’s not just ChatGPT you need to be thinking about.

AI language models are rife with different political biases

New research explains you’ll get more right- or left-wing answers, depending on which AI model you ask.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.