Skip to Content
Uncategorized

On Negative Results

There’s a very interesting article by John Ioannidis in PLoS Medicine, the free online journal. Most current published research findings might well be false, he says. There are several factors, and I think it’s worth presenting them in detail: 1….
August 30, 2005

There’s a very interesting article by John Ioannidis in PLoS Medicine, the free online journal. Most current published research findings might well be false, he says. There are several factors, and I think it’s worth presenting them in detail:

1. Many research studies are small, with only a few dozen participants.

2. In many scientific fields, the “effect sizes” (a measure of how much a risk factor such as smoking increases a person’s risk of disease, or how much a treatment is likely to improve a disease) are small. Research findings are more likely true in scientific fields with large effects, such as the impact of smoking on cancer, than in scientific fields where postulated effects are small, such as genetic risk factors for diseases where many different genes are involved in causation. If the effect sizes are very small in a particular field, says Ioannidis, it is “likely to be plagued by almost ubiquitous false-positive claims.”

3. Financial and other interests and prejudices can also lead to untrue results.

4. “The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true,” which may explain why we sometimes see “major excitement followed rapidly by severe disappointments in fields that draw wide attention.”

This ought to be an eye-opener…. The solution? More publication of preliminary findings, negative studies (which often suffer that fate of the file-drawer effect), confirmations, and refutations. PLoS says, “the editors encourage authors to discuss biases, study limitations, and potential confounding factors. We acknowledge that most studies published should be viewed as hypothesis-generating, rather than conclusive.” And maybe this will also temper journalists’ tendency to offer every new study as the Next Big Thing.

Keep Reading

Most Popular

Europe's AI Act concept
Europe's AI Act concept

A quick guide to the most important AI law you’ve never heard of

The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

Uber Autonomous Vehicles parked in a lot
Uber Autonomous Vehicles parked in a lot

It will soon be easy for self-driving cars to hide in plain sight. We shouldn’t let them.

If they ever hit our roads for real, other drivers need to know exactly what they are.

supermassive black hole at center of Milky Way
supermassive black hole at center of Milky Way

This is the first image of the black hole at the center of our galaxy

The stunning image was made possible by linking eight existing radio observatories across the globe.

transplant surgery
transplant surgery

The gene-edited pig heart given to a dying patient was infected with a pig virus

The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.