If academic discoveries turn out to be wrong, one drug company wants its money back.
That’s the tough-minded proposal floated today by the chief medical officer of Merck & Co., one of the world’s 10 largest drug companies, as a way to fix the “reproducibility crisis,” or how many, if not most, published scientific reports turn out to be incorrect.
Michael Rosenblatt, Merck’s executive vice president and chief medical officer, said bad results from academic labs caused pharmaceutical companies to waste millions and “threatens the entire biomedical research enterprise.”
The problem of irreproducible research has been getting attention thanks partly to the efforts of a group of psychologists who have been redoing scores of classic experiments and have found most don’t mean much.
Wrong results are also a problem for translational research—the kind drug companies do when they try to turn biological discoveries into actual medicines. Since companies don’t want their cash draining down ratholes, they’re among the few organizations that have taken the trouble to doublecheck results.
The results aren’t pretty. Back in 2012, the biotechnology company Amgen dropped a bomb on academic science when it said it found only six of 53 “landmark” cancer papers stood up to efforts to reproduce the results of promising new research. Other studies that drug companies say can’t be replicated include one that found a cancer drug might treat Alzheimer’s and another that showed a particular gene was linked to diabetes in mice.
Rosenblatt says the costs of repeating wrong research are adding up. He says on average it takes “approximately two to six scientific personnel one to two years of work in an industry laboratory” to try to reproduce original experiments at an average cost of $500,000 to $2 million.
In his editorial, published today in Science Translational Medicine, Merck’s medical chief paints a dire picture:
As the public, government, and private funders of research comprehend the extent of the problem, trust in the scientific enterprise erodes, and confidence in the ability of the scientific community to address this problem wanes. In addition, there is considerable potential for reputational damage to scientists, universities, and entire fields (for example, cancer biology, genomics, and psychology).
Why is science wrong so often? Merck lists the usual suspects: pressure to publish and win grants, careerism, poor training of students, and journals that don’t review reports rigorously enough.
Instead of trying to fix cultural problems in labs or passing new regulations, Merck thinks some punitive economic incentives are in order, specifically, a “full or partial money-back guarantee.” That is, if research that drug companies pay for turns out to be wrong, universities would have to give back the funding they got. Merck thinks this will put the pressure right where it belongs, on the scientists.
It’s unlikely that universities will jump at Merck’s offer for more accountability. That’s because they are set up to collect R&D money, not return it. “The issue is certainly serious—but if this became a requirement it would stop [university-industry] research in its tracks,” says David Winwood, a business development executive at the Pennington Biomedical Research Center in Baton Rouge, Louisiana. “Few if any public schools would have either the (financial) capacity or, I suspect, the legal authority, to enter into such an agreement.”
Drug companies aren’t saints, either. Suppressing and massaging negative results from drug trials isn’t uncommon and it’s a lot more likely to harm patients than bungled academic research. Yet at least drug firms can pay an economic price for mistakes: in 2004, Merck had to recall the pain drug Vioxx and pay out billions in damages after it became clear that the pill posed a deadly risk the company knew all about.
The other problem with Merck’s proposal to universities is it would open a kind of Pandora’s box of accountability. According to the Association of University Technology Managers, a trade body that Winwood is currently president of, companies paid for $4.6 billion in “sponsored” research at U.S. universities, hospitals, and research centers in 2014.
The federal government, on the other hand, spent $37.9 billion.
So is most taxpayer-funded research wrong, too? Maybe it’s taxpayers, not Merck, who should get a check in the mail.
This new data poisoning tool lets artists fight back against generative AI
The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models.
Rogue superintelligence and merging with machines: Inside the mind of OpenAI’s chief scientist
An exclusive conversation with Ilya Sutskever on his fears for the future of AI and why they’ve made him change the focus of his life’s work.
The Biggest Questions: What is death?
New neuroscience is challenging our understanding of the dying process—bringing opportunities for the living.
Data analytics reveal real business value
Sophisticated analytics tools mine insights from data, optimizing operational processes across the enterprise.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.