Skip to Content

Healthy Skepticism

Medical technologies are too rarely evaluated with scientific rigor.
February 21, 2012

We can become so enamored of new technology in medicine that we sometimes forget it needs to be tested to prove it can accomplish what it seems to promise.

Recently, for example, many health-care systems have embraced a range of remote telemonitoring products to track the health of patients at home. Some patients use such monitoring apps and gadgets on their own initiative (see “The Patient of the Future”). It seems logical that more data for doctors and patients is a good thing. In medicine, however, what matters is whether the innovation helps people live longer, with better health and quality of life. We have not yet assessed whether health-tracking apps actually do this.

In reality, few new technologies are subjected to rigorous evaluation. Unlike new drugs, most can legally be put into practice without formal testing. Moreover, the testing that is done is often carried out by the people who developed the technology, so study designs are susceptible to subtle and often unintended bias.

With colleagues, I recently published the results of a large NIH-funded randomized trial of a highly touted telemonitoring system for patients with heart failure. Patients called an automated phone system every day to answer a series of questions about their symptoms. Certain responses triggered alerts to doctors. It seemed like a simple, effective technology destined to transform medicine and was promoted to hospitals as highly effective. Unfortunately, our study failed to find any benefit. These results were confirmed within months by another large independent evaluation of a similar product.

The lesson: we cannot assume on the basis of mere common sense that a promising technology will work. We need to have our assumptions confirmed by independent studies. This principle applies to a host of medical technologies, including other self-monitoring and reporting tools and electronic health records.

There are many reasons why a technology might not realize its promise. The trickiest have to do with implementation: even a technology that works perfectly and is capable of great things may be used in a way that makes for little or no impact on patients.

Medical technologies are not as simple as pills that a patient takes or does not take. Existing practices may not be able to accommodate a novel approach. Information generated by monitoring systems may be misused, misunderstood, or ignored. For whatever reason, “can’t miss” technologies often do.

Developers of new technologies need to prototype and test with those complexities in mind. At the same time, independent tests of medical technologies need to become faster, cheaper, and more routine. Ultimately, the promotion and implementation of technologies like home monitoring must be based on the results that matter most: the extent to which we have made the patient feel better and live longer.

Harlan Krumholz is the Harold H. Hines Jr. professor of medicine, epidemiology, and public health at the Yale University School of Medicine.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.