Skip to Content

Healthy Skepticism

Medical technologies are too rarely evaluated with scientific rigor.
February 21, 2012

We can become so enamored of new technology in medicine that we sometimes forget it needs to be tested to prove it can accomplish what it seems to promise.

Recently, for example, many health-care systems have embraced a range of remote telemonitoring products to track the health of patients at home. Some patients use such monitoring apps and gadgets on their own initiative (see “The Patient of the Future”). It seems logical that more data for doctors and patients is a good thing. In medicine, however, what matters is whether the innovation helps people live longer, with better health and quality of life. We have not yet assessed whether health-tracking apps actually do this.

In reality, few new technologies are subjected to rigorous evaluation. Unlike new drugs, most can legally be put into practice without formal testing. Moreover, the testing that is done is often carried out by the people who developed the technology, so study designs are susceptible to subtle and often unintended bias.

With colleagues, I recently published the results of a large NIH-funded randomized trial of a highly touted telemonitoring system for patients with heart failure. Patients called an automated phone system every day to answer a series of questions about their symptoms. Certain responses triggered alerts to doctors. It seemed like a simple, effective technology destined to transform medicine and was promoted to hospitals as highly effective. Unfortunately, our study failed to find any benefit. These results were confirmed within months by another large independent evaluation of a similar product.

The lesson: we cannot assume on the basis of mere common sense that a promising technology will work. We need to have our assumptions confirmed by independent studies. This principle applies to a host of medical technologies, including other self-monitoring and reporting tools and electronic health records.

There are many reasons why a technology might not realize its promise. The trickiest have to do with implementation: even a technology that works perfectly and is capable of great things may be used in a way that makes for little or no impact on patients.

Medical technologies are not as simple as pills that a patient takes or does not take. Existing practices may not be able to accommodate a novel approach. Information generated by monitoring systems may be misused, misunderstood, or ignored. For whatever reason, “can’t miss” technologies often do.

Developers of new technologies need to prototype and test with those complexities in mind. At the same time, independent tests of medical technologies need to become faster, cheaper, and more routine. Ultimately, the promotion and implementation of technologies like home monitoring must be based on the results that matter most: the extent to which we have made the patient feel better and live longer.

Harlan Krumholz is the Harold H. Hines Jr. professor of medicine, epidemiology, and public health at the Yale University School of Medicine.

Keep Reading

Most Popular

AGI is just chatter for now concept
AGI is just chatter for now concept

The hype around DeepMind’s new AI model misses what’s actually cool about it

Some worry that the chatter about these tools is doing the whole field a disservice.

Hoan Ton-That, CEO of Clearview AI
Hoan Ton-That, CEO of Clearview AI

The walls are closing in on Clearview AI

The controversial face recognition company was just fined $10 million for scraping UK faces from the web. That might not be the end of it.

Europe's AI Act concept
Europe's AI Act concept

A quick guide to the most important AI law you’ve never heard of

The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.