We can become so enamored of new technology in medicine that we sometimes forget it needs to be tested to prove it can accomplish what it seems to promise.
Recently, for example, many health-care systems have embraced a range of remote telemonitoring products to track the health of patients at home. Some patients use such monitoring apps and gadgets on their own initiative (see “The Patient of the Future”). It seems logical that more data for doctors and patients is a good thing. In medicine, however, what matters is whether the innovation helps people live longer, with better health and quality of life. We have not yet assessed whether health-tracking apps actually do this.
In reality, few new technologies are subjected to rigorous evaluation. Unlike new drugs, most can legally be put into practice without formal testing. Moreover, the testing that is done is often carried out by the people who developed the technology, so study designs are susceptible to subtle and often unintended bias.
With colleagues, I recently published the results of a large NIH-funded randomized trial of a highly touted telemonitoring system for patients with heart failure. Patients called an automated phone system every day to answer a series of questions about their symptoms. Certain responses triggered alerts to doctors. It seemed like a simple, effective technology destined to transform medicine and was promoted to hospitals as highly effective. Unfortunately, our study failed to find any benefit. These results were confirmed within months by another large independent evaluation of a similar product.
The lesson: we cannot assume on the basis of mere common sense that a promising technology will work. We need to have our assumptions confirmed by independent studies. This principle applies to a host of medical technologies, including other self-monitoring and reporting tools and electronic health records.
There are many reasons why a technology might not realize its promise. The trickiest have to do with implementation: even a technology that works perfectly and is capable of great things may be used in a way that makes for little or no impact on patients.
Medical technologies are not as simple as pills that a patient takes or does not take. Existing practices may not be able to accommodate a novel approach. Information generated by monitoring systems may be misused, misunderstood, or ignored. For whatever reason, “can’t miss” technologies often do.
Developers of new technologies need to prototype and test with those complexities in mind. At the same time, independent tests of medical technologies need to become faster, cheaper, and more routine. Ultimately, the promotion and implementation of technologies like home monitoring must be based on the results that matter most: the extent to which we have made the patient feel better and live longer.
Harlan Krumholz is the Harold H. Hines Jr. professor of medicine, epidemiology, and public health at the Yale University School of Medicine.
This new data poisoning tool lets artists fight back against generative AI
The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models.
The Biggest Questions: What is death?
New neuroscience is challenging our understanding of the dying process—bringing opportunities for the living.
Rogue superintelligence and merging with machines: Inside the mind of OpenAI’s chief scientist
An exclusive conversation with Ilya Sutskever on his fears for the future of AI and why they’ve made him change the focus of his life’s work.
How to fix the internet
If we want online discourse to improve, we need to move beyond the big platforms.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.