Checking the checkup: A screenshot of the Isabel software shows a decision tree for diagnosing diseases. Red flags indicate problems that would require urgent attention.
Similarly, IBM researchers are exposing Watson to medical textbooks and journal articles. It’s also seeing anonymized medical records held by WellPoint, a large U.S. health insurance company that plans to eventually have Watson analyze patient records and then give its suggestions to doctors. Someday, Watson could also be pointed to blogs or other websites where patients talk about their responses to medical treatments.
Watson still needs extensive testing before IBM can begin selling its services in the next year or two. That’s why IBM is working with doctors, including Chase and his students at Columbia, who pose questions to the program to see how well the computer answers. IBM can then fix errors they spot—like the time the computer included penicillin’s discoverer, Alexander Fleming, in a list of antibiotics.
Maude says he thinks such glitches will always plague Watson. “You can do something automatically, but the accuracy and the relevance will go down,” he says. “We’ve found that’s often where the wild things come in.” Michael Segal, the doctor who founded SimulConsult, similarly argues that there are instances when a computer can be tripped up by nuances in medical terminology. For instance, he says, the term “congenital” generally means present at birth. But in the context of neuromuscular disorders, “congenital” can be used to describe something that has its onset in the first two years. “It’s just a nightmare to try to do this in an automated way,” he says.
Chase says that even with its occasional hiccups, Watson outperforms any other diagnostic software he has tried, not only because of its broad knowledge base but because doctors can ask it open-ended questions in “natural language,” such as “What is the recommended antibiotic for a pregnant woman with Lyme disease and a penicillin allergy?” IBM is also working with Nuance Communications, a maker of voice recognition software, so that doctors will be able to speak their queries to Watson rather than having to enter them at a keyboard.
But getting the technology right is just part of the project. Resistance from doctors could be a bigger hurdle to getting Watson into widespread use; Chase says he has encountered “insecurity” from doctors when he has talked to them about the program. In trying to sell Isabel, Maude stresses to physicians that while software can help them make a diagnosis, it can’t do it for them. But even that makes some of them bristle. “They do think it takes away some of the magic of being a doctor,” he says. “The ability to diagnose—for them it’s their core skill. It’s kind of what their peers most revere about them.”