Testing on themselves: A graph charts progression of the neurological disease ALS in a number of patients before and after taking lithium.
The researchers could also compare lithium takers to controls in a more nuanced way. By compiling data from patients with similar backgrounds and disease characteristics who did not take lithium, they created a model predicting the course of the disease in that group. They could then determine whether a patient who fit those criteria pretreatment deviated from that progression after taking the drug. The answer was no, meaning lithium had no effect–positive or negative–on the disease.
Criticisms of the approach mimic those typically made of observational trials, which lack a “blind” placebo control group. Outside of a controlled clinical trial, it’s difficult to determine whether the drug or some other factor was the key to the outcome. “The problem with PatientsLikeMe is that it involves observational information. If patients think medication is helping, they will be biased toward recognizing whatever positive events they have and vice versa,” says Paul Bleicher, founder of PhaseForward, a clinical trial data-management company. “Most people aren’t even aware of the bias. That’s why blind trials exist.”
University of Pittsburgh’s Roberts concurs. “What kinds of patients are willing to report their data? Is it the full range of disease? Were people who didn’t do well as likely to report findings as those who didn’t?” he asks. However, Roberts is also optimistic, pointing out that statistical methods can correct for many of these issues. “As long as you are really careful about understanding the possible biases, I think you can begin to approximate the control you have in clinical trials,” he says.
“The types of things you can get from observational studies are the generation of valuable hypotheses, which is not easy to do,” says Bleicher, who also works for Humedica, a health-care informatics company that collects data from electronic medical records. “Using databases, you can come up with observations you believe are strong enough to be worthy of doing a controlled trial.”
Swati Aggarwal, a physician at Massachusetts General Hospital in Boston who led the Lancet study on lithium and ALS, sees PatientsLikeMe as a rich resource for accessing the ALS community. “We could use the database to try to understand why patients don’t like to use BPAP” (bilevel positive airway pressure), a ventilator to help patients breathe, she says.
PatientsLikeMe is currently building models for its other disease communities and next plans to look at the effects of some treatments for multiple sclerosis, as well as nondrug factors in ALS. “The diseases we focus on tend to be those with patients who know more about their health than the medical community does,” says Heywood. “It’s easier to get patients to tell us than to get medical systems to change.”