All five donors received conflicting results, said Kutz. For example, having submitted his own DNA as a sample, Kutz learned that he was simultaneously at less than average risk, average, and above-average risk of developing prostate cancer, depending on the company that did the test. “I believe these results show the tests are not ready for prime time,” he said.
That should be of little surprise to those closely following the field. Craig Venter, who led the private arm of the human genome project, published a commentary with similar findings in the journal Nature last year. Francis Collins, head of the National Institutes of Health, wrote about a similar discovery when he tested different services for his newest book, The Language of Life.
The discrepancy comes from the fact that the different companies calculate risk using different algorithms, which vary in their criteria for which genetic variants to include. During the hearing, Ashley Gould, general counsel for 23andMe, agreed that the field needs standards for calculating risks. The company has already reached out to the FDA and the NIH for help in developing such standards.
Both Jeff Shuren, director of the Center for Devices and Radiological Health at the FDA, and James Evans, professor of genetics and medicine at the University of North Carolina, said they believed that most tests’ analytic results–their ability to correctly detect a specific variation– were accurate. But calculating and assessing clinical validity–using the information to accurately predict disease risk–is an entirely new science. “No one knows how to interpret this data,” said Evans. The FDA has begun discussing regulation of a broader category of laboratory tests, and how to define and regulate clinical validity is a major issue.
The GAO report also found problems with how the companies dealt with consumers of African or Asian descent. Because most genetic research has been done on Caucasians, it’s unclear how well the findings translate to people of different ethnicities. While the companies do point this out on their websites and in their reports, they didn’t prominently disclose this when consumers were ordering and paying for the tests, Kutz said.
Some companies named in the report and the congressional hearing complained that they were being lumped with unnamed companies that came off as outright charlatans, such as the ones selling supplements purportedly tailored to an individual’s DNA. However, the more well-known companies didn’t necessarily shine in comparison. In the audiotapes, a representative from Pathway told a caller it was okay to surreptitiously collect her fiancé’s DNA and have it tested as a gift. That would be illegal in 33 states. When one caller asked whether being classified as having a high risk for developing breast cancer meant she would definitely get the disease, a representative from Navigenics responded, “you’d be in the high risk of, you know, pretty much getting it.”