Geneticist Craig Venter and colleagues have tested two of the leading consumer genomics services and declared the fledgling industry to be promising, but still very early in terms of how useful the information might be.

Venter, the founder of the J. Craig Venter Institute in San Diego, CA, and collaborators from that institute and from Scripps Translational Science Institute in La Jolla, CA, sent the saliva of five individuals–they don’t say whose spit they used–to 23andme and Navigenics, two major online DNA-testing companies both based in the San Francisco Bay area. Venter’s team then analyzed and compared the results to see if the sites provided consistent information.

The team compared the five sets of DNA results for the risk of developing 13 diseases, including colon cancer, lupus, type 2 diabetes, and restless leg syndrome.

The results are published today in a commentary in the journal Nature, along with suggestions on how to improve genetic testing as a direct-to-consumer product.

Venter’s group found that the raw genetic sequencing data supplied by each company was almost 100 percent consistent. That is, for the 500,000 to 1 million genetic markers tested for each person, the As, Cs, Ts, and Gs were almost exactly the same.

How the genetic testing sites interpreted the data was less consistent. Each used studies in the scientific literature that have scanned human populations for DNA markers associated with risk factors in order to predict whether a person will succumb to a particular disease. A person might have, say, a 30 percent increased risk for type 2 diabetes if they have a particular version of the relevant genetic marker.

For the seven diseases analyzed by the researchers, only about half of the risk factors provided by 23andme and Navigenics agreed for the five patients. For instance, for lupus and type 2 diabetes, three of the five subjects received conflicting results.

Digging deeper, the researchers found that some of the individual risk factors were strikingly different. For psoriasis, 23andme reported a risk factor of 4.02 (four times greater) for one individual, while Navigenics reported only a 1.25 risk factor (25 percent greater), a threefold difference.

In my own experiments, I compared results delivered by several online genetic testing websites: 23andme and Navigenics, and also Iceland-based deCodeme. My results for heart attack produced three different overall heart-attack risks–high from Navigenics, medium from 23andme, and low from deCodeme. I found several less-striking contradictions for diabetes, macular degeneration, and other traits.

The differences arise for two main reasons: because different companies sometimes use different markers, and combinations of markers, to determine an overall risk score for a disease, and because the algorithms the sites use differ in how they weigh the risk factors for different genetic markers.

The Venter study notes that, in some cases, companies define the average population disease risk differently. “Navigenics distinguishes population disease risk between men and women (for example, men are more likely to have heart attacks than women), whereas 23andMe primarily takes into account age (for example, incidence of rheumatoid arthritis increases with age),” the authors write. “This ambiguity in the definition of a ‘population’ underscores the caution one must exercise when interpreting absolute risk results.”

The Venter study makes several recommendations for direct-to-consumer genetic testing companies. These include a greater focus on the genetic variations that have a high impact on disease risk and a call to use more markers that provide information on risk factors for taking medications, such as the blood-thinner warfarin and cholesterol-lowering statins. (23andme does provide a marker that can indicate a risk of side effects for warfarin.) The researchers also suggest that sites better explain how their markers fit into the overall impact of genes on a disease (for example, whether a given marker represents 5 percent, 20 percent, or 100 percent of the genetic impact on a disease).

Recommendations to the genetics community include conducting clinical studies to validate genetic markers for disease and for behavioral traits in actual patients tracked over time, and giving more attention to non-Caucasian ethnicities.

23andme cofounder Anne Wojcicki and Navigenics cofounder David Agus both agree with the recommendations. “There is a distinct need for transparency and quality in genetic association studies,” says Agus. “We are giving critical information to individuals to help them with their personal health. That information needs to be correct, or we have done a disservice.”

“We plan on working with 23andme to codevelop standards for the field,” Agus adds. This is a voluntary effort that so far has faltered. 23andme science and policy liaison Andro Hsu says there might be a need for a neutral third party that creates standards–he suggested the Centers for Disease Control. Others have talked about the Food and Drug Administration, or even the National Institute of Standards and Technology.

What Venter and company didn’t mention is the word “regulation”–which is likely to happen in some form if the industry is to ensure that information is accurate and consistent. The Nature commentary also doesn’t call for an aggressive and comprehensive effort to validate genetic tests by running studies in the clinic to see which of these markers really do predict disease–a project that may be needed to accelerate the day when these tests become more medically useful for individuals.

Venter does, however, suggest that the latest DNA sequencing technology is rapidly producing a more accurate and thorough alternative to the sort of single DNA markers used by these companies. In just the past few months, the ability of scientists to sequence the entire six billion nucleotides of a person’s genome has become inexpensive enough that it may soon replace the chips currently used by testing companies. Existing tests scan a person’s genome for up to one million markers covering most genetic variations associated with human disease, but they don’t nearly cover them all.

Just two years ago, it cost $1 million to sequence an entire genome; now the price is rapidly dropping to under $50,000, and may be as little as $5,000 by next year.

“Once we have at least 10,000 human genomes and the complete phenotype [disease profiles] with these genomes, we will be able to make correlations that are impossible to do at the present time,” says Venter. “At that stage, genetic testing will be a good investment for private companies and the government.”