Assembling the Digital Sky

U.S. astronomers are gathering terabytes of data into a worldwide "virtual observatory" that will be accessible to scientists and laymen alike.

By David Essex

November 22, 2002

Scientists in the United States, armed with a $10 million grant from the National Science Foundation, are building a National Virtual Observatory (NVO) that will make the world's huge store of astronomical data available to anyone with a Web browser.

"History has shown us that the greatest leaps forward have occurred not when you observe the universe through just one window, but when you compare the views of the universe obtained through different windows," says Ray Norris, deputy director of the  Australia Telescope National Facility in Epping, New South Wales, Australia. "The NVO will enable any astronomer to do this easily, combining all available data on one object or one region of the sky, or perhaps even using data-mining techniques to look for subtle correlations between the properties of a class of objects when viewed through different windows."

The hope is to dramatically advance this computational approach to astronomy. "I can imagine entire research projects being done from NVO data," says Bob Hanisch, the NVO project manager and an astronomer at the Space Telescope Science Institute in Baltimore.

The inspiration for the NVO is the Sloan Digital Sky Survey, an electronic catalog of images in multiple wavelengths spanning half the northern sky-100 million celestial objects in all, encoded in four databases and viewable from a Web portal. The NVO will take the Sloan survey and combine it with other, smaller U.S. and international surveys, including some maintained by the United Kingdom, Australia, India, and European Union.

As the virtual in NVO suggests, the project is more about computing than the optical telescope images and gamma ray, infrared, radio, ultraviolet, and X-ray snapshots of the heavens collected in the surveys. The main hardware platform will be the emerging "grids" that federate research centers' supercomputers, servers, and high-speed networks into single, powerful computing resources. The NVO will both depend on grid computing and demonstrate its usefulness, astronomy being an uncommonly good test case, say NVO advocates, because of its large yet manageable universe of free, publicly available data.

Story continues below

Building out grids is more the task of participating grid-computing hotbeds such as the San Diego Supercomputer Center (SDSC). For their part, NVO architects will instead  tackle other challenges on the bleeding edge of computing, most of which involve managing large distributed databases. The trick is to make a collection of fundamentally different databases (some in Oracle, others in SQL Server, for example) work uniformly with the software that displays and analyzes the information. The databases themselves will usually remain in separate locations to avoiding clogging network bandwidth, but performance will still be an issue, especially when researchers want to run complex queries. In response, Hanisch says, NVO data centers plan to offer additional services that take over such jobs from remote PCs.

Other database-intensive disciplines, such as bioinformatics, astrophysics, and the earth sciences, stand to gain from potential advances in grid computing and database technology. Bioinformatics is eyeing the NVO for new approaches to storing and exchanging multi-gigabyte maps of the human genome. Earth scientists are also involved in the NVO research effort because, Hanisch says, like astronomers, they work by comparing data from different instruments.