Supercomputing Resurrected
Last year, Japan fired up an ultrafast computer that puts its closest competitors to shame. What will it take for the United States to catch up?
Even in a field defined by continuous breakthroughs, the achievement was a shocker: last March the Japanese government fired up a computer that soon proved to be the fastest in the world, in some cases outperforming the next-fastest computer by a factor of 10. The Earth Simulator, built by NEC, took four years to assemble and cost at least $350 million. It quickly delivered real-world scientific results in global-climate modeling, completing simulations that made other computers look crude. Scientists worldwide lined up for the limited amount of computer time available to researchers outside Japan. By June, just weeks after the machine hummed to life, three of the six finalists for the prestigious Gordon Bell awards in high-performance computing had run their projects on the Earth Simulator.
A smattering of articles last spring covered the news, quoting experts who compared the Earth Simulator to Sputnik-another instance of the United States’ having been severely outclassed in a critical technology. But outside the rarefied circles of high-end computing, the story soon died. U.S. computer vendors have been downplaying the achievement, dismissing the Earth Simulator as “old technology” or “too specialized” to be of much use, even insisting that it was a “publicity stunt.” “Give us $400 million to spend on a single computer, and we could build something just as fast,” says Peter Ungaro, vice president of high-performance computing at IBM.
“I love that,” scoffs Gordon Bell, designer of the first minicomputer for Digital Equipment and a luminary in high-performance computing. “How is IBM going to do it? Where is the technology? I want to bet $1,000 that in the next year, IBM can’t match the cost performance of the Earth Simulator on any system they have.” In fact, IBM recently won a Department of Energy contract to build a pair of machines designed to run at two to nine times the speed of the Earth Simulator, but the project will take until 2005 to complete. Like many of those involved in high-powered scientific computing, Bell believes that Japan’s achievement has exposed a gaping hole in the development of supercomputer systems in the United States-a hole that money alone can’t fill.
What happened that allowed NEC to take such a tremendous lead in computing power? Simply put, the Japanese government saw fit to subsidize the development of the world’s most expensive computer. The project’s goal was not to grab bragging rights from the United States, but to advance scientists’ understanding of the global climate by creating a machine that performs better modeling and weather simulations than ever before.
At the same time, U.S. government funding for research on high-end computing was waning in response to the deeply felt U.S. notion that supercomputer developers-like welfare moms-should take care of themselves rather than survive on government handouts. Compared with any other part of the computer market, the market for supercomputers is small and slow growing, so when public funding dried up, private investment in high-performance architectures dried up too. For the past decade or so, the U.S. emphasis in supercomputing has therefore been on linking clusters of commodity processors-those designed for everyday business applications-in what are known as massively parallel configurations. That approach is a stark contrast to the Japanese vision of specialized architectures developed solely for the high-performance market.
Granted, the commodity approach has gone far: at this writing two commodity machines, the twin Hewlett-Packard-built ASCI Q supercomputers at Los Alamos National Laboratory in New Mexico, rank as second-fastest in the world (as measured by Top500.org, a nonprofit analysis group). The idea of harnessing many low-end processors to do complicated tasks has captured the public imagination as well, with projects such as SETI@home, which enlists the desktop computers of more than four million volunteers to scan radio telescope data for patterns indicative of alien intelligence. Beowulf clusters, which use a method developed in 1994 for linking PCs together to maximize their processing power, have made it even easier to reach high-performance levels with relatively low capital investment. Without question, the commodity approach has proved itself for many applications that at one time ran on specialized “big iron.”
But in spite of these gains, the United States has fallen painfully short in the very field where computing muscle matters most and where the nation has the most to gain: in simulating such complex systems as weather on the macroscopic end and protein folding on the microscopic. This simulation capability is increasingly vital for the advancement of basic science, as well as for national security.
Making the private sector pay for this capability is “like the defense industry’s saying nuclear submarines have to have some sort of commercial spinoff,” says Horst Simon, director of the National Energy Research Scientific Computing Center in Oakland, CA, home to the 12th-fastest computer. “We’ve embarked on a direction in the United States that is not going to work.”
The Need for Speed
What are the real advantages of making computers ever faster? Why, after all, can’t we use a machine that takes a month or a week to complete a task instead of a day or an hour? For many problems, we can. But the truth is, we’re just beginning to gain the computing power to understand what is going on in systems with thousands or millions of variables; even the fastest machines are just now revealing the promise of what’s to come.
Take, for instance, greenhouse gases and the way they affect the global climate, one of the problems the Earth Simulator was built to study. With computers fast enough to predict climate changes accurately, we can know with far greater certainty what level of atmospheric carbon dioxide will melt the polar ice caps. Similarly, because the Earth Simulator models the planet’s climate at an incredible degree of granularity, it can carry out simulations that account for the effects of such local phenomena as thunderstorms. These phenomena may affect areas only 10 kilometers wide-in contrast to the 30 to 50 kilometers most weather models use as the standard grid size.
Or take the difficulties we’ve encountered trying to understand and harness nuclear fusion-that perpetually just-out-of-reach panacea for our energy problems. “It can take a decade to perform a single [fusion] experiment,” says Thomas Sterling, faculty associate at the Center for Advanced Computing Research at Caltech. “Faster computers would accelerate these projects by decades, allowing us not only to design safe reactors that give us the power to run the planet, but also to know how to get rid of the waste.”
One recent example of both the promise and the limitations of today’s most powerful computers came from IBM’s ASCI White machine, the world’s fourth-fastest supercomputer, which IBM researchers used to investigate how materials crack and deform under stress. The study, announced last spring, simulated the behavior of a billion copper atoms. A billion certainly sounds like a lot of variables-until you realize that it would take more than a hundred trillion times that number of atoms to make up even a cubic centimeter of copper.
“There’s a notion out there that high-performance computing is a mature industry, where all the problems have been solved, and we’ve moved on,” says Burton Smith, chief scientist at Cray, a pioneering supercomputer company in Seattle. “That is false. The embarrassment of the Earth Simulator reveals the fact that there is still plenty more understanding to be had.”
WHO MAKES THE MOST SUPERFAST COMPUTERS? | |||||||||||||||||||||||||||
Specifications of Fastest Machine | |||||||||||||||||||||||||||
Company | Number in Top 500 Name Speed (Gigaflops) Location | ||||||||||||||||||||||||||
Hewlett-Packard | 137 ASCI Q 7,727 | Los Alamos National Laboratory, NM | |||||||||||||||||||||||||
IBM | 129 ASCI White 7,226 | Lawrence Livermore National Laboratory, CA | |||||||||||||||||||||||||
Sun Microsystems | 88 HPC 4500 420 | Swedish Armed Forces, Stockholm, Sweden | |||||||||||||||||||||||||
Silicon Graphics | 45 ASCI Blue Mountain 1,608 | Los Alamos National Mountain Laboratory, NM | |||||||||||||||||||||||||
Cray | 22 T3E 1200 1,166 | Unknown (U.S. government) | |||||||||||||||||||||||||
NEC | 15 Earth Simulator 35,860 | Earth Simulator Center, Simulator Yokohama, Japan |
Current and Proposed Supercomputer Architectures | |||
Architectural Approach | Description | Advantages | Main Proponents |
Commodity clusters (operational) | Hundreds or thousands of off-the-shelf servers with low-bandwidth links | Low-cost contruction; efficient with problems that can be broken into chunks | Hewlett-Packard, IBM, Silicon Graphics |
Vector computing (operational) | Hundreds of custom-built orocessors with high-bandwidth connectors | More time spent computing, less time communicating | Cray, NEC |
Streaming (experimental) | Intermediary values of calculations stored in local memory | Speed; on-chip data transfer for reducing the “memory bottleneck” | Stanford University |
Processor-in-memory (experimental) | Processing circuits and short-term memory interspersed on the same chip | Speed; shorter distance between processors and memory | University of Southern California, Caltech, IBM |
Cascade (experimental) | Data, rather than software, held in processor’s local memory | Fewer calls to memory in cases where data sets are larger than programs | Cray, Caltech |
Computing’s Apollo Project?
For the last decade, the U.S. high-performance-computing community has been standing on the shoulders of giants. Many directors of centers for scientific computing say they believe the United States is at a critical decision point, where choice of projects and the amount of funding invested in new high-performance-computing architectures could affect future security and prosperity in tangible ways.
“It’s really going to take a combination of good ideas coming out of universities and government funding and good industrial engineering to address this nasty problem,” says Bell. “Building a new chip is right at the hairy edge of what a university can accomplish; then you need someone with the resources to do detailed engineering stuff like cooling and connections and so on. It’s going to take a lot of effort.”
But if it’s done right, an entirely new golden age of science could flower. One of the most striking aspects of the Earth Simulator project is its openness. Scientists are communicating despite language and geographical barriers. They are testing theories and conducting simulations that have the potential to improve our understanding of the world and benefit all of us. A few months ago, Sterling brokered a meeting between Tetsuya Sato, director of the Earth Simulator facility, and John Gyakum, a McGill University professor who is one of the world’s leading experts on the ways small weather systems such as thunderstorms affect global weather patterns. Before the Earth Simulator there had been no computer that could easily factor such small systems into large-scale climate simulations. Now there may be. “They have opened themselves to collaboration because they care above all about scientific results,” says Sterling. “And what they’re doing is important to everyone on the planet.”
So it’s not just to advance computer science that more and smarter computing is required. It’s to advance every science. “Science in the 21st century rests on three pillars,” says the Energy Department’s Decker. “As always, there’s theory and experiments. But simulation is going to be the third pillar for scientific discovery. Given the problems we’re faced with, we clearly want to be at the cutting edge with our science. If the performance of our computers is an order of magnitude less than what we know they can be even today, then we won’t be.”