Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

Not By Reason Alone

Should we stop computer science research to prevent the evolution of intelligent machines that might someday surpass humanity? Absolutely not.

In a recent Wired magazine article, Bill Joy argued that the consequences of research on robotics, genetic engineering and nanotechnology may lead to ” knowledge-enabled mass destruction…hugely amplified by the power of self-replication.” His medicine: “relinquishment…by limiting our pursuit of certain kinds of knowledge.” I don’t buy it.

What troubles me with this argument is the arrogant notion that human logic can anticipate the effects of intended or unintended acts, and the more arrogant notion that human reasoning can determine the course of the universe. Let me explain and offer some alternatives.

We are seldom able to assess where we are headed. In 1963, when we built time-shared computers, we did it to spread the cost of a $2 million processor among many users. In 1970, when DARPA pioneered the Arpanet, it did so to avoid buying expensive computers for its contractors, who were told to share their networked machines. Both efforts succeeded, not for these goals, but because they enabled people to share information. The Internet was launched to interconnect networks of computers-no one anticipated that its biggest application would be the Web. Radar was designed for war but ended up as a cornerstone of air transportation. Nuclear weapons research put nuclear medicine on the map. Thousands of innovations all share the same pattern-the early assessment is unrelated to the outcome.

This story is part of our September/October 2000 Issue
See the rest of the issue
Subscribe

So limited is our ability to assess consequences that it’s not even helped by hindsight: On balance, are cars a good or bad thing for society? How about nuclear power, or nuclear medicine? We are unable to judge whether something we invented more than 50 years ago is good or bad for us today. Yet Joy wants us to make these judgments prospectively, to determine which technologies we should forgo!

Developments that today seem fearful may turn into mirages. Take the spiritual machines of Ray Kurzweil that concern Bill Joy. I have a lot of respect for Ray, and I welcome his ideas, as I do Bill’s, however outlandish or controversial they may be. But we should draw a clear line between what is imagined and what is likely. To blur this line is tantamount to quackery. Just because chips and machines are getting faster doesn’t mean they’ll get smarter, let alone lead to self-replication. If you move your arms faster you won’t get brighter. Despite fashionable hyperventilation about intelligent agents, today’s computer systems are not intelligent in the normal sense of the word. Nor do we see on the research horizon the critical technologies that would lead them there. Should we stop computer science and AI research in the belief that intelligent machines someday will reproduce themselves and surpass us? I say no. We should wait to find out whether the potential dangers are supported by more than our imagination.

Since we can’t see where we are headed, should we stop research altogether? This reminds me of a wise old airline employee to whom I was bragging that I had stopped flying with his company because of their lousy safety record. “Listen, sir,” he said to me. “If your exit visa from this life is stamped ‘death by aircraft,’ even if you stay in your bed, the airplane will find you and crash upon you.” At this, the dawn of the technology century, it is not fashionable to pay attention to forces outside reason. We should reconsider. All the more so, if we are under the illusion that we understand enough of our universe to successfully regulate its future course, as Joy suggests.

We shouldn’t forget that what we do as human beings is part of nature. I am not advocating that we do as we please, on the grounds that it is natural, but rather that we hold nature-including our actions-in awe. As we fashion grand strategies to “regulate the ozone problem,” or any other complex aspect of our world, we should be respectful of the unpredictable ways nature may react. And we should approach with equal respect the presumption that the natural human urge to probe our universe should be restricted.

I suggest we broaden our perspective to the fullness of our humanity, which besides reason includes feelings and beliefs. Sometimes, as we drive the car of scientific and technological progress, we’ll veer because our reason says so. At other times we’ll follow our feelings, or we’ll be guided by faith. Most of the time, we’ll steer with all three of these human forces guiding us in concert, as they have guided human actions for thousands of years. As we do so, we should stay vigilant, ready to stop, when danger is imminent, using our full humanity to make that determination. If we do so, our turning point will be very different from where it may seem today, based on early rational assessments…that have failed us so often. Let us have faith in ourselves, our fellow human beings and our universe. And let’s keep in mind that our car is not the only moving thing out there.

Get stories like this before anyone else with First Look.

Subscribe today
Already a Premium subscriber? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look: exclusive early access to important stories, before they’re available to anyone else

    Insider Conversations: listen in on in-depth calls between our editors and today’s thought leaders

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.