A View from Oren Etzioni

No, the Experts Don’t Think Superintelligent AI is a Threat to Humanity

Ask the people who should really know.

  • September 20, 2016

If you believe everything you read, you are probably quite worried about the prospect of a superintelligent, killer AI. The Guardian, a British newspaper, warned recently that “we’re like children playing with a bomb,” and a recent Newsweek headline reads, “Artificial Intelligence Is Coming, and It Could Wipe Us Out.”

Numerous such headlines, fueled by comments from the likes of Elon Musk and Stephen Hawking, are strongly influenced by the work of one man: professor Nick Bostrom, author of the philosophical treatise Superintelligence: Paths, Dangers, and Strategies.

Bostrom is an Oxford philosopher, but quantitative assessment of risks is the province of actuarial science. He may be dubbed the world’s first prominent “actuarial philosopher,” though the term seems an oxymoron given that philosophy is an arena for conceptual arguments, and risk assessment is a data-driven statistical exercise.

So what do the data say? Bostrom aggregates the results of four different surveys of groups such as participants in a conference called “Philosophy and Theory of AI,” held in 2011 in Thessaloniki, Greece, and members of the Greek Association for Artificial Intelligence (he does not provide response rates or the phrasing of questions, and he does not account for the reliance on data collected in Greece).

His findings are presented as probabilities that human-level AI will be attained by a certain time:

By 2022: 10 percent.

By 2040: 50 percent.

By 2075: 90 percent.

This aggregate of four surveys is the main source of data on the advent of human-level intelligence in over 300 pages of philosophical arguments, fables, and metaphors. 

To get a more accurate assessment of the opinion of leading researchers in the field, I turned to the Fellows of the American Association for Artificial Intelligence, a group of researchers who are recognized as having made significant, sustained contributions to the field.

In early March 2016, AAAI sent out an anonymous survey on my behalf, posing the following question to 193 fellows:

“In his book, Nick Bostrom has defined Superintelligence as ‘an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.’ When do you think we will achieve Superintelligence?”

Over the next week or so, 80 fellows responded (a 41 percent response rate), and their responses are summarized below: 

In essence, according to 92.5 percent of the respondents, superintelligence is beyond the foreseeable horizon. This interpretation is also supported by written comments shared by the fellows.

Even though the survey was anonymous, 44 fellows chose to identify themselves, including Geoff Hinton (deep-learning luminary), Ed Feigenbaum (Stanford, Turing Award winner), Rodney Brooks (leading roboticist), and Peter Norvig (Google).

The respondents also shared several comments, including the following:

“Way, way, way more than 25 years. Centuries most likely. But not never.”

“We’re competing with millions of years’ evolution of the human brain. We can write single-purpose programs that can compete with humans, and sometimes excel, but the world is not neatly compartmentalized into single-problem questions.”

“Nick Bostrom is a professional scare monger. His Institute’s role is to find existential threats to humanity. He sees them everywhere. I am tempted to refer to him as the ‘Donald Trump’ of AI.”

Surveys do, of course, have limited scientific value. They are notoriously sensitive to question phrasing, selection of respondents, etc. However, it is the one source of data that Bostrom himself turned to.

Another methodology would be to extrapolate from the current state of AI to the future. However, this is difficult because we do not have a quantitative measurement of the current state of human-level intelligence. We have achieved superintelligence in board games like chess and Go (see “Google’s AI Masters Go a Decade Earlier than Expected”), and yet our programs failed to score above 60 percent on eighth grade science tests, as the Allen Institute’s research has shown (see “The Best AI Program Still Flunks an Eighth Grade Science Test”), or above 48 percent in disambiguating simple sentences (see “Tougher Turing Test Exposes Chatbots’ Stupidity”).

There are many valid concerns about AI, from its impact on jobs to its uses in autonomous weapons systems and even to the potential risk of superintelligence. However, predictions that superintelligence is on the foreseeable horizon are not supported by the available data. Moreover, doom-and-gloom predictions often fail to consider the potential benefits of AI in preventing medical errors, reducing car accidents, and more.

Finally, it’s possible that AI systems could collaborate with people to create a symbiotic superintelligence. That would be very different from the pernicious and autonomous kind envisioned by Professor Bostrom.

Oren Etzioni is CEO of the Allen Institute for Artificial Intelligence and Professor of Computer Science at the University of Washington.

Updated November 2, 2016:

I’m delighted that Professors Dafoe & Russell, who responded to my article here, and I seem to be in agreement on three critical matters. One, we should refrain from ad hominem attacks. Here, I have to offer an apology: I should not have quoted the anonymous AAAI Fellow who likened Dr. Bostrom to Donald Trump. I didn’t mean to lend my voice to that comparison; I sincerely apologized to Bostrom for this misstep via e-mail, an apology that he graciously accepted. Two, as scientists, we need to assess statements about the risk of AI based on data. That was the key point of my brief article, and the article offered unique data on this topic. Three, we also concur that the media has misrepresented the implications of Bostrom’s work—a topic of major concern that was allayed to some extent by the White House report on AI. Of course, we do differ on many of the details and the intricacies of the arguments, but time will tell.

Weren't able to make it to EmTech Digital?
We've got you covered.

Watch videos here

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.