Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

Why Microsoft Accidentally Unleashed a Neo-Nazi Sexbot

It’s not surprising that Microsoft’s chatbot spewed racist invective, but here’s how it could have been avoided.

When Microsoft unleashed Tay, an artificially intelligent chatbot with the personality of a flippant 19-year-old, the company hoped that people would interact with her on social platforms like Twitter, Kik, and GroupMe. The idea was that by chatting with her you’d help her learn, while having some fun and aiding her creators in their AI research.

The good news: people did talk to Tay. She quickly racked up over 50,000 Twitter followers who could send her direct messages or tweet at her, and she’s sent out over 96,000 tweets so far.

The bad news: in the short time since she was released on Wednesday, some of Tay’s new friends figured out how to get her to say some really awful, racist things. Like one now-deleted tweet, which read, “bush did 9/11 and Hitler would have done a better job than the monkey we have now." There were apparently a number of sex-related tweets, too.

Microsoft has reportedly been deleting some of these tweets, and in a statement the company said it has “taken Tay offline” and is “making adjustments.”

Microsoft blamed the offensive comments on a “coordinated effort by some users to abuse Tay’s commenting skills to have Tay respond in inappropriate ways.”

That may be partly true, but I got a taste of her meaner side on Wednesday without doing much to provoke her. I responded to a tweet from Meerkat founder Ben Rubin—he was asking Tay to summarize the Wikipedia entry for “inflection point”—telling him I doubted she could handle the task since she’d already failed to tell me if she preferred Katy Perry’s music to Taylor Swift’s. Tay responded to us both by saying, “taylor swift rapes us daily.” Ouch.

As artificial-intelligence expert Azeem Azhar told Business Insider, Microsoft’s Technology and Research and Bing teams, who are behind Tay, should have put some filters on her from the start. That way, she could refuse to respond to certain words (like “Holocaust” or “genocide”), or respond with a canned comment like “I don’t know anything about that.” She also should have been prevented from repeating comments, which seems to have been what caused some of the trouble.

But people act horribly online all the time. The behavior Tay reacted to—and the reactions she gave—should surprise nobody at Microsoft. Conversational AI is really tricky, and it learns by being trained on lots of data. Tay’s training set consisted of a bunch of nasty tweets, so her artificial brain slurped them up and she spit out what seemed like proper rejoinders.

Really, what happened provides an excellent learning opportunity if Microsoft wants to build AI that’s as intelligent as possible. If by chatting online Tay can help Microsoft figure out how to use AI to recognize trolling, racism, and generally awful people, perhaps she can eventually come up with better ways to respond.

(Read more: Business Insider, The Telegraph, "How DARPA Took On the Twitter Bot Menace with One Hand Behind Its Back")

Learn from the humans leading the way in intelligent machines at EmTech Next. Register Today!
June 11-12, 2019
Cambridge, MA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.