Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

Microsoft Wants You to Educate Its Virtual Assistant

Microsoft says the AI systems behind its new personal assistant will allow it to quickly become much smarter as more people use it.

An app that really approaches the capabilities of a human personal assistant would change the way people use computing devices of all kinds.

Microsoft’s new virtual personal assistant Cortana will meet its public in the coming months as the company’s upgraded mobile operating system rolls out to existing and new devices. But according to Larry Heck, a researcher who contributed to the artificial intelligence underpinning Cortana, first impressions will quickly become outdated.

cortana logo on device
Teach me: Microsoft says that its virtual assistant app, Cortana, will quickly get smarter thanks to analysis of data generated as people interact with it.

That’s because Microsoft has designed Cortana to get significantly smarter as people use it. Heck claims that Microsoft’s commitment to that approach will allow its virtual assistant, currently described as a beta product, to come closer than competitors like Siri or Google Now to delivering the experience of interacting with a real person. “We can update Cortana for all users on an ongoing basis,” he says. “Cortana should be able to improve really fast.”

Cortana learns about its users—for example, by asking for information such as a favorite sports team, or by deducing the whereabouts of your workplace from your phone’s location data. And Heck’s effort to use data flowing through Cortana to continually improve it will likely be crucial to its future.

Much like Apple’s Siri (see “Social Intelligence”), Cortana has been given a sassy personality to draw people into back-and-forth conversations. Making people feel they can talk back to Cortana raises the chances that Microsoft will learn what they really wanted when the app doesn’t get it right the first time, says Heck.

“It might make a mistake, but two or three turns of the conversation later we find what a person really meant,” he says. “That gets data coming back to us that we can feed back into the machine learning.”

Siri can also handle follow-up questions or commands in some situations, but Cortana does so with greater sophistication. For example, Siri, Google Now, and Cortana all return a list of suggestions when asked, “Find me cheap Japanese restaurants nearby.” But only Cortana responds when then asked, “How long will it take me to get to [restaurant name]?” or “Which ones are open now?”

Cortana can also keep track of the context of a conversation for longer: it understands what is meant if someone goes on to ask, “How long would it take to walk?”

Heck’s team will use tricks from the world of online search to work out when something has gone wrong. A person hitting the back button, or repeating a question in a rephrased form, could indicate that Cortana misunderstood.

Norman Winarsky, vice president of SRI Ventures at SRI International, says Heck’s approach is crucial to getting people to treat Microsoft’s app like a capable human assistant. “The quality of the data they gather and how they learn to use it is going to determine the success of Cortana,” says Winarsky, who cofounded the startup that invented Siri as a spinout from a Department of Defense project carried out by SRI (see “TR10: Intelligent Software Assistant”). “If that works, it will create overwhelming value, but they’re taking on a hard challenge.”

Microsoft’s decision to give Cortana a proto-personality and encourage conversation will help provide the feedback needed to make that strategy work, says Winarsky. But it is also likely to make people less tolerant of any mistakes. “There’s a very strong penalty for mistakes in any kind of dialogue,” he says. “If you’re offered something wrong in that kind of medium, especially by voice, you get very impatient very quickly.”

That may be one reason Google chose to make its own voice-operated mobile assistant, Google Now, personality-free and more like a “dumb” search engine (see “Google’s Answer to Siri Thinks Ahead”). Although hard data is lacking, few iPhone users seem to use Siri very much because they feel its performance falls short of Apple’s hype.

Winarsky says he thinks virtual-assistant technology is best confined to more specialized niches for now. Siri was originally designed only for travel and entertainment before Apple snapped it up and broadened its remit. “In a market vertical, we could understand the intent and language and vocabulary much better,” says Winarsky. “The broader you make it, the more brittle it is.” SRI’s more recent virtual-assistant spinouts—Desti for travel and TempoAI for calendar management—have also been focused on specific areas.

Heck argues that Microsoft’s technology is strong enough to support Cortana’s relatively wide scope of duties without compromising reliability. He also says that the way Cortana is connected to the infrastructure behind Microsoft’s search engine Bing—the kind of resource unavailable to Apple—raises the prospect of major leaps in what the app can understand. Heck is currently using some 300,000 computers to process Web data in Bing’s memory banks in an attempt to find ways to automatically extract knowledge and enable Cortana to respond to new kinds of queries.

“If we can connect Cortana on top of that, we can have just as deep a conversation about fly fishing as about finding restaurants,” says Heck, who hopes to extend Microsoft’s Satori knowledge database—a competitor to Google’s own Knowledge Graph (see “How a Database of the World’s Knowledge Shapes Google’s Future”). “I think Cortana will evolve to be this personal assistant you always go to first, like you go to Web search because of the breadth of things it knows about.”

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.