Google's Answer to Siri Thinks Ahead
The company’s data stockpile and investment in AI means a smartphone helper that answers queries before you even ask them.
Siri, the virtual assistant built into iPhones, launched to great fanfare last October and soon inspired a crowd of copycat apps, heated online arguments about its effectiveness, and an Apple ad campaign in which it played the starring role. It also inspired speculation that Google’s ad revenue was in jeopardy (see “Does Apple’s Siri Threaten Google’s Search Monopoly?”) and that the search and ads company would be forced to invent its own personal assistant.
Almost a year later, Google’s vision of how a smartphone can become a trusted, all-knowing assistant is rolling out to consumers in the form of Google Now. It’s a feature of the newest iteration of Android, Jelly Bean, which is so far available on only a handful of smartphones, and suggests that Google has ambitions to go well beyond what Siri has shown so far.
Google Now doesn’t have a pretend personality like Apple’s sassy assistant, instead just appearing as a familiar search box. But just like Siri, it can take voice commands related to phone functions such as setting reminders or sending messages, and field requests for information such as “How old is the Eiffel Tower?” and “Where can I find a good Chinese restaurant?”
Also like Siri, Google Now responds with speech. However, rather than passing along queries to third-party services such as Yelp for answers, Google’s helper makes use of the company’s recently launched Knowledge Graph, a database that categorizes information in useful ways (see “Google’s New Brain Could Have a Big Impact”).
Google Now also introduces a new trick. It combines the constant stream of data a smartphone collects on its owner with clues about the person’s life that Google can sift from Web searches and e-mails to guess what he or she would ask it for next. This enables Google Now not only to meet a user’s needs but also, in some cases, to preëmpt them. Virtual index cards appear offering information it thinks you need to know at a particular time.
“That’s actually been a goal for us with Android from the beginning,” says Hugo Barra, director of product management for Android, when asked why Google has moved to position a souped-up version of search at the heart of Android. The desire to offer useful information without a person even asking “comes from Larry [Page, Google’s cofounder],” adds Barra, “if you read the  founders’ letter, he said that one of the company goals is to get out of the way of the user.”
Barra adds that the intimacy of people’s relationships with their smartphones makes Android one of the best places to take that to an extreme—by pulling together everything Google knows about the world, and you. “The best thing about Google Now is that it uses every system that Google has built in the last 10 years. It touches almost every back-end system at Google,” says Barra, and hides that power behind a simple, automatic interface. The result, he says, is an “increase in a person’s tranquility, as opposed to having to install an app or do a search or open the browser to navigate to a webpage.”
Most of us rely heavily on Google’s search capability, whatever smartphone we own. But having the search engine come to you, rather than vice versa, can be uncanny. Thanks to Google Now, as I stroll around San Francisco, live bus times are offered to me whenever I pull my phone from my pocket at a bus stop. And when I get up in the morning, Google Now presents a panel summarizing my optimum transit journey to work along with specific buses and an estimate of the time the trip will take. (If I drove to work, it would show a driving route and traffic conditions.) Google Now will show the status of a flight if an airline confirmation e-mail in my inbox shows I’ll be taking it or if I did a Google Web search for a flight number from my work computer—providing I’ve logged into my Google account. The search history trick is also used to guess which sports results you want to know about. I searched for “Giants playoffs” once. Combined with my location, this means I now automatically see a live score for San Francisco Giants games and the final score once they’re over.
Google Now can automatically notify a user about the weather, traffic, upcoming appointments, flights, nearby businesses such as restaurants and cafés, sports results, public transit and travel information, and movie show times. It’s also smart enough to gauge that some things matter more than others. Unusually bad traffic on your commuting route, for example, warrants an audible notification, while events just appear on the screen the next time you check your phone.
Like many Google products—and Siri, too—Google Now is built on top of machine learning, a branch of AI concerned with using large amounts of data to inform decisions. Also like other Google products, Google Now gets better as the company collects more data and refines its algorithms.
Perhaps one of Google Now’s smartest features is the way it lets the user sidestep the result of that machine learning if it doesn’t work, in the process giving feedback that could be used to improve it. A spoken question doesn’t yield just a card showing the answer and it being read aloud, but also the familiar readable list of blue search results, which can be invaluable if your query was misunderstood. You can also dismiss an information card offered by Google Now, after which you get the option to ban it forever, or tune how often it appears with a few taps.
Krzysztof Gajos, an assistant professor at Harvard who researches how to create intelligent, interactive software systems, says that Google has negotiated a problem that has crippled other attempts to create smarter software. “People take seconds saved for granted but perceive even two seconds of delay as negative,” he says. “It has a serious cost unless you have designed in an alternative.” It’s best of all if that alternative is something familiar and quick to use, such as Google’s search results, he says.
Google Now could become even smarter by drawing on more sources of information, says Gajos. “Location seems to be the best source they have now,” he says, noting that Apple is making use of location too, by automatically presenting loyalty cards or boarding passes through the “Passbook” app when a person goes somewhere these could be used. Gajos adds that academic research into intelligent software systems has shown that estimates of emotional state can be useful. “Inferring frustration or distraction can be very valuable to an adaptive system,” he says, since it can avoid causing annoyance.
No one is likely to know better than Barra the real direction Google will take in its effort to make every Android super-smart. But he’s not giving much away, beyond saying that the number of things the app can automatically offer will increase fast. One thing for sure is that Google Now won’t be losing its anonymity and getting a Siri-style personality anytime soon, despite the fact that some users have warmed to Apple’s virtual assistant (see “Social Intelligence”).
“It’s a design choice, and there was certainly some internal debate on this,” says Barra. “Giving it a specific personality would violate Google’s relationship with some people in a certain way. It can be any of many different personalities, depending on who you are.”