Skip to Content
Artificial intelligence

Google says its new chatbot Meena is the best in the world

person using a laptop
person using a laptop
person using a laptopNeONBRAND | Unsplash

Google has released a neural-network-powered chatbot called Meena that it claims is better than any other chatbot out there.

Data slurp: Meena was trained on a whopping 341 gigabytes of public social-media chatter—8.5 times as much data as OpenAI’s GPT-2. Google says Meena can talk about pretty much anything, and can even make up (bad) jokes.

Why it matters: Open-ended conversation that covers a wide range of topics is hard, and most chatbots can’t keep up. At some point most say things that make no sense or reveal a lack of basic knowledge about the world. A chatbot that avoids such mistakes will go a long way toward making AIs feel more human, and make characters in video games more lifelike. 

Sense and specificity: To put Meena to the test, Google has developed a new metric it calls the Sensibleness and Specificity Average (SSA), which captures important attributes for natural conversations, such as whether each utterance makes sense in context—which many chatbots can do—and is specific to what has just been said, which is harder.

What do you mean? For example, if you say “I like tennis” and a chatbot replies “That’s nice,” the response makes sense but is not specific. Many chatbots rely on tricks like this to hide the fact that they don’t know what you’re talking about. On the other hand, a response such as “Me too—I can’t get enough of Roger Federer” is specific. Google used crowdworkers to generate sample conversations and to score utterances in around 100 conversations. Meena got an SSA score of 79%, compared with 56% for Mitsuku, a state-of-the-art chatbot that has won the Loebner Prize for the last four years. Even human conversation partners only scored 86% in this new test. 

Can I talk to Meena? Not yet. Google says it won’t be releasing a public demo until it has vetted the model for safety and bias, which is probably a good thing. When Microsoft released its chatbot Tay on Twitter in 2016, it started spewing racist, misogynistic invective within hours. 

Deep Dive

Artificial intelligence

open sourcing language models concept
open sourcing language models concept

Meta has built a massive new language AI—and it’s giving it away for free

Facebook’s parent company is inviting researchers to pore over and pick apart the flaws in its version of GPT-3

images created by Google Imagen
images created by Google Imagen

The dark secret behind those cute AI-generated animal images

Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.

Yann LeCun
Yann LeCun

Yann LeCun has a bold new vision for the future of AI

One of the godfathers of deep learning pulls together old ideas to sketch out a fresh path for AI, but raises as many questions as he answers.

AGI is just chatter for now concept
AGI is just chatter for now concept

The hype around DeepMind’s new AI model misses what’s actually cool about it

Some worry that the chatter about these tools is doing the whole field a disservice.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.