AI Beats Humans at Reading Comprehension, but It Still Doesn’t Truly Comprehend Language
While Alibaba and Microsoft may have developed AIs that outperform humans at a comprehension test, there are still tough natural language challenges facing machines.
The challenge: The Stanford University quiz, based on 500 Wikipedia articles, tests comprehension of words and sentences with questions like: “Which group headlined the Super Bowl 50 half-time show?”
The scores: Humans get 82.304. Alibaba’s AI achieved 82.44, Microsoft’s 82.650.
What they say: Alibaba chief scientist Luo Si says it “means objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines,” and plans to use the technology in real-world applications like customer service.
But: This isn’t comprehension the way humans think of it. It’s neat, but the AI doesn’t really understand what it reads—it doesn’t know what “British rock group Coldplay” really is, besides it being the answer to the Super Bowl question. And there are far harder language problems that humans still beat computers at.
Deep Dive
Artificial intelligence
Large language models can do jaw-dropping things. But nobody knows exactly why.
And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.
OpenAI teases an amazing new generative video model called Sora
The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.
Google’s Gemini is now in everything. Here’s how you can try it out.
Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.
Google DeepMind’s new generative model makes Super Mario–like games from scratch
Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.