While Alibaba and Microsoft may have developed AIs that outperform humans at a comprehension test, there are still tough natural language challenges facing machines.
The challenge: The Stanford University quiz, based on 500 Wikipedia articles, tests comprehension of words and sentences with questions like: “Which group headlined the Super Bowl 50 half-time show?”
The scores: Humans get 82.304. Alibaba’s AI achieved 82.44, Microsoft’s 82.650.
What they say: Alibaba chief scientist Luo Si says it “means objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines,” and plans to use the technology in real-world applications like customer service.
But: This isn’t comprehension the way humans think of it. It’s neat, but the AI doesn’t really understand what it reads—it doesn’t know what “British rock group Coldplay” really is, besides it being the answer to the Super Bowl question. And there are far harder language problems that humans still beat computers at.
This artist is dominating AI-generated art. And he’s not happy about it.
Greg Rutkowski is a more popular prompt than Picasso.
What does GPT-3 “know” about me?
Large language models are trained on troves of personal data hoovered from the internet. So I wanted to know: What does it have on me?
DeepMind has predicted the structure of almost every protein known to science
And it’s giving the data away for free, which could spur new scientific discoveries.
An AI that can design new proteins could help unlock new cures and materials
The machine-learning tool could help researchers discover entirely new proteins not yet known to science.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.