Skip to Content

Baidu Is Bringing Intelligent AR to the Masses

An augmented-reality system powered by computer vision and deep learning will add an extra layer to the real world for millions of people.

The Chinese Internet giant Baidu has announced a new AR platform that will allow people to make use of the technology from within many of the company’s own apps.

The system, called DuSee, is claimed to make use of "sophisticated computer vision and deep learning" to “understand the 3-D environment, and create virtual objects that have rich interactions with the user and the real world.” The platform seems to sit somewhere between basic AR systems, like those used in Pokémon Go, and more advanced technologies, such as Google’s Tango. The former simply drops animations onto a device’s screen to make it look like they’re in situ without any analysis of the real world; the latter uses special 3-D camera technology to map the local environment and overlay visual graphics.

Instead, DuSee seems to make use of a smartphone’s single camera, using computer vision software to detect and interpret what it sees in order to add a layer of extra information for the user.

Baidu has offered up two examples of the system in use. One shows the software recognizing the presence of a 2-D map of Shanghai, then render it as a 3-D illustration reminiscent of SimCity. Another identifies a shampoo advert, adding virtual petals that echo its floral branding while keeping them locked in place as the ad moves in space.

Perhaps most exciting, though, is the fact that Baidu plans to bake this platform into its most successful apps, including its Mobile Baidu search software. That app alone has hundreds of millions of users, so the AR will find its way into the lives of many people, very quickly.

Baidu has been doubling down on its development of AI systems in recent years. While in the past it’s flirted with some goofy-yet-impressive image manipulation software, it’s also been using its homegrown Paddle deep-learning system to create more useful products and prototypes. Examples include a headset that uses computer vision to convert images to audio that tells visually impaired people what is in front of them, and powerful speech recognition for its voice interfaces. Indeed, the company points out that it’s “excited about future directions such as integrating AR with speech recognition and natural language processing" into DuSee.

With its cute characters and compelling gameplay, Pokémon Go—which hasn't yet launched in China—has whetted the world’s appetite for AR. Baidu, though, may be the first to offer a truly intelligent augmented reality experience to a mass audience.

(Read more: Baidu, TechCrunch, “How AI Is Feeding China’s Internet Dragon,” “Baidu’s Deep-Learning System Rivals People at Speech Recognition”)

Keep Reading

Most Popular

Rendering of Waterfront Toronto project
Rendering of Waterfront Toronto project

Toronto wants to kill the smart city forever

The city wants to get right what Sidewalk Labs got so wrong.

windows desktop with anime image from Wallpaper Engine
windows desktop with anime image from Wallpaper Engine

Chinese gamers are using a Steam wallpaper app to get porn past the censors

Wallpaper Engine has become a haven for ingenious Chinese users who use it to smuggle adult content as desktop wallpaper. But how long can it last?

Yann LeCun
Yann LeCun

Yann LeCun has a bold new vision for the future of AI

One of the godfathers of deep learning pulls together old ideas to sketch out a fresh path for AI, but raises as many questions as he answers.

Linux hack concept
Linux hack concept

The US military wants to understand the most important software on Earth

Open-source code runs on every computer on the planet—and keeps America’s critical infrastructure going. DARPA is worried about how well it can be trusted

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.