Skip to Content

Startup Uses Computer Vision to Make Augmented Reality in Cities More Precise

Making virtual images look good against a real-world backdrop is not easy, but this might be a good fix.
August 4, 2017
Blippar, a mobile augmented-reality startup, says it’s using computer vision to more precisely determine where to place virtual images in real space.

If you’ve ever played around with an augmented-reality app like Pokémon Go on a smartphone, you know that even though it can be fun, the virtual images you see through the eye of your phone’s display often don’t look quite right against the real world in the background.  

Part of the problem is that AR apps tend to depend on a combination of your phone’s GPS and compass to figure out where you are and, thus, where to show these images on your screen, yet GPS doesn’t always work that well if you’re wandering around a bustling city. This can result in virtual objects that appear jittery and out of place, rather than nicely juxtaposed with your surroundings.

An augmented reality startup called Blippar is working on a different method that it thinks can lead to much better-looking AR apps: employing computer vision to help figure out where you are and what direction you’re facing, relative to the busy urban space around you, in a way that it says is often more accurate in cities than GPS.

Omar Tayeb, a Blippar cofounder and its chief technology officer, says the startup is doing this by licensing a trove of image data for major cities—essentially, another company’s version of Google Street View (it won’t say which company it’s going with). It indexes the pictures and then matches them up with what a smartphone user sees through the phone’s camera, in order to find the closest match to their location (the phone may also use GPS or cell-tower triangulation to determine where you are; that’s not up to Blippar, he says). So far, he says, the company has tested it out in San Francisco, London, and Mountain View, California.

With the city image data it’s using, Tayeb says, Blippar has the ability to get pictures of buildings from different angles, which help determine how far away you are from one and from what angle you’re looking at it. This also more precisely determines where a virtual sign or other image should go.

You can get a sense for how this could look in a video shot on an iPhone that’s running a prototype app that Blippar is using internally.

The graphics in the video look rough: cyclists ride through bands of color overlaid on the street, and the sandwich board floats weirdly above the ground.

But the images do appear swiftly, and their locations seem to make sense. Tayeb says that Blippar’s method of location estimation is accurate within eight meters on average, and in most cases under three meters. GPS on a phone is typically accurate within five meters if you’re in an open area; it gets worse when you’re in a place with lots of buildings, trees, and so on.

Tayeb says Blippar plans to release Android and iPhone apps publicly in the next three months that will demonstrate how its positioning technique could work with content like real estate listings or restaurant reviews overlaid on real-world locations; eventually it hopes that other companies will license the technology for their own apps, too.

Keep Reading

Most Popular

Scientists are finding signals of long covid in blood. They could lead to new treatments.

Faults in a certain part of the immune system might be at the root of some long covid cases, new research suggests.

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.