Skip to Content
Humans and technology

HoloLens can now guide the blind through complicated buildings

The headset’s ability to map a space and talk people through it may prove more important than the mixing-imagery-with-reality stuff.
Scott Olson | Getty

HoloLens, Microsoft’s pricey face computer, is made for mixing digital images with the real world. But a group of scientists found it’s really good at a totally unanticipated application: helping blind people find their way through buildings and offering a better sense of where objects are around them.

The researchers, at the California Institute of Technology, created a new guiding app for HoloLens by taking advantage of the device’s real-time room and object mapping capability, as well as speakers that can make audio seem to be coming from different points in three-dimensional space. They used these features to map a complicated path through a campus building and created a virtual guide that helps a blind person navigate it, calling out directions like “Follow me” from what seems like a meter or so ahead of the person, according to work recently published on the bioRxiv website.

An accompanying video shows how this plays out in reality. A female voice directs a HoloLens-wearing study subject, who is blind, by saying things like “Railings on both sides,” “Up stairs,” and “Right turn ahead.” The man follows the commands, walking easily from a first-floor lobby up a set of staircases, around several corners, and past a few doorways until he arrives at a room on the second floor.

He’s one of seven subjects who tried the application. All got to their destination on the first try, though one briefly got off track. Markus Meister, a professor at Caltech and coauthor of the study, thinks the research could eventually lead to a device that could be offered to visually impaired visitors at places like hotels or malls, helping them get around unfamiliar areas more easily. There are already some tools that can be used this way outdoors, such as turn-by-turn mapping apps—but indoors, as Meister notes, there aren’t as many options.

The World Health Organization estimates that 253 million people are blind or visually impaired, so the potential market for such an application could be huge. But there’s still a lot of work to be done. For now, any routes from one point to another must be scanned in advance, and there isn’t a way to track other people who might walk through the space as the HoloLens wearer is navigating it.

But the study subject in the video, at least, was impressed with the work thus far.

“That was pretty cool,” he says, chuckling, at the end of the clip.

Deep Dive

Humans and technology

anti-choice surveillance tactics
anti-choice surveillance tactics

Anti-abortion activists are collecting the data they’ll need for prosecutions post-Roe

Body cams and license plates are already being used to track people arriving at abortion clinics.

Chinese livestreamer and beauty influencer Li Jiaqi, also known as "king of lipstick," is seen in a subway station in Shanghai
Chinese livestreamer and beauty influencer Li Jiaqi, also known as "king of lipstick," is seen in a subway station in Shanghai

How China’s biggest online influencers fell from their thrones

Three top livestreaming personalities on the platform Taobao commanded legions of fans who bought billions of dollars’ worth of goods—until they suddenly went dark.

animal crossing concepts
animal crossing concepts

Inside the experimental world of animal infrastructure

Wildlife crossings cut down on roadkill. But are they really a boon for conservation?

Meta bombards cancer patients with quack ads
Meta bombards cancer patients with quack ads

Facebook is bombarding cancer patients with ads for unproven treatments

Clinics offering debunked cancer treatments are still allowed to advertise, despite the company’s stated efforts to control medical misinformation.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.