Merging Video with Maps

A new system uses panoramic images to create navigation videos that highlight turns and landmarks.

A novel navigation system under development at Microsoft aims to tweak users' visual memory with carefully chosen video clips of a route. Developed with researchers from the University of Konstanz in Germany, the software creates video using 360-degree panoramic images of the street that are strung together. Such images have already been gathered by several different mapping companies for many roads around the world. The navigation system, called Videomap, adjusts the speed of the video and the picture to highlight key areas along the route.

Drive-through: New navigation software uses panoramic images to create a video preview of the route. As the video plays, users can also follow the route on the map (shown here in green and on the next page, larger image). Credit: Microsoft

"What we wanted to do is build a system where we could give [drivers] those visual cues before they got into the car," says Billy Chen, a researcher in the MSN Advanced Engineering group. Ideally, he says, the driver would feel as if she's driven the route before, even if she's never been on those streets.

Videomap still provides written directions and a map with a highlighted route. But unlike existing software, such as Google Maps or MapQuest, the system also allows users to watch a video of their drive. The video slows down to highlight turns or speeds up to minimize the total length of the clip. Memorable landmarks are also highlighted, though at present the researchers have to select them from the video manually.

"As we pass a landmark, the field of view will expand to encompass that landmark and create a landmark thumbnail [image]," Chen says. The video freezes on this image for a few seconds to imprint it in the driver's memory, so that she will recognize it during the drive.

Story continues below

Algorithms also automatically adjust the video to incorporate something Chen calls "turn anticipation." Before a right-hand turn, for example, the video will slow down and focus on images on the right-hand side of the street. This smoothes out the video and draws the driver's attention to the turn. Still images of the street at each turn are also embedded in the map and the written directions.

The system was tested on 20 users, using images of streets in Austria. The participants were given driving directions using the standard map and text, as well as thumbnails for each intersection. Each participant was allotted five minutes to study the information. The drivers were then shown a video simulation of the drive and asked which way the car should turn at various points along the way. They were then asked to do the same thing for a different route, this time using Videomap directions.