Faster Maintenance with Augmented Reality

Augmented reality helps Marine mechanics carry out repair work.

In the not-too-distant future, it might be possible to slip on a pair of augmented-reality (AR) goggles instead of fumbling with a manual while trying to repair a car engine. Instructions overlaid on the real world would show how to complete a task by identifying, for example, exactly where the ignition coil was, and how to wire it up correctly.

Faster fix: A U.S. Marine technician wears an augmented-reality headset as he carries out a maintenance task inside an armored vehicle. Credit: Steven Henderson and Steven Feiner

Multimedia   Click here to see the augmented-reality system from the user’s perspective.

A new AR system developed at Columbia University starts to do just this, and testing performed by Marine mechanics suggests that it can help users find and begin a maintenance task in almost half the usual time.

AR has long shown potential for both entertainment and practical applications, and the first commercial applications are starting to appear in smart phones, thanks to cheaper, more compact computer chips, cameras, and other sensors. So far, however, these apps have been mainly limited to providing directions. But researchers are also working on many practical applications, including ways to help with specific repair and maintenance tasks.

The Columbia researchers worked with mechanics from the U.S. Marine Corps to measure the benefits of using an AR headset when performing repairs to a light armored vehicle. Currently, Marine mechanics have to refer to a technical manual on a laptop while performing maintenance or repairs inside the vehicle, which has many electric, hydraulic, and mechanical components in a tight space.

A user wears a head-worn display, and the AR system provides assistance by showing 3-D arrows that point to a relevant component, text instructions, floating labels and warnings, and animated, 3-D models of the appropriate tools. An Android-powered G1 smart phone attached to the mechanic's wrist provides touchscreen controls for cueing up the next sequence of instructions.

Story continues below

The idea was to present a user with the "information they need to find and fix problems in a way that is going to be more efficient and accurate," says Steven Feiner, a professor of computer science and director of the Computer Graphics and User Interfaces Laboratory at Columbia, who carried out the research with Steven Henderson, an assistant professor at the United States Military Academy's Department of Systems Engineering. Henderson and Feiner presented their paper at the International Symposium on Mixed and Augmented Reality (ISMAR 09) in Orlando, FL, last Thursday, where it won the conference's Best Paper award.

The work "provides more insights into what AR can contribute in the repair and maintenance domain, and in what specific situations AR interfaces can be helpful and advantageous," says Tobias Höllerer, cochair of ISMAR 09 and associate professor at the University of California, Santa Barbara.