Look, Listen, Walk

"Augmented reality" researchers are using location-aware handhelds to change the way we experience the world around us.

You've seen them. Maybe you're one of them. They're the zombies of the New Media Era: the unthinking, the unseeing, the undead. They are all around us.

• The man who sits on the subway, his headphones obscuring his hearing, so closed off from the people next to him that he starts singing out loud.
• The woman talking on her cell phone walking down the street, her eyes half shut, her thoughts miles away, until she sinks up to her ankles in a puddle of melting snow.
• The man in the coffee shop who has his laptop screen up more to shield himself from having to engage in conversation with amiable strangers than to get any work done.
• The person driving home from work who gets so wrapped up in the chatter of talk radio that he doesn't notice when he drives past his exit.

These people are using mobile technology to cut themselves off from the world.

What if we could use these same technologies to engage with the world more fully? That's a goal of research and experimentation in "augmented reality."

People have been talking about virtual reality for decades now-the idea that we can create imaginary environments that engage all of our senses and that we can move through as if they were real environments. Augmented reality turns that premise on its head-heightening our awareness of the real world by annotating it with information conveyed by mobile technologies. Augmented reality has powerful new applications for education, tourism, and storytelling.

In early February, a powerful demonstration of augmented reality took place at Boston's Museum of Science. Eric Klopfer, an MIT professor of urban studies and planning, along with a team of researchers from the Education Arcade (an MIT-based consortium devoted to promoting the pedagogical use of computer and video games) conducted what they called "a Hi-Tech Who Done It." The activity was designed for middle-school kids and their parents. Participants were assigned to teams, consisting of three adult-child pairs, and given a handheld. For the next few hours, they would search high and low for clues of the whereabouts and identity of the notorious Pink Flamingo Gang. Thieves have stolen an artifact and substituted a fake in its place. Thanks to museum's newly installed Wi-Fi network and the players' location-aware handhelds, each gallery offered the opportunity to interview cyber-suspects, download objects, examine them with virtual equipment, and trade their findings.

Each parent-child unit was assigned a different role-biologists, detectives, or technologists-enabling them to use different tools on the evidence they gathered. As I followed the eager participants about the museum, they used walkie-talkies to share information and to call impromptu meetings to compare notes; at one point, a hyperventilating sixth grade girl lectured some other kid's parents about what she learned about the modern synthetic material found in the sample picked up near the shattered mummy case. Racing against time and against rival teams, the kids, parents in tow, sprinted from hall to hall.

I was with one of the teams when they solved the puzzle. A young girl thrust her arms in the air and shouted, "We are the smartest people in the whole museum!" What a visceral experience of empowerment! The same girl said that everyone else in her family was smart in science but that on this occasion, she felt like a genius.

Story continues below

Talking to the parents afterward, one woman told the research team, "This is the longest time I've ever spent having a substantial conversation with my son in as long as I can remember-without any fighting." Many of the others had in the past dragged their kids to the museum kicking and screaming. This time, however, these same kids wanted to go back and spend more time looking at exhibits they had brushed past in their investigations.

The activity had forced the kids to really pay attention to what they were looking at, to ask and answer new questions, and to process the information in new ways. These kids weren't moving in orderly lines through the science museum; they owned that space. It wasn't a sanctuary; it was their playground.