The Giant Who Walks Amongst Us

Far from inhibiting the imagination, Augmented Reality is allowing researchers to create story characters that can interact with readers.

Giant Jimmy Jones is a friendly, helpful giant. In fact, this book character is so helpful, he can make the sun shine on an otherwise gray village. The giant simply walks across the page, reaches up to the cloud cover and pushes it out of the sun's way so the villagers can catch some rays.

Those light rays may be virtual, but the book this scene pops out of is not.

Using augmented reality (AR), the technology behind the interactive version of Giant Jimmy Jones, New Zealand author Gavin Bishop recently collaborated with Mark Billinghurst and his colleagues at the Human Interface Technology Laboratory New Zealand (HIT Lab NZ) to turn the book into not only a storytelling device, but also a storytelling experience.

A child can flip through its pages and read it like a conventional book. But with a handheld display and computer vision tracking technology, the child can watch the story literally come to life.

"You can see animated virtual characters overlaid on the real book pages and hear the voice of Gavin Bishop reading the story," says Billinghurst, director of the HIT Lab NZ..

While Giant Jimmy Jones currently only exists in a lab setting, there are scores of others being developed at places such as Georgia Tech University’s Augmented Environments Laboratory.

Technology is not the hindrance to turning books into interactive devices whose readers can exist within them and manipulate their stories. The most difficult roadblock stems from the limitations of physical books, most notably the reality that embedding markers that can interact with VR-headgear is expensive and produces ugly visual results on the page. 

Virtual Stories

Unlike virtual reality that exists only within the confines of a computer-generated world, augmented reality includes virtual space digitally, seamlessly overlaid onto a real environment, explains Kelly L. Dempski, a senior researcher at Chicago-based Accenture Technology Labs.

For example, AR can be used in tandem with a child reading from a book and a computer loaded with AR software. The child, then, could virtually place characters from his story anywhere in the room.

The challenge, of course, is to get the technology to work with the user.

Story continues below

A wearable computer in the form of a head-mounted display (HMD) is worn like a fighter-pilot helmet, fitting over the head and eyes, and projecting images within the user's visual field.

Improvements have led to devices that resemble glasses, but even these are unwieldy, unsanitary and limited to one user at a time. The HMDs also suffer tracking problems, or perfect registration, which means that the virtual overlay doesn't quite match up with the physical space upon which it is projected.