Researchers at Microsoft and Mitsubishi are developing a new touch-screen system that lets people type text, click hyperlinks, and navigate maps from both the front and back of a portable device. A semitransparent image of the fingers touching the back of the device is superimposed on the front so that users can see what they're touching.

Multitouch screens, popularized by gadgets such as PDAs and Apple's iPhone, are proving to be more versatile input devices than keypads. But the more people touch their screens, says Patrick Baudisch, a Microsoft researcher involved in the touch-screen project, the more content they cover up. "Touch has certain promise but certain problems," he says. "The smaller the touch screen gets, the bigger your fingers are in proportion ... Multitouch multiplies the promise and multiplies the problems. You can have a whole hand over your PDA screen, and that's a no go."

The current prototype, which illustrates a concept that the researchers call LucidTouch, is "hacked together" from existing products, says Daniel Wigdor, a researcher at Mitsubishi Electric Research Lab and a PhD candidate at the University of Toronto. The team started with a seven-inch, commercial, single-input touch screen. To the back of the screen, they glued a touch pad capable of detecting multiple inputs. "This allowed us to have a screen on the front and a gesture pad [on the back] that could have multiple points," says Wigdor. "But what that didn't give us was the ability to see the hands." So, he says, the researchers added a boom with a Web camera to the back of the gadget.

The image from the Web camera and the touch information from the gesture pad are processed by software running on a desktop computer, to which the prototype is connected. The software subtracts the background from the image of the hands, Wigdor explains, and flips it around so that the superimposed image is in the same position as the user's hands. Additionally, pointers are added to the fingers so that a user can precisely select targets on the touch pad that might be smaller than her finger. In October, a paper describing the research will be presented at the User Interface Software and Technology symposium in Rhode Island.

Admittedly, this prototype has several limitations. Most glaringly, it's impractical to attach a boom and camera to the back of a handheld device. In their paper, the researchers suggest a number of different approaches for more-compact LucidTouch prototypes. The gesture pad on the back could actually provide an image of the user's fingers as well as touch information, explains Wigdor. The pad uses an array of capacitors, devices that store electrical charge. Fingers create a tiny electrical field that changes the capacitance of the array, depending on their distance from it. This distance can be tuned, says Wigdor, so that the pad can register the entire finger, and not just the fingertip touching it. Another approach, he says, would be to use an array of tiny, single-pixel light sensors that could map fingers' locations. Or the device could use an array of flashing, infrared-light-emitting diodes; sensors would then detect the light's reflection off of a hand, Wigdor explains.