A Better, Cheaper Multitouch Interface

Over the past few years, the world has fallen in love with multitouch displays. But today’s consumer interfaces have some drawbacks: touch screens such as those on the iPhone and Plastic Logic’s upcoming e-reader only work with a finger, not a stylus or even a gloved hand. Other displays, such as Microsoft’s Surface and Perceptive Pixel’s wall-sized screens, are rigid, relatively expensive, and currently fairly bulky.

New research from New York University, however, promises to make multitouch interfaces that are cheap and flexible and can be used by fingers and objects alike. The technology, called Inexpensive Multi-Touch Pressure Acquisition Devices (IMPAD), can be made paper thin, can easily scale down to fit on small portable devices, or can scale up to cover an entire table or wall. The researchers will present IMPAD next week at the Computer Human Interaction conference in Boston.

The iPhone captures information about touch by measuring a change in capacitance when a finger or other conducting object comes in contact with the display. Surface screens use cameras to see the position of objects on the tabletop. Perceptive Pixel’s displays also use cameras, but in a different way. Those cameras are used to track infrared light as it scatters in the presence of a finger or stylus. While Perceptive Pixel’s touch screens collect pressure information, it’s still impractical to use cameras for smaller or touch interfaces. IMPAD takes a different approach by measuring a change in electrical resistance when a person or object applies different pressure to a specially designed pad, consisting of only a few layers of materials.

“One of the problems that’s been endemic to multitouch sensors is … you’re either touching it or not touching it,” explains Ken Perlin, a professor of media research at NYU. “A significant amount of potentially useful information is thrown away because the sensor isn’t capturing the subtleties.” But with a pressure-sensitive touch pad, a device can see how hard a person presses, opening up another dimension of the user interface. The researchers have shown that their pressure-sensitive touch pad can be used for virtual sculpting and painting applications and for a simulated mouse with left clicks, right clicks, and drags, as well as for musical instruments like a piano keyboard. (See video.)

The hardware that composes the demonstrated prototype is relatively straightforward, explains Ilya Rosenberg, a graduate researcher and lead author on the IMPAD paper. It consists of two plastic sheets, about 8 inches by 10 inches, each with parallel lines of electrodes, spaced a quarter inch apart. The sheets are arranged so that the electrodes cross, creating a grid; each intersection is essentially a pressure sensor. Crucially, both sheets are covered with a layer of force sensitive resistor (FSR) ink, a type of ink that has microscopic bumps on its surface. When something coated in the ink is pressed, the bumps move together and touch, conducting electricity. “The harder you press, the more it conducts,” says Rosenberg.

FSR ink has been used for decades, but mostly on musical instruments such as electronic drums or keyboards, Rosenberg says. In making their touch pad, the researchers had to ensure that the pad could detect the exact placement of a finger even though the sensors are a quarter inch apart–something that designers of the electronic instruments didn’t need to consider.

Ideally, the researchers wanted to be able to measure to a resolution of 100 dots per square inch, but they didn’t want the complications and expense of wiring up such a large number of sensors. So they developed an algorithm that takes the input at each electrode intersection and interpolates the position of an object, even one as small as a stylus tip. It also lets them distinguish between two fingers pressing side by side. The output from the pad is sent to a computer, mapping the intensity and placement of pressure. Currently, data from the entire pad can be collected 50 to 200 times per second.

The simplicity and high resolution of the pad is one of the researchers’ main achievements, says Patrick Baudisch, a researcher at the Hasso Plattner Institute, in Germany, and at Microsoft Research. Baudisch is currently collaborating with Perlin’s group on the IMPAD project. “The pad gives you an animated pressure picture but has only 20 connectors or so coming out of it,” he says. “This sounds like it’s not a big deal, but it makes it feasible to use it on very small mobile devices such as our nanoTouch,” a screen the size of a credit card that has touch sensitivity on the back and sides.

Bill Buxton, principal researcher at Microsoft, says that the NYU work is “interesting and distinct in a number of ways,” including in its ability to sense more than just a finger or a stylus. “You can use whatever best suits the task,” he says. Also, he notes that while the prototype is an opaque touch pad, the concept could easily be applied to forthcoming flexible displays, as the ink and the electrodes can be made transparent.

Perceptive Pixel’s Jeff Han agrees that capturing information about the amount of force applied to the screen is an important part of a touch interface. However, he notes that integrating such a sensor with a high-fidelity display is the hard part. Making sure that the touch interface and the display play well together is still a significant challenge.

Perlin says that he envisions the technology replacing capacitive touch screens, especially in mobile phones. Hospital beds and wheelchairs could also be equipped with IMPAD screens to indicate when pressure sores might occur. Construction materials could use the technology to monitor stress on buildings, and skinlike outer layers could be made for robots that can detect touch.

The researchers are currently in the first stages of forming a spinoff company to test the commercial possibilities of the technology.