Thermal Camera Turns Many Things into Interactive Surfaces

The ways we interact with smartphones and laptops are well-established, but it’s not yet known whether voice, touch, or gestures will emerge as the best way to use wearable devices like head-worn computers, which may not have touch screens and aren’t suited to mousing around.

The augmented-reality company Metaio is experimenting with a wholly different method called Thermal Touch: it detects the traces of heat left behind when you touch something, which could enable you to use all kinds of things—walls, books, and perhaps even toys—as interactive touch surfaces.

The technology combines two kinds of cameras. A thermal camera detects the residual heat from a touch (as opposed to what happens when a finger just approaches an object). Simultaneously, a camera that registers visible light determines the location of the object being touched, so that Metaio’s software can map that touch on the object in three dimensions. Both cameras are combined in a device called the Optris PI 200 that Daniel Kurz, head of Metaio’s advanced technologies group, straps to a computer or a tablet to test the technology.

You won’t find a thermal camera in, say, Google Glass today, but Kurz is optimistic that this kind of sensor will make its way into smartphones and wearable computers in the near future, much like sensors such as accelerometers and magnetometers have done. In January, Flir released a thermal imaging camera module that manufacturers can add to smartphones and tablets. (The company also plans to release a $349 iPhone case equipped with the thermal sensor in July.)

In a demonstration that Kurz gave me over a Skype video chat from Metaio’s Munich headquarters, the object serving as his touch surface was a piece of paper printed with a pattern that made it look like a little stone wall. When Kurz and I looked at the paper through a computer display, we could see a virtual keypad displayed on it that stuck to its spot even if the cameras moved a bit. He reached out a finger, touching the paper in front of him to punch in numbers; each time he hit one, a blue oval popped up atop the number on the display in recognition of the touch.

Kurz says Thermal Touch can work across a range of temperatures, as long as the surface you’re touching is either hotter or colder than your body. The bigger the difference between you and what you’re touching, though, the easier it is to track.

Kurz says Thermal Touch will work on many surfaces—with the exception of metal, which quickly disperses heat—assuming they have features or patterns that can be tracked. More information will emerge in a paper to be presented in September at the International Symposium on Mixed and Augmented Reality.  

There are many technological issues to fix. Metaio is working on reducing Thermal Touch’s latency, for example; it currently notes a touch after it has occurred. And right now the prototype detects touches only on essentially flat surfaces or three-dimensional objects that the computer knows in advance.