Thermal Camera Turns Many Things into Interactive Surfaces

Could heat-tracking be the key to interacting with wearable computers?

Wearable gadgets offer new ways to use technology in daily life, but we need better ways to control them.

The ways we interact with smartphones and laptops are well-established, but it’s not yet known whether voice, touch, or gestures will emerge as the best way to use wearable devices like head-worn computers, which may not have touch screens and aren’t suited to mousing around.

Metaio’s Thermal Touch prototype
Press here: Metaio’s Thermal Touch prototype uses a thermal camera to sense touch and a regular digital camera to track where the touch occurred, allowing many objects to serve as interactive touch surfaces.

The augmented-reality company Metaio is experimenting with a wholly different method called Thermal Touch: it detects the traces of heat left behind when you touch something, which could enable you to use all kinds of things—walls, books, and perhaps even toys—as interactive touch surfaces.

The technology combines two kinds of cameras. A thermal camera detects the residual heat from a touch (as opposed to what happens when a finger just approaches an object). Simultaneously, a camera that registers visible light determines the location of the object being touched, so that Metaio’s software can map that touch on the object in three dimensions. Both cameras are combined in a device called the Optris PI 200 that Daniel Kurz, head of Metaio’s advanced technologies group, straps to a computer or a tablet to test the technology.

You won’t find a thermal camera in, say, Google Glass today, but Kurz is optimistic that this kind of sensor will make its way into smartphones and wearable computers in the near future, much like sensors such as accelerometers and magnetometers have done. In January, Flir released a thermal imaging camera module that manufacturers can add to smartphones and tablets. (The company also plans to release a $349 iPhone case equipped with the thermal sensor in July.)

In a demonstration that Kurz gave me over a Skype video chat from Metaio’s Munich headquarters, the object serving as his touch surface was a piece of paper printed with a pattern that made it look like a little stone wall. When Kurz and I looked at the paper through a computer display, we could see a virtual keypad displayed on it that stuck to its spot even if the cameras moved a bit. He reached out a finger, touching the paper in front of him to punch in numbers; each time he hit one, a blue oval popped up atop the number on the display in recognition of the touch.

Kurz says Thermal Touch can work across a range of temperatures, as long as the surface you’re touching is either hotter or colder than your body. The bigger the difference between you and what you’re touching, though, the easier it is to track.

Kurz says Thermal Touch will work on many surfaces—with the exception of metal, which quickly disperses heat—assuming they have features or patterns that can be tracked. More information will emerge in a paper to be presented in September at the International Symposium on Mixed and Augmented Reality.  

There are many technological issues to fix. Metaio is working on reducing Thermal Touch’s latency, for example; it currently notes a touch after it has occurred. And right now the prototype detects touches only on essentially flat surfaces or three-dimensional objects that the computer knows in advance.

Uh oh–you've read all five of your free articles for this month.

Insider Online Only

$19.95/yr US PRICE


From the latest smartphones to advances in quantum computing, the hardware behind today's digital age is rapidly changing.

You've read of free articles this month.