Skip to Content

Thermal Camera Turns Many Things into Interactive Surfaces

Could heat-tracking be the key to interacting with wearable computers?
June 13, 2014

The ways we interact with smartphones and laptops are well-established, but it’s not yet known whether voice, touch, or gestures will emerge as the best way to use wearable devices like head-worn computers, which may not have touch screens and aren’t suited to mousing around.

Metaio’s Thermal Touch prototype
Press here: Metaio’s Thermal Touch prototype uses a thermal camera to sense touch and a regular digital camera to track where the touch occurred, allowing many objects to serve as interactive touch surfaces.

The augmented-reality company Metaio is experimenting with a wholly different method called Thermal Touch: it detects the traces of heat left behind when you touch something, which could enable you to use all kinds of things—walls, books, and perhaps even toys—as interactive touch surfaces.

The technology combines two kinds of cameras. A thermal camera detects the residual heat from a touch (as opposed to what happens when a finger just approaches an object). Simultaneously, a camera that registers visible light determines the location of the object being touched, so that Metaio’s software can map that touch on the object in three dimensions. Both cameras are combined in a device called the Optris PI 200 that Daniel Kurz, head of Metaio’s advanced technologies group, straps to a computer or a tablet to test the technology.

You won’t find a thermal camera in, say, Google Glass today, but Kurz is optimistic that this kind of sensor will make its way into smartphones and wearable computers in the near future, much like sensors such as accelerometers and magnetometers have done. In January, Flir released a thermal imaging camera module that manufacturers can add to smartphones and tablets. (The company also plans to release a $349 iPhone case equipped with the thermal sensor in July.)

In a demonstration that Kurz gave me over a Skype video chat from Metaio’s Munich headquarters, the object serving as his touch surface was a piece of paper printed with a pattern that made it look like a little stone wall. When Kurz and I looked at the paper through a computer display, we could see a virtual keypad displayed on it that stuck to its spot even if the cameras moved a bit. He reached out a finger, touching the paper in front of him to punch in numbers; each time he hit one, a blue oval popped up atop the number on the display in recognition of the touch.

Kurz says Thermal Touch can work across a range of temperatures, as long as the surface you’re touching is either hotter or colder than your body. The bigger the difference between you and what you’re touching, though, the easier it is to track.

Kurz says Thermal Touch will work on many surfaces—with the exception of metal, which quickly disperses heat—assuming they have features or patterns that can be tracked. More information will emerge in a paper to be presented in September at the International Symposium on Mixed and Augmented Reality.  

There are many technological issues to fix. Metaio is working on reducing Thermal Touch’s latency, for example; it currently notes a touch after it has occurred. And right now the prototype detects touches only on essentially flat surfaces or three-dimensional objects that the computer knows in advance.

Keep Reading

Most Popular

transplant surgery
transplant surgery

The gene-edited pig heart given to a dying patient was infected with a pig virus

The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.

open sourcing language models concept
open sourcing language models concept

Meta has built a massive new language AI—and it’s giving it away for free

Facebook’s parent company is inviting researchers to pore over and pick apart the flaws in its version of GPT-3

Muhammad bin Salman funds anti-aging research
Muhammad bin Salman funds anti-aging research

Saudi Arabia plans to spend $1 billion a year discovering treatments to slow aging

The oil kingdom fears that its population is aging at an accelerated rate and hopes to test drugs to reverse the problem. First up might be the diabetes drug metformin.

images created by Google Imagen
images created by Google Imagen

The dark secret behind those cute AI-generated animal images

Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.