These large, thin OLED screens can be controlled with voice and gesture controls. Credit: Samsung

While the picture quality and screen size is likely to be the most immediately striking thing about the 55-inch Super OLED TV Samsung unveiled at the Consumer Electronics Show in Las Vegas this week, some of the less visible technologies in the TV could prove to be more significant in the long term. The Super OLED is, like virtually all new high-televisions, a smart TV, capable of running local applications and accessing the Internet. What's new is Samsung's approach to the thorny challenge of the smart TV interface, using motion and voice control.

The interface problem arises because TV is what's been dubbed as a "lean back" experience. Most users prefer a simple remote control that allows them to turn the TV on and off, select a channel, and adjust the volume. But navigating a video streaming service, or sending a tweet, are relatively complex activities typically associated with the "lean forward" experience of computers where at least a keyboard (even if only an on-screen tablet one) is available.

One solution is to make the remote control considerably more complicated, incorporating a complete keyboard, which was the tack initially taken by Sony (among others) with its Internet TV, created in partnership with Google. But the approach failed to gain traction. Far more promising has been the idea of using gestural interfaces, which would allow users to control devices without the need for any remote control at all—for example, a user could simply sweep an arm through the air to scroll through a page of search results. The Super OLED uses a built-in camera to capture motion in the foreground to control its smart TV services, supplementing the motion controls with voice controls picked up by a pair of built-in microphones. And it doesn't seem like it'll be long before other manufacturers incorporate similar features: at its CES keynote presentation, Microsoft discussed how it intends to adapt the Kinect system originally developed for its XBox 360 game console for interactive TV applications.

Science purists might find much to complain about in the newest installment of the Iron Man franchise, starring Robert Downey Jr. Admittedly, Tony Stark "creates an element," and heroes and villains alike seem able to break into high-level computer systems with little more than the wave of an iPhone look-a-like. But I expect computer scientists and designers will be impressed by the movie's natural user interfaces.

There's a long tradition of the interfaces envisioned in movies becoming research projects in real life. For years, techies have chased the "Minority Report interface" inspired by a scene in the Tom Cruise action flick in which the main character does his police work by donning a pair of gloves and diving into a hands-on manipulation of his data. Johnny Lee, a researcher in Microsoft's hardware division, gained acclaim for hacking together a version of the interface using a Nintendo Wii. The company Oblong has been working for years on the g-speak interface, a slicker implementation of the same concept.

Earlier this year at South By Southwest Interactive, a computing and design conference in Austin, Texas, I noticed that Iron Man had stolen the designers' hearts. I attended multiple panels where designers showed clips from the first film and described what it would take to make that vision a reality.

Iron Man 2 had several enticing scenes of the main character interacting with his computer. Leaving aside the computer's improbable level of intelligence, Stark interacts with it through sophisticated voice recognition. This is a feature taken for granted in almost all science fiction films. More intriguing are the suggestions for gestural interfaces.

When Stark is mid-design and doesn't like what he's working on, he grabs it off the projection and throws it into the trash. This allows him to throw away a virtual idea with as much expression as can be used with an idea that's taken physical form. At one point, he performs a 3-D scan of a physical model because he wants to create a version that he can manipulate easily. He lifts a projected image off the physical object and becomes able to spin it, change its size, and alter it with flicks of the hand.

What makes the interface look most attractive is how physically involved Stark becomes in design. With the power of voice and gesture combined, Stark is able to give small, quiet commands when contemplative, and become more expansive and hands-on when excited. The vision of a computing device that's able to adapt so smoothly to the user's mood and circumstance is compelling to say the least.

Stark's natural interface also displays a problem that designers still have to overcome with this type of design. Watching closely, it's clear that it's ambitious to call such an interface "natural." Stark knows an entire vocabulary of gestures that would not be obvious to someone approaching the interface for the first time. For natural user interfaces to take off in the real world, designers will have to convince users that learning this new method of interaction provides value that can't be had with keyboard and mouse.

Tired of constantly pressing the wrong buttons on a too-sensitive, tiny touch screen? Researchers at the Ishikawa Komuro Laboratory at the University of Tokyo have created a camera system that attaches to a mobile device to let it track mid-air finger movements and translate those movements into commands.

The camera recognizes if the finger is moving toward it and away from it and at what speed. This lets a user move a mouse, zoom and scroll pictures, digitally draw and type, without ever touching the screen.

Phones that recognize gestures could help users avoid fumbling around on touch screens, or alleviate the physical caused by from typing. Microsoft's Project Natal will use a similar, full-body motion-tracking interface for gaming.