There could be a revolution brewing in billboard
advertising. Instead of simply presenting a static image, why not let people
interact with the advertisement? This is the vision of electronics giant
Samsung and interactive advertising company Reactrix
Systems. The two companies have partnered to bring 57-inch interactive
displays to Hilton hotel lobbies by the end of the year. These displays can “see”
people standing up to 15 feet away from the screen as they wave their hands to
play games, navigate menus, and use maps.
With the buzz surrounding the Wii, the iPhone, and
Microsoft’s Surface, “people are more open and ready to interact using their
hands and gestures,” says Matt Bell, chief scientist and founder of Reactrix. It’s
easy to see how a gesture-based
interface might work well for video games and virtual worlds, and certainly
companies such as Belgian startup Softkinetic
make systems for those very needs. But Reactrix is aiming for the out-of-home
advertising market, traditionally dominated by large static displays like
billboards. Founded in 2001, Reactrix has some experience already: today, its
interactive floor displays attract crowds in shopping centers across the
country.
Advertisement
The basic idea behind Reactrix’s system, and even low-end
gesture-based technologies such as the Sony PlayStation Eye, is to use a camera
to detect a person’s body, and then use computer vision algorithms to make
sense of the images. Reactrix and Softkinetic systems differ from the
PlayStation Eye, however, in that they record 3-D information as opposed to
just two-dimensional information. There are many types of cameras that
can capture 3-D scenes, says Bell,
but in its current models made with Samsung, the company is using a
stereoscopic camera with two lenses. Next to the camera is an infrared light
that projects an invisible pattern onto the people in front of the screen. Each
lens captures a slightly different view of what’s going on, and, based on the
disparity in the images, the system can distinguish distance down to a fraction
of an inch. Bell
adds that the projected light pattern helps the system’s accuracy in uneven
lighting.
This story is only available to subscribers.
Don’t settle for half the story.
Get paywall-free access to technology news for the here and now.
When the camera collects the information, it automatically
dumps it into a specialized processor to analyze the depth data, bypassing
software that wouldn’t be able to compute fast enough. “Once that’s done, we
have a full-depth image showing the distance to every object,” Bell says. At this point,
Reactrix’s unique algorithms take over. One of the differentiating factors
between Softkinetic and Reactrix is that the former focuses on the detailed
motion of parts of a single body, whereas the latter strives to disambiguate
people and objects. Bell
doesn’t provide details, but he says that the code is designed to figure out
scenarios such as when people are holding hands, or if people are standing
shoulder to shoulder.
On top of the hardware and algorithms, Bell says, Reactrix is also thinking about
the best design for the user interface. As with touch-screen technology,
gesture-based interactions have been toyed around with before, but it’s still
unclear what sort of interface would work best for most people. There are a few
interactions that lend themselves well to a gesture interface, such as a boxing
game or sliding pictures across a screen. However, engineers still haven’t
figured out the best way for people to interact with a virtual button, for
instance. It may seem trivial, but it’s unclear how to press a button when
there’s nothing to touch. “There’s an exciting opportunity here to create the
standard gestural interaction with displays,” says Bell. “We want to be at the forefront of
creating that.”
Regarding the forthcoming Hilton displays, Bell says he expects that travelers will be
able to play games that relate to local attractions and navigate menus for more
information. In this way, he says, people have fun interacting with
advertisements, instead of just passively flipping through a brochure.
With its floor displays already available in U.S. shopping
centers, “Reactrix has proven the value of interactive marketing solutions for
use in public spaces and, specifically, in use with crowds, for which it is
difficult to track individual people’s body movements,” says Michel Tombroff,
CEO of Softkinetic. He suspects that the market for gesture-based technology
will grow in the coming years, thanks in part to the falling price of 3-D
cameras.
The engineers who build these cameras and computer vision
systems have made great strides in recent years, says Scott Klemmer, a
computer-science professor at Stanford
University. “Cheap
cameras and sensing [systems] are going to usher in a new genre of user
interfaces,” he says.
Bell
says that the falling price and shrinking size of these cameras is one of the
main reasons that his company partnered with Samsung. The display company, he
says, should be able to find a compact and cost-effective way to integrate the
camera technologies, Reactrix processors, and algorithms into commercial
displays that can have a home outside a Hilton hotel lobby.