Behavior-Monitoring Machines

A new breed of computer software keeps tabs on how people and objects act. To really work, though, these programs have to learn and adapt.

By Erik Sherman

November 12, 2003

He sees you when you're sleeping, he knows when you're awake. Ever wonder how Santa has time to keep tabs on the world while building toys? Maybe he's using behaviometric software-applications designed to understand the behavior of people and things and to react appropriately in order to provide security, facilitate health care, and enhance productivity.

In a limited sense, computers have always had to understand highly select behavior. A word processor, for instance, must know when people want to copy text blocks and when they want to delete them. But these user actions are highly constrained, with people only allowed to perform expected actions. The genius of behaviometric software is that it tackles more unpredictable actions, whether performed by people, animals, machines, or a combination of all three. What developers must do is measure and describe behavior in manageable ways while enabling computers to learn more about the targeted behavior over time through observation.

Researchers have known for many years that humans can extract essential characteristics of behavior quickly and easily. "In the 1960s, the psychologist Gunnar Johansson performed a series of famous experiments in which he attached lights to people's limbs and recorded videos of them performing different activities, such as walking, running, and dancing," says M. Alex O. Vasilescu, a researcher at New York University's Media Research Lab in New York and a doctoral candidate at the University of Toronto. People were asked to watch videos where only the lights were visible and then classify the activity. Usually, Vasilescu says, observers were able to "perform this task with ease, and they could sometimes determine gender and even recognize specific individuals." The difficulty scientists have is in figuring out how to view the task in a simpler and more essential form.

Story continues below

The process of paring away the unnecessary can take decades. Take, for example, a project at the Technion-Israel Institute of Technology in Israel, in which a pair of students wrote an application that can identify people with near 100 percent accuracy by their typing styles. "People have studied problems of trying to match keystroke sequence to a typist for almost 20 years," says Ran El-Yaniv, a Technion professor of computer science and expert in machine learning who was one of the supervising faculty. When the project started, El-Yaniv, says, software could make highly accurate identifications this ways only if the person was required to type a predetermined sequence of characters. Move outside that keyboarding script, and the programs couldn't keep up.

The Technion project found a way to ignore the actual content of the typing by concentrating on the number of milliseconds it took people to type a character. The team wrote software that monitors how quickly a typist pressed and released a key. Individual times are grouped to take slight variations into account. Researchers have also found that pressing a given key would affect the typing of the next letter. Because there were a finite number of key combinations, researchers could use existing algorithms to create a probabilistic model for an individual user. Someone whose typing did not fit the model was likely not the person. The tricky part was determining whether there was a fit with the model. It is called a single-class identification problem-like designing a system to know what a dog looks like, then showing it a cat and asking if that animal belongs to the class.