Face Recognition Software Goes Public

In March, Silicon Valley startup Riya began offering a test version of software that can help users search through digital photo albums by using facial recognition software. Riya applies technology that can examine a photo, capture visual clues about faces and clothing, and also collect data encoded in the file, such as the time and date when it was created. The software combines all this information, giving users the ability to search photos by the people in them.

The process is quite simple: one uploads photos to the site and manually adds names to the faces, which trains the software to recognize people. The more "training" photos one uses, the more easily the software can match faces with names. Then, as additional photos are uploaded, the software automatically tags them with the names of the people in them.

To be sure, there's a novelty factor, says Robert Lowe, director of venture development and strategy at Carnegie Mellon University and CEO of Pittsburgh Pattern Recognition, which sells object and face recognition software for surveillance purposes. Riya's new tool will most likely drive people to its service -- which some liken to Flickr, the popular photo-sharing website. Unlike Riya, though, Flickr requires users to always manually input data about their photographs.

It may sound like just an advance in search capabilities, but allowing virtually anyone to use face recognition software is raising eyebrows among some privacy advocates. "The privacy issues are very complex and multifaceted," says Lowe, "it's a legitimate concern."

Consumer face recognition is a nascent industry, but for decades researchers at a number of universities -- Carnegie-Mellon, MIT, Michigan State University -- have been developing algorithms that can find and identify objects and faces in pictures and video. Additionally, the technology for picking out faces in a photo has been used in surveillance and law enforcement, with varying degrees of success and adoption, in recent years. Pittsburgh Pattern Recognition, for instance, sells software to the U.S. intelligence community that can pick out not only faces, but also cars and street signs, says Lowe.

Still, Riya (which reportedly drew interest from both Google and Microsoft last year) and another company, MyHeritage.com, are at the leading edge of this latest trend to put face-recognition technology in the hands of the general public.

Like most facial recognition software, Riya's technology can be separated into two parts: a scanning and metadata collection process, and a matching and tagging process.

The first scans an image and detects that a face is present; this is done as pictures are uploaded to the site. The software scans the picture, pixel by pixel, explains Burak Gokturk, the company's chief technology officer, looking for characteristic components of faces: shape and position of an eye, or color variation on the face. Once a face is found, if the picture is being used as a training photograph, the software draws a box around it so a user can identify it. If the software recognizes the face, it's matched with the appropriate name from other pictures in the collection.

But identifying individual faces is tricky, and the software can't find faces in all photos, says Gokturk. For instance, if an image is obscured, it's difficult to identify. "We most likely won't be able to detect your face if you paint it green," he says.