In March, Silicon Valley startup Riya began offering a test version of software that can help users search through digital photo albums by using facial recognition software. Riya applies technology that can examine a photo, capture visual clues about faces and clothing, and also collect data encoded in the file, such as the time and date when it was created. The software combines all this information, giving users the ability to search photos by the people in them.

The process is quite simple: one uploads photos to the site and manually adds names to the faces, which trains the software to recognize people. The more “training” photos one uses, the more easily the software can match faces with names. Then, as additional photos are uploaded, the software automatically tags them with the names of the people in them.

To be sure, there’s a novelty factor, says Robert Lowe, director of venture development and strategy at Carnegie Mellon University and CEO of Pittsburgh Pattern Recognition, which sells object and face recognition software for surveillance purposes. Riya’s new tool will most likely drive people to its service – which some liken to Flickr, the popular photo-sharing website. Unlike Riya, though, Flickr requires users to always manually input data about their photographs.

It may sound like just an advance in search capabilities, but allowing virtually anyone to use face recognition software is raising eyebrows among some privacy advocates. “The privacy issues are very complex and multifaceted,” says Lowe, “it’s a legitimate concern.”

Consumer face recognition is a nascent industry, but for decades researchers at a number of universities – Carnegie-Mellon, MIT, Michigan State University – have been developing algorithms that can find and identify objects and faces in pictures and video. Additionally, the technology for picking out faces in a photo has been used in surveillance and law enforcement, with varying degrees of success and adoption, in recent years. Pittsburgh Pattern Recognition, for instance, sells software to the U.S. intelligence community that can pick out not only faces, but also cars and street signs, says Lowe.

Still, Riya (which reportedly drew interest from both Google and Microsoft last year) and another company, MyHeritage.com, are at the leading edge of this latest trend to put face-recognition technology in the hands of the general public.

Like most facial recognition software, Riya’s technology can be separated into two parts: a scanning and metadata collection process, and a matching and tagging process.

The first scans an image and detects that a face is present; this is done as pictures are uploaded to the site. The software scans the picture, pixel by pixel, explains Burak Gokturk, the company’s chief technology officer, looking for characteristic components of faces: shape and position of an eye, or color variation on the face. Once a face is found, if the picture is being used as a training photograph, the software draws a box around it so a user can identify it. If the software recognizes the face, it’s matched with the appropriate name from other pictures in the collection.

But identifying individual faces is tricky, and the software can’t find faces in all photos, says Gokturk. For instance, if an image is obscured, it’s difficult to identify. “We most likely won’t be able to detect your face if you paint it green,” he says.

The second part of face recognition involves matching unknown faces to those already identified. This relies on both math and the way that people take photos. For instance, it’s common for a user to take 10 or more photos at a party, and for the same people to show up multiple times. Riya takes advantage photo groupings to supplement its face analysis algorithms, Gokturk says.

In addition to scanning photos for faces, the software also notes characteristics of clothes, such as color, texture, and shape, and links those with faces. “At an event, you rarely change the shirt you’re wearing,” says Gokturk.

The software also looks for pictures with people together, such as a husband and wife, along with the date the photo was taken, to help the software take shortcuts in identifying people. For instance, if an individual is facing directly at the camera in one photo, but turned slightly, hiding some features, in another taken a second later, it’s still likely that it’s the same person.

By integrating such clues, Riya has developed face recognition software that may not be the most accurate, but can do the job with a degree of accuracy that could be “good enough” for consumers, says Stan Bileschi, a computer vision researcher at MIT.

Once the software is trained – Riya recommends uploading 500 photos initially – its able to correctly identify people 80 percent of the time, Gokturk says. He adds that the company is constantly improving its product, as well as exploiting social-networking capabilities of the site to improve accuracy. For instance, if your friends have already identified people in pictures, you can use their tagged information to identify common friends.

Given all the headline news recently about the National Security Agency amassing a database of information about U.S. citizens (or at least phone calls), privacy issues are highly visible right now. And with facial recognition tools becoming available, some worry that unethical individuals could use them to carry out crimes such as identity theft or cyber-stalking. “This is a legitimate concern – but face recognition technology like Riya should be the least of their worries,” says MIT’s Bileschi. “Remember that every time you do something on the Internet, like search, every time you make a purchase in any form other than cash, and every time you use your fast-lane pass to get on the highway faster, your individual presence is being recorded…Face recognition technology is not nearly as accurate or as pervasive as these forms of personal tagging.”

Additionally, Riya has rules that restrict access to photos. For example, you can decide if you want photos to be viewable publicly or only to you and some designated others. Or worried about being recognized in the background of a stranger’s photo? Riya’s software only tags people that one has trained it to recognize.

It’s important that companies offering this technology establish rules and restrictions on its use, says Carnegie Mellon’s Lowe. In the end, the concern may not stem from a particular application, however, but from the management of all the online databases with photos. Currently, Mellon estimates, most people know if they are in these databases. But with more online repositories storing photos and other personal information, he says, “everyone needs to be more vigilant about managing the data that’s out there.”