A Camera App that Gets to Know Your Friends
Klik uses facial-recognition technology to identify who you’re shooting.
For a couple of years, Face.com has offered websites and apps a facial-recognition service that can identify people in photographs, figure out how many faces there are in the picture, which is male or female, and how old they might be. Facebook is widely believed to be one of its customers, though Face.com refuses to comment on their relationship.
But with mobile photo sharing gaining popularity, Face.com CEO Gil Hirsch says the company—which started out by building face-finding and -tagging Facebook apps—wanted to build a mobile app that, unlike existing apps that use the company’s technology, would give users real-time feedback about who their cell-phone camera is pointed at. It has done so with Klik, a free smart-phone camera app with the ability to recognize faces in real time and, if it can’t recognize them, learn who it is you’re shooting.
Originally released in January, the latest version of Klik rolled out on Thursday for the iPhone (an Android version is coming, but Face.com won’t say when). It’s a bit like Instagram, but with an AI twist.
Klik connects to your Facebook account and scans tagged photos of your friends, a process that can take a few hours. Once it’s ready, though, Klik can determine who you’re looking at before you’ve pressed the shutter. It also recognizes faces in photos that are already stored on your phone.
You can take photos, dress them up with simple filters, annotate them with messages and location data, and share them on Facebook or Twitter, through e-mail, or with other Klik users.
Hirsch can think of all kinds of applications for his company’s facial-recognition technology, from organizing family photos to enabling a service that could tell you more about whoever is standing in front of you.
Basically, Hirsch says, Klik picks up the presence of faces on the screen by scanning the video feed on the phone, frame by frame and pixel by pixel, searching for specific patterns it thinks make up a face. It tracks the face so it can still identify it even if it’s in profile. Klik sends that visual data to the company’s servers for processing, and returns with its best guess as to who’s in the picture.
Going through a number of photos saved on my iPhone, Klik did a decent job at picking different friends. It did best with close-up shots, as long as there were no more than a few people in the frame, and seemed to have trouble with shots where people’s eyes were closed or their faces were obscured by sunglasses or hats. It also had trouble when people were far away or not looking right into the camera.
When you focus the Klik viewfinder on a person (or several people), the name of the person Klik thinks it’s seeing quickly pops up on the screen near that person’s head. But if the name is incorrect—perhaps because you’re not Facebook friends with them, or because you are but that person doesn’t have a lot of photos of themselves on Facebook—you can hit the “learn” button to teach Klik who it is. Klik will ask you to center the person’s head on the screen and then try again to determine who it is. If it still doesn’t make a match, you can search for the right person in your address book or type in their name to tag a photo. The next time you take a photo of that person with Klik, it should be better able to guess their name.
Klik sometimes makes understandable mistakes. It can identify me correctly with 95 percent certainty, but it also thinks there’s a chance I could be my mom, who does look like me, or an old friend from New York to whom I bear a decent resemblance.
Alessandro Acquisti, an associate professor at Carnegie Mellon University who has studied facial-recognition software, is concerned about possible privacy issues that could crop up over time if apps like Klik become more widely used and accepted. What happens, he wonders, if a Klik user’s Facebook friend doesn’t want to be recognized by the app? (As it turns out, Facebook’s privacy settings allow you to turn off your friends’ ability to share your photos with outside apps that are pulling data from Facebook.)
Acquisti does think that allowing users to correct errors is very powerful, though, and could dramatically decrease false identifications. “They’re effectively enlisting the users to improve their own algorithms,” he says.
Become an Insider to get the story behind the story — and before anyone else.