Face Recognition: Clever or Just Plain Creepy?

Continued from page 1

Look-alike: Once a person has been identified, iPhoto will try to find other images containing his or her likeness. Credit: Simson Garfinkel and Beth Rosenberg

iPhoto '09 is certainly the friendlier of the two. The first time you run iPhoto, it searches out all the faces in your photo library; this took about four hours on a dual-core iMac. Next, you click on a photo of someone you know; click "Name" and fill in the text box beneath your subject's face. iPhoto will run through your photo library looking for other photos of the same person. (The recognition seems to be based on features inside a recognition box that is bounded by the left and right temples, eyebrows, and chin.)

Overall, iPhoto does a surprisingly good job finding a bunch of photos of the person you've selected and "named." But in the process, it finds photos of other people as well. So your next task is to tell iPhoto which photos it got right and which are wrong. iPhoto uses this information to update its mathematical models. It then looks back through your photo library for other photos of the same person. If it can't find any, you can manually point one out to give iPhoto another starting point; it will then seek out more. You can also click on a photo and ask iPhoto to try to figure out who is in the picture; if you confirm iPhoto's guess, the model gets better still.

We were astonished at how good iPhoto was at finding photos of our kids. Amazingly, iPhoto could even distinguish between our identical twins. (The trick is that one of them has a face that's a bit thinner and taller than the other's.) We were disappointed, however, that it found many more photos of one twin than the other, although we photograph both in equal numbers--and often in the same shot. A study of their photographs revealed something that we hadn't noticed, but iPhoto had: one twin always looks directly at the camera, but the other tends to tilt his head away, and iPhoto's face recognition doesn't work if the program only sees one eye. We also have lots of photos of kids in face paint. iPhoto found practically none of those, except for when the paint was confined to the middle of the child's forehead--which is outside its recognition box.

It's tempting to read a lot into iPhoto's recognition system. Searching for photos of Beth produced lots of photos of Simson's ex-girlfriends. It's tempting to say that iPhoto knows what Simson likes, but this could also be a bias in our test corpus: pick random photos out of Simson's library, and you're sure to find a bunch of his ex-girlfriends.

iPhoto was also surprisingly good at finding photos of our cats, especially the ones with white or orange fur. Unfortunately, it failed to find the tabbies--presumably, facial features are harder to distinguish when the eyes are the same color as the cheeks. And iPhoto does a startling job at finding and recognizing faces in shadows and other low-contrast situations. That's because iPhoto cranks up the contrast between face and background, presumably to make it easier to get out the features.