Golle trained his program using 8,000 images collected from the same website. Through trial and error, his software gradually learned to tell cats and dogs apart, based on a statistical analysis of color and texture in each photo. The pink of the dogs’ tongues and the green of the cats’ eyes provided strong clues, Golle says, but it is only by studying color and texture information from so many images that his program could attack the problem. “Machine learning is very good at aggregating information,” Golle says.
However, although each individual picture was recognized 83 percent of the time, the full CAPTCHA test requires 12 pictures to be identified simultaneously, so the attack actually works only 10.3 percent of the time.
Golle says that an easy countermeasure would be for Asirra to present more pictures, which would further drive down the success rate of the attack. Microsoft did not respond to our requests for comment.
Despite all this progress, it’s unclear whether or not real spammers are currently using AI attacks against real CAPTCHAs. Websense Security Labs, in San Diego, has released reports about spammers cracking CAPTCHAs, but often this involves simply having low-paid workers solve CAPTCHAs manually.
Luis von Ahn, a computer scientist at Carnegie Mellon University, who helped coin the term CAPTCHA, says that it’s not clear that any common CAPTCHAs have been broken by machine attack in the real world. “I don’t know of anybody who’s thinking of getting rid of the CAPTCHA because it doesn’t work,” he says.
However, von Ahn notes that using humans comes at a cost. Even if workers are paid just $3 per 1,000 CAPTCHAs, that is expensive, he says, especially since most of the hacked Web mail accounts will be shut down soon after they begin to send out spam. So a truly automated attack would reduce the cost to spammers and greatly increase the number of successful attacks they could afford, he says.
But until computers start to get much smarter, CAPTCHA creators will always be able to implement a few simple tweaks to make a CAPTCHA much harder. “I do think there will be a day when, essentially, CAPTCHAs are going to be useless,” von Ahn says. “But I don’t think it’s this year, or next.”