How Spam is Improving AI

Very interesting article! It is fascinating that the Turing Test itself has become a practical issue. I'd like to point out a trend I have seen in AI, and use that to project the evolution of CAPTCHAs.

I took my first AI course in 1962 (a graduate EE course at MIT, taught by Prof James Slagle). At the time, the focus was on heuristics to "prune the choice tree".

Looking at progress in AI for the following 40 years, this approach was not where the successes were. Pragmatic AI successes stemmed not from pruning the tree, but rather from faster and cheaper computing that could afford to look deeper into the tree -- pruning not necessary. I don't know whether the new CAPTCHA attacks fit that description, but I suspect they do.

So what can we do to make better CAPTCHAs? Suggestions in the article, like identifying more pictures to reduce the probability of N hits, miss the point of history. That is at best a temporary expedient; as computing gets faster and cheaper, it will be necessary to identify more and more pictures -- a major nuisance to legitimate users.

Perhaps a better approach would be to look again at the thing that makes a Turing test such a high threshold: the unstructured richness of human intelligence. Suppose the user had no idea what kind of test would appear as the CATCHA? That is more in the spirit of the Turing test than the limited scope of a CATCHA today. This time you must tell a cat from a dog; the next login you must identify a state or a country from a map; after that, you must name a tune that plays on your speaker.

Of course, this "unstructured richness" has its problems as well. The article made the point that, as text is distorted more and more, people start to have as much trouble identifying it as computer programs do. By making the test less structured, we run the risk of some humans being unable to solve the puzzle. For instance, the map test that I mentioned assumes some proficiency with geography. I watch quiz shows that leave me appalled at the lack of geographical knowledge of too many contestants. As for the music identification, those same quiz shows impress on me that, while I may be a genius at pre-1960 music, I'm a complete dunce at post-1980 music that many contestants identify immediately.

So the choice of test is a challenge to find unstructured, unpredictable knowledge that is, at the same time, universal to humans. And the test must have unambiguous responses, so that the CATCHA program itself does not have to pass the Turing test.

DaveT

The problem with current CAPTCHA's is that they weren't a very good Turing test to begin with (using computer fonts to generate an image).

OCR technology has been around for a long time, and even after mutilating characters, the problem is we're still limited to only 60 or so "images" to choose from (a-z, A-Z, 0-9).
I posted an idea about the ideal Turing test, one that could be used to make a CAPTCHA that is theoretically unbreakable.
Read more about it here:

http://www.yuniti.com/BetterCaptcha

The idea is to use a hurdle in technology - image recognition - as a hurdle in cracking CAPTCHA's.  In essence, using google's image labeling platform and database to have users enter a word that describes the image.

Because the source of images is infinately large (millions of images), caching of results would not be an option.

Better yet, words could be limited to the English language for english sites, making it more difficult to outsource to "CAPTCHA typers" in China/Russia/etc.

What will happen to the internet when AI is solved? What if no test could fool our future intelligent machines? How will web sites distinguish between humans and machines? I see this as a future business opportunity. We will need some form of personal authentication service. Maybe, the website's own AI will be used to authenticate users via their webcams and/or computer microphone.

AI as your thinking about it will never be solved. Yes machines will become more human, but humans will also get more like machines. We will eventually meet some ware in the middle, and the entire concept of AI will become ridiculous.