From these side-by-side comparisons, the software learns to translate between the languages – extracting statistical patterns that indicate that a particular grouping of words in Arabic, say, tends to correspond to certain words in English. The system translates phrase by phrase, so if it encounters the words “interest rate,” it will associate them with banks and finance, not curiosity and speed. And the machine-learning approach means the translations should improve with time. “The more data you add, the better the performance will be,” says Knight.
Until recently, this statistical approach, which has roots in 1940s wartime cryptography, was too slow to be useful. But on a modern PC, Language Weaver’s software can translate 5,000 words a minute at state-of-the-art accuracy levels; on a network of servers, it can handle up to 500,000 words a minute.
The names of the company’s U.S. government customers are tightly under wraps, but Benjamin says feedback from the intelligence community has been overwhelmingly enthusiastic and that the technology “played an important role in a mission that saved lives.”
But human translators won’t lose their jobs anytime soon. To accurately do some translations – of, say, technical material – Language Weaver’s system must be trained on similar texts translated by hand. And some experts are skeptical, since the statistical approach hasn’t solved the deeper problem of getting computers to understand natural language.
“It is good for the field that companies like Kevin’s succeed,” says Sergei Nirenburg, a machine translation expert at the University of Maryland, Baltimore County. But, he adds, “you will still have to file the product under ‘Good Uses for Crummy Machine Translation.’”
Nevertheless, Language Weaver has been profitable since late 2003 and is now investigating business applications. At ISI, Knight and Marcu are testing software for a handheld translator that can handle questions and answers between doctors and patients. An outside company is also developing a real-time Arabic-English translator for instant messaging built around the Language Weaver software.
But the biggest market may be multilingual search: typing a query in English could bring up scores of previously invisible foreign websites that could be translated into English. In five years, says Knight, expect to be able to read any Web page in practically any language. “You’ll wonder how you ever did without it,” he says. “But you’ll still laugh at the translation mistakes.”