Wednesday, January 18, 2006

Repetez, en anglais, s'il vous plait

Most commercial language translation software is pretty bad -- but there may be a better way.

By Kate Greene

While the quality of computer-rendered translations has improved greatly over the past 20 years, some results are still just as grammatically goofy as the instructions on a chopstick wrapper. Take, for example, a website for a Japanese apple farm that was converted to English using Google's automatic translation service:

"The Someya apple garden it will pass very! It is planted in 1954, furthermore even now, it exceeds tree's age 50 year old prosperously, large - coming the tree abnormal play alligator apple is fructified. The tasty apple where temperature difference of day and night tightened to be extreme Gunma prefecture Numata city which four seasons are clear large the nature, hard is created."*

Yes, the big picture gets across, but much is lost by Google's Japanese-to-English translation algorithm. Google has been offering its translation feature for a number of years, as has the Canadian-based internet company Babel Fish. More recently, though, commercial software developers have begun exploring translation beyond a static webpage or electronic document and are applying the technology to the real-time Internet instant-messaging conversations. Earlier this month, AvMedia released an instant-messaging translator designed to make chatting with friends who speak German, Spanish, French, Italian, and Portuguese easier for English speakers, and vice versa (French can also be translated to German, and German to French).

But all of this software still lacks sufficient accuracy to be useful in demanding situations, such as business negotiations or military planning. This is probably because most commercial software follows a traditional approach to machine translation, says Kevin Knight, computer scientist at the University of Southern California’s Information Sciences Institute (ISI) and co-founder of the California-based company Language Weaver.

Traditionally, machine translation software has depended on algorithms that sort through thousands of grammar rules for the two languages to be translated, Knight says. The problem, he explains, is that so many rules need to be written manually, as do the exceptions to these rules, and inaccuracy creeps in when complex sets of rules contradict each other. “If you write the 5000^th rule, sometimes you break things,” Knight says.

With Language Weaver and his research at USC, Knight, as well as a handful of other researchers throughout the world, approach the problem differently. Instead of following rigid grammatical rules, Language Weaver matches correct words and phrases across languages based on the probability that such words and phrases are correct in a given context.

This statistical approach draws from a large number of examples from already translated documents, says Michael Collins, a computer engineer at MIT who uses the same method for a software application he's building to perform German-to-English translations. IBM pioneered this approach in the 1990s, he says, in part, by taking advantage of a huge database of Canadian parliamentary proceedings published in both French and English versions.

The statistical variety of machine translation not only produces better results than the traditional method, says Knight, but also the software is designed to continue to improve on its own. The more translated documents the software encounters, the more likely it will match phrases correctly. “A few years ago, for our Chinese and Arabic languages, all we could get was the basic topic of what an article was about,” Knight says. “Now, the resolution is at the sentence level.” [Continued on next page]

---

*Correction, January 18, 2006, 10:00 a.m. EST: In the original version of this story, we cited the following translation of the Someya Apple Farm website: “The apple orchard with big trees over 50 years old. The natural environment around Numata, with the huge temperture difference between day and night, creates uniquly delitious apple.” In fact, this was an excerpt from Google's translation of the apple farm's own English version of its website, not from Google's translation of the original Japanese page. Therefore it was not a valid example of the poor quality of some machine translation algorithms. In the story we have now substituted Google's translation of the original Japanese site. Thanks to our readers for pointing out the error. - Eds.