Wednesday, January 18, 2006

Repetez, en anglais, s'il vous plait

Most commercial language translation software is pretty bad -- but there may be a better way.

By Kate Greene

Traditionally, machine translation software has depended on algorithms that sort through thousands of grammar rules for the two languages to be translated, Knight says. The problem, he explains, is that so many rules need to be written manually, as do the exceptions to these rules, and inaccuracy creeps in when complex sets of rules contradict each other. "If you write the 5000th rule, sometimes you break things," Knight says.

With Language Weaver and his research at USC, Knight, as well as a handful of other researchers throughout the world, approach the problem differently. Instead of following rigid grammatical rules, Language Weaver matches correct words and phrases across languages based on the probability that such words and phrases are correct in a given context.

This statistical approach draws from a large number of examples from already translated documents, says Michael Collins, a computer engineer at MIT who uses the same method for a software application he's building to perform German-to-English translations. IBM pioneered this approach in the 1990s, he says, in part, by taking advantage of a huge database of Canadian parliamentary proceedings published in both French and English versions.

The statistical variety of machine translation not only produces better results than the traditional method, says Knight, but also the software is designed to continue to improve on its own. The more translated documents the software encounters, the more likely it will match phrases correctly. "A few years ago, for our Chinese and Arabic languages, all we could get was the basic topic of what an article was about," Knight says. "Now, the resolution is at the sentence level." [Continued on next page]

---

*Correction, January 18, 2006, 10:00 a.m. EST: In the original version of this story, we cited the following translation of the Someya Apple Farm website: "The apple orchard with big trees over 50 years old. The natural environment around Numata, with the huge temperture difference between day and night, creates uniquly delitious apple." In fact, this was an excerpt from Google's translation of the apple farm's own English version of its website, not from Google's translation of the original Japanese page. Therefore it was not a valid example of the poor quality of some machine translation algorithms. In the story we have now substituted Google's translation of the original Japanese site. Thanks to our readers for pointing out the error. - Eds.