We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Rewriting Life

Sign Language

Translation software deciphers billboards and street signs.

In our global economy, the ability to understand languages other than one’s native tongue grows more important every day. That is why Jie Yang, a computer scientist at Carnegie Mellon University, and his colleagues have created software that reads Chinese signs and quickly translates them into English using nothing more than a palm-size computer equipped with a small camera.

“If you’re trying to get somewhere, it helps to be able to understand the signs around you,” says Yang, who plans to develop translation modules for Japanese, Arabic, and Korean if funding is available. “There’s a growing demand for products like this,” he explains.

The seemingly simple task of interpreting street signs actually represents a bundle of problematic processes-from machine vision to character recognition-with which researchers in computer science and artificial intelligence have been struggling for years. Yang chose some of the most promising software solutions and squeezed them into a handheld Pocket PC device. His system starts by scanning an image for hard edges-a sign’s borders. Next, it searches for cues such as contrasting, similar-size characters. Optical character recognition software “reads” the words, and other algorithms cluster them into plausible groupings.

This story is part of our March 2003 Issue
See the rest of the issue

Then comes the hard part: the actual translation. “Ambiguity is the biggest problem,” says Alex Waibel, director of Carnegie Mellon’s Interactive Systems Laboratory, where Yang is based. “Words can have so many different meanings in so many different languages in so many different situations.” That’s why a device that renders a literal translation can’t always do the job. If a Chinese sign meaning “Please do not touch” were translated word for word, for example, it would say, “Please do not move hand.”

To deal with such ambiguities, Yang considered three dominant machine-translation approaches, each with its own advantages. One, knowledge-based translation, requires humans to program rules that describe how a language’s words relate to one another. Slot Grammar, an IBM version of this approach, would break a sentence such as “John gave Mary a book” into the verb “gave” and three related words that fill slots: a subject (John), a direct object (book), and an indirect object (Mary). The IBM system then uses these relationships to search the knowledge base for similar relationships in the target language, and those relationships help generate a sensible translation.

At Microsoft Research in Redmond, WA, scientists rely on an example-based system that taps raw computing power rather than programmed rules. The Microsoft technology compares millions of words and phrases of translated text. Along the way, it generates an enormous bilingual dictionary of word relationships, phrases, and idioms. Then, in a process similar to sentence diagramming, it breaks target text into “logical forms” and finds the best-matching forms among those in the dictionary.

The third approach is statistical. Like Microsoft’s approach, it starts by crunching reams of word relationships. But unlike example-based systems, it scores the best translations using a statistical model that identifies the phrases and words in each language that properly match under a variety of circumstances.

Yang tried both the example-based and the statistical approach to see which translator would perform better. Right now, each delivers at least partially accurate translations 82 percent of the time, compared with 70 to 80 percent for most other translation systems, he says. Yang and other researchers agree that combining all three approaches should prove the most promising long-range strategy. “There’s no other way we can keep up,” says Steve Richardson, head of the machine translation project at Microsoft’s Natural Language Processing Group.

The harder researchers push toward 100 percent accuracy, unfortunately, the more stubborn the problem becomes. Machines will “never be as good as humans” at translation, says Yang. But, Waibel adds, they “are getting better all the time.”

AI and robotics are changing the future of work.  Learn from the humans leading the way at EmTech Next 2019.

Register now
More from Rewriting Life

Reprogramming our bodies to make us healthier.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivered daily

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.