Will Knight

A View from Will Knight

Twitter’s Success Offers Hope for Crafty Human Brainpower

The way humans assist Twitter’s search and advertising algorithms shows that machines still need a helping hand in some important areas.

  • November 7, 2013

Twitter’s listing on the NYSE arguably reflects renewed faith in human traders after Facebook’s error-strewn debut on the more heavily computerized NASDAQ in May 2012. But there’s another, more subtle, reason why Twitter’s success should warm the hearts of those of us who still exist in meat-space: Twitter’s search and advertising system relies on human computers as well as Big Data algorithms.

Human help: The original Mechanical Turk

Here’s how that works. When there’s a sudden surge in activity related to a new word or phrase, Twitter’s software collect those strings so that they can be searched for, or matched with ads, but it also automatically feeds them to humans workers to attach appropriate meaning. These workers are employed via Amazon’s Mechanical Turk, a crowdsourcing platform that allows hard-to-compute tasks to be divided up and allocated to humans (the service was named after the chess-playing “automaton” pictured above, which defeated its opponents not using ingenious engineering but thanks to a skilled human player concealed beneath the chess board).

Twitter requires human help because new information often emerges rapidly, and the meaning of that information may be complex, convoluted, and entangled in real-world context. A great example is the phrase #bindersfullofwomen, which gained momentum during the second presidential debate last July. Algorithms that might normally help you find relevant tweets—or might automatically serve relevant advertisements—by looking for clearly relevant strings such as “#Obama,” “#Romney,” “#PresidentialDebate,” simply cannot make sense of this kind of information. Algorithms might match tweets referencing #bindersfullofwomen to ads for Staples of Office Depot, but humans who just watched the debate should know better.

This trend may turn out to be an important in other areas, especially since some experts predict that increased automation will destroy many more jobs than it creates in certain sectos (see “How Technology is Destroying Jobs”).

It’s a trend that I first became aware of in robotics, where some researchers have found that combining human workers abilities with those of robots can be more efficient than using either humans or robots alone (see “Smart Robots Can Now Work Right Next to Auto Workers” and “This Robot Could Transform Manufacturing”).

How human abilities could be merged with information technologies is an even more fascinating to consider. And it’s an idea that actually stretches back for decades, gaining prominence through a paper called “Man-Machine Computer Symbiosis”, published by J. C. R. Licklider. It’s also the subject of a new book by the technology journalist Clive Thompson, “Smarter Than You Think”. For some thoughtful perspective, check out this article in The New York Review of Books by chess legend Garry Kasparov, in which he describes so-called Advanced Chess, where games are held between humans and computer programs working together.

The big question is, will more sophisticated, lucrative, and rewarding, ways of combining human and machine intelligence emerge? If they do, then Twitters hidden army of human helpers could perhaps point to a valuable long-term technological trend.

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.
Subscribe today

Uh oh–you've read all five of your free articles for this month.

Insider Premium

$179.95/yr US PRICE

More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe to Insider Premium.

  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look. Exclusive early access to stories.

    Insider Conversations. Join in and ask questions as our editors talk to innovators from around the world.

You've read of free articles this month.