New Search Tool Uses Human Guides
Within a few years of its birth in 1993, the Web was already so vast that the old-fashioned, human-centered ways of finding information–asking a librarian, browsing the collection, or even consulting a human-compiled Web catalog such as the early Yahoo–were rendered useless. Google won millions of users starting in 1998 because its popularity-based PageRank algorithm seemed to magically and unerringly produce the Web pages most relevant to a user’s query.
But the Web has grown orders of magnitude bigger since the founding of Google, and neither the company nor its competitors have come up with new automatic search algorithms as seemingly magical or game changing as PageRank. Now some entrepreneurs believe it’s time to replace the algorithmic search engine with humans.
ChaCha, a free advertising-supported service launched last year by former MIT AI Lab research scientist Scott Jones and software entrepreneur Brad Bostic, doesn’t exactly give up on the concept of computerized search. Web wanderers in search of answers are free to settle for the algorithmic results served up by ChaCha’s own search engine. But the site’s real calling card is its collection of 29,000 human guides, who earn $5 to $10 per hour working with users in live chat sessions to locate the Web’s best answers to their queries.
Web services that tap the brainpower of real humans are all the rage. Many now-familiar sites such as Digg and Wikipedia depend on the “wisdom of the crowd”–users who contribute, edit, and collectively rank information items. But newer ventures depend on individuals. Yahoo Answers, where anyone may submit a question and anyone else may respond, has proved immensely popular, attracting more than 60 million users (despite the varying quality of the site’s answers). More recently, Amazon’s Mechanical Turk, a marketplace where individuals can earn small amounts for completing “simple tasks that people do better than computers,” in Amazon’s words, has provoked much discussion among followers of the user-centered Web 2.0 movement.
But for all its fashionability, ChaCha’s idea of outsourcing Web searches to human guides has yet to prove itself as a plausible or profitable alternative to self-serve algorithmic search engines. Amazon CEO Jeff Bezos apparently believes in ChaCha’s possibilities: this month his personal venture-capital fund, Bezos Expeditions, led a funding round that netted the company $6 million. Yet Jones himself admits that often, ChaCha’s guides aren’t able to provide users with results surpassing those a conventional search engine might produce.
“About one-third of the time, people are getting what I call the ‘magical experience,’ where they’ve tried finding the information in other places, they come to ChaCha, and they’re shown something extraordinary and different,” says Jones, who is also the founder and chairman of music metadata provider Gracenote. “That’s obviously not good enough yet. But it’s up from 5 percent of the time just two or three months ago. What gives us hope is the trend line.”
For organizations trying to outdo conventional search methods, the difficulties are many. In the ideal case, according to Jones, a visitor to ChaCha enters a question or keywords, clicks the “Search With Guide” button, and is quickly routed to a guide with expertise on the general area of the visitor’s query. The guide may go straight to work or may send instant messages helping the visitor narrow down the question. As the guide locates Web resources, links appear on the users’ screens. Depending on the usefulness of the results, the visitor can rate the guide’s performance as “bad,” “OK,” or “great.”
But in reality, there are complications. Sometimes these are purely mechanical. In my own visits to ChaCha, I sometimes lost communications with my guides, as if our connections were cut or the guides had simply wandered away. Other times, guides gave up and forwarded me to other guides, who forwarded me to still other guides. And on average, my sessions with guides lasted 10 to 15 minutes and produced about five links. At Google, that’s enough time to locate thousands of links.
Of course, the hidden cost of using Google or any other algorithmic search engine is the time one must spend sifting through page after page of results. ChaCha’s guides, by contrast, are instructed to point visitors to just a handful of Web pages with the most precise answers to their questions. But here, too, ChaCha’s performance is mixed. In one guided search, I was looking for information about the rates that big Web companies like Google and Yahoo pay to Internet service providers for network access. The guide sent many of the same links I had already discovered in previous Google searches–leading me to suspect that the guide, too, was simply using Google. In another search, I wanted a list of Frank Gehry buildings currently under construction around the world. The third link sent by the guide turned out to be exactly what I needed–and I’m certain I would never have located it on my own.
Jones says the company is working to correct technical problems and make the “magical experiences” more common. For one thing, it’s improving the special software that guides use behind the scenes to tap the Web. The latest version of the application, which is being introduced to guides this week, gives them access not only to a range of conventional search engines and article archives, but also to the so-called deep web–the hoards of information stored in databases that aren’t crawled or indexed by the conventional search engines. “If your question is about robotics, our guides might direct you to NASA’s databases,” Jones says. “If it’s about medicine, it might be NIH [the National Institutes of Health] or CDC [the Centers for Disease Control].”
The new application also lets guides bookmark the proven information sources in their areas of expertise, and it will show those bookmarks to other guides helping visitors explore those areas. The results of every successful guided search, moreover, are loaded back into ChaCha’s conventional search index so that future algorithmic searches on the same subjects include “human-touched” results, as Jones puts it. As the number of completed searches and experienced guides increases–the company is recruiting guides at a rate of 10,000 per month, Jones says–the quality of ChaCha’s index should rise.
As for the guide program itself, ChaCha seems to attract earnest folk who have enough free time to search the Web on others’ behalf for minimum-wage pay, and enough of a Boy Scout streak to like it. “I enjoy working as a guide because I love to help people,” says William Holliday, who spends about four hours a day on ChaCha and has earned $900 since joining the site last October. “This allows me to do that without it taking too much time away from my family. I am able to set my own schedule.” Holliday even runs a blog dedicated to helping other ChaCha guides.
Not all of ChaCha’s 29,000 guides are as conscientious, of course. Jones says ChaCha’s engineers are looking for a way around the all-too-human problem of soldiering, i.e., guides who deliberately work below capacity. “I have had the same situation as you, where guides have left me on hold for 10 minutes,” Jones told me. “We call them ‘milkers’–guides who stay online just to get paid. We are trying to figure out a way to stop them, but it’s taking a bit of time because sometimes the problem is simply that the application has crashed on their end.”
Like any search company, ChaCha is ultimately in the business of selling ads, not search results. As soon as a guided search begins, the right-hand side of ChaCha’s results page fills up with Google AdSense ads related to the visitor’s search terms. Jones says the all-important click-through rate–the frequency with which visitors actually visit the links in the text ads, which determines how much ChaCha earns for displaying them–is “very good” at ChaCha compared with other sites. Already, ad revenues cover about a third of ChaCha’s major expense: payments to guides. “Once a majority of user experiences with a guide are favorable, we think the site will pay for itself,” Jones says.
If the company’s guides become more adept, positive word of mouth could send more users ChaCha’s way, especially for searches in which Google strikes out or users don’t even know which keywords will lead them to their quarry. The seeming strength of ChaCha’s operation is that its guides earn real money, and therefore–unlike contributors to Digg or Wikipedia or Yahoo Answers–have a vested interest in finding the best results. Its weakness may be that in an era of divided attention and instantaneous electronic self-service–when teenagers prefer three-minute YouTube videos to network TV and drivers pay hundreds of dollars for automated GPS devices rather than pull over to ask for directions–ChaCha’s guided searches move at an excruciatingly human pace.
Geoffrey Hinton tells us why he’s now scared of the tech he helped build
“I have suddenly switched my views on whether these things are going to be more intelligent than us.”
ChatGPT is going to change education, not destroy it
The narrative around cheating students doesn’t tell the whole story. Meet the teachers who think generative AI could actually make learning better.
Meet the people who use Notion to plan their whole lives
The workplace tool’s appeal extends far beyond organizing work projects. Many users find it’s just as useful for managing their free time.
Learning to code isn’t enough
Historically, learn-to-code efforts have provided opportunities for the few, but new efforts are aiming to be inclusive.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.