We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Smarter Way to Dig Up Experts

Data-mining techniques could make it easier to locate expertise.

Finding an expert can be tricky. It’s hard to know whom to trust, and even harder to know if they’ll respond to requests for help. Within large corporations and organizations where specific expertise is a prized commodity, this can be a significant day-to-day problem.

This week, at the Computer-Human Interaction (CHI2009) conference, in Boston, computer scientists demonstrated ways to find experts more accurately. Using data-mining techniques, software can help determine what skills a person practices regularly, and how likely she is to respond to requests for help.

Large businesses often keep information about employees’ expertise in a central directory or database. But expertise is difficult to assess with a simple search query, explains Volker Wulf, an associate professor at the University of Siegen, in Germany. It’s an attempt to model knowledge–an abstract concept that’s tricky for a computer to understand. Wulf’s work focuses on digging deeper than a job description or corporate directory, while also respecting a person’s privacy.

Some software, such as IBM’s Altas, can automatically build profiles of employees within an organization, but this tends to be too invasive, Wulf says. Other approaches, such as allowing users to fill out their own profiles, require too much work from them and can be unreliable.

Wulf and his colleagues built a system that searches through parts of a person’s computer to determine her areas of expertise. If an organization deploys the system, its employees can build their own profiles, but they can also designate folders to be searched automatically. The system mines the documents in those folders for keywords that suggest the user’s area of expertise. For example, if an employee has saved lots of files discussing JavaScript and other Web programming topics, the system will conclude that she is an expert in these areas. It will then send this information to a central server, which functions as the clearinghouse for all users’ profiles. The benefit, Wulf says, is that the system can get a true sense of the user’s expertise, including how it changes over time, without poking into areas that people don’t want exposed publicly.

In addition to identifying areas of expertise, N. Sadat Shami, an IBM researcher, says that it’s equally important to figure out which experts are most likely to respond to requests for help. “If the person doesn’t respond, the whole search is futile,” he says.

Shami is studying how people decide which experts to contact. He found that people typically look for experts who involve themselves in social activities related to their areas of expertise–for example, those who contribute to discussion forums or maintain a related blog. Shami believes that such activities are a reliable sign that an expert is willing to participate in a wider community, especially since it takes effort to maintain a high level of participation in these activities.

Shami believes that searching expertise should include some social data, yielding as top results experts who are not only knowledgeable but also approachable. Eventually, it could also be possible to feed information about users’ interactions with experts back into the system to further improve results.

Peter Pirolli at the Palo Alto Research Center, who studies Web searching behavior, says that analyzing social networks can also reveal another type of expert: one who is good at carrying information from one specialized group to another. The ability to carry information between specialized groups can be crucial to innovation, and may be something that companies want to look for when hiring, Pirolli says. He suggests that this type of expert could be found by analyzing social networks and looking for people with strong ties to members of two distinct expert clusters.

Wulf stresses that any method designed to identify expertise must pay attention to how much data mining users will accept, particularly if it is to be built into a commercial system (as Wulf’s colleagues are doing with a startup called C3 Networking). In testing his system, Wulf says that users often worried about the consequences of exposing information about their own knowledge. Some bosses feared that competitors could use the information to hire experts away from their company; some workers worried that fellow employees might overwhelm them with requests for help.

Companies using recommendation systems need to think about how to manage the social situations that will result from the system, says Wulf: “We make things visible that were not visible before, for the good and for the bad.”

Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.

Subscribe today
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Online Only.
  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.