Looked at from one angle, the social Web is a fizzing arena of human expression and collaboration. From another, it is a kind of immense and freely available focus group that can reveal valuable insights into what consumers think and want. Increasingly, businesses are turning to technologies that can extract a signal from its noise.
Text mining software from Collective Intellect, a company based in Boulder, Colorado, scrutinizes posts on Twitter, Facebook, MySpace, blogs, and message boards to help businesses learn what’s being said about their products. The company’s software uses a suite of algorithms to scan text and work out what a person is referring to and with what emotional tone.
Crucially, it doesn’t just perform a Google-like keyword search; it also tries to pin down meaning. “ ‘Apple’ is a good example,” says Greg Greenstreet, the company’s chief technology officer. “It can mean a tech company, or it can mean fruit.” Collective Intellect’s software tries to distinguish between such meanings by looking at other usages of a word in similar contexts. As a result, it can make judgments and associations not unlike a human. “If I say ‘Steve Jobs,’ you and our software can both know which company I’m talking about,” says Greenstreet.
A user of Collective Intellect’s software begins by defining a few keywords of interest. The tool returns several clusters of results that can be accepted or rejected to teach the system what the user is interested in. Someone exploring consumer feeling about the brand Crocs, for example, would reject clusters of results about crocodiles and accept those about shoes. After that, Collective Intellect will search for online discussion related to the defined topic and send regular, detailed reports. An online dashboard shows posting activity over time and the volume of positive, neutral, and negative posts.
MTV provides one example of a new strategy that such tools make possible. “Traditional market research is too slow for them,” says Greenstreet. “They want to know, the moment it happened, whether people thought it was cool when that girl got punched on Jersey Shore.” Online chatter about TV shows can also reveal new advertising opportunities, he adds: “I can prove to, say, Clorox that the people that watch a particular show care about a particular product.”