Software That Knows What You Like

Shopping online from the comfort of your desk chair is certainly easier than traveling to a store and lugging home heavy bags. But for all its effortlessness, online shopping falls short when it comes to finding something you weren’t looking for but would like to buy. Recommendation systems, such as those built by Amazon, try to uncover these gems, but many fall short of appropriately catering to an individual shopper.

Now a Seattle-based startup called Cleverset thinks it has the secret to the next-generation recommendation system: a type of computer modeling found mainly in artificial-intelligence research labs. Cleverset’s system weighs the importance of the relationship among individual shoppers, their behavior on the site, the behavior of similar shoppers, and external factors such as seasons, holidays, and events like the Super Bowl. Using these ever-changing relationships, Cleverset’s system serves up products that are statistically likely to match what the customer will find interesting.

Online retailers can have millions of products in their warehouses, but a consumer only has a limited view of what’s available when she comes to a site, says Bruce D’Ambrosio, Cleverset’s founder and a professor of electrical engineering and computer science at Oregon State University. “You’ve got gigabytes of stuff behind your website,” he says, “and you only have a megapixel of display.” The challenge for most online stores is finding the best products and information to show in that tiny space on the screen.

Recommendation systems have been around for nearly as long as online retail sites have existed, and each varies slightly in its approach. Many systems just match products to people by looking at the products that others have bought. For instance, if you are looking at a blender, and people who bought the blender also bought a toaster oven, then the system would suggest a toaster oven to you. The problem here, says D’Ambrosio, is that all this analysis of purchases happens offline, and the system has no awareness of what a consumer is trying to accomplish at that specific point in time.

Cleverset uses an approach called statistical relational modeling, developed in the past decade, in which each piece of information in a data set is linked together based on its relationship to every other piece of information. This contrasts with the previous view of looking at data as if in an Excel spreadsheet, where everything carries an equal weight.

Statistical relational modeling has, for the most part, stayed cooped up in research labs. It’s been used to develop technologies such as natural-language processing (to extract relationships from text), bioinformatics (to find relationships between genes and proteins), and computer vision (to help robots see scenes as collections of related items). Daphne Koller, a professor of computer science at Stanford University, says that statistical relational modeling is good in these instances because there is a lot of uncertainly within the data sets. Relationships can be established, she says, and then statistics must be used to determine the likelihood and importance of each relationship.

In the case of Cleverset, the system starts collecting data and forming relationships within that data the instant a person hits the retailer’s website. D’Ambrosio says that, as with many site analytics tools, Cleverset relies on little programs that retailers install on their websites. These programs can track the previous site that the consumer viewed, and if it was a search engine, it logs the keywords used. As the user clicks on items, Cleverset’s system creates a more detailed view of his interests and compares it with those of other people using the site. What sets the system apart is that it organizes customers’ behaviors into a data set that includes information on how various behaviors relate to each other. The system also pulls in outside information, such as whether or not a person is shopping during a Super Bowl commercial break.

While Cleverset was founded in 2000, its technology has only recently reached the point at which the results are good enough to make a significant difference in the competitive e-commerce industry. D’Ambrosio says that sites that use Cleverset–which include Overstock.com and Wine Enthusiast–experience, on average, a 20 percent increase in revenue per customer. The company is also earning some media buzz: when Cleverset presented its technology at the Web 2.0 Summit in San Francisco last month, it came away with two audience-voted awards: “Best in Show” and “Most Likely to Exit First.”

Stanford’s Koller says that a recommendation system such as Cleverset’s “fit neatly into the framework of statistical relational modeling because it’s all about relationships.” She argues, though, that it might be impossible to make a single system fit every kind of e-commerce site. For instance, Netflix, which launched a competition to build a better system, uses different methods than a site that recommends clothes. (See “The $1 Million Netflix Challenge.”)

Cleverset works with each site to tailor its technology appropriately, says D’Ambrosio, which will be important, as the company soon plans to launch with a number of undisclosed “very large retailers” that bring in $100 million or more annually. D’Ambrosio adds that the technology is still improving, and he and his team see future versions of their system including even more input from merchandisers about how their customers use their site.