The $1 Million Netflix Challenge

VP Jim Bennett discusses how recommendation systems suggest your next movie and the challenges of building a better one.

Earlier this week, Netflix, the online movie rental service, announced it will award $1 million to anyone who can come up with an algorithm that improves the accuracy of its movie recommendation service.

Netflix' star rating system helps determine personalized movie recommendations. Now the company is looking to outside developers to improve those recommendations.

In doing so, the company is putting out a call to researchers who specialize in machine learning--the type of artificial intelligence used to build systems that recommend music, books, and movies. The entrant who can increase the accuracy of the Netflix recommendation system, which is called Cinematch, by 10 percent by 2011 will win the prize.

Recommendation systems such as those used by Netflix, Amazon, and other Web retailers are based on the principle that if two people enjoy the same product, they're likely to have other favorites in common too.

But behind this simple premise is a complex algorithm that incorporates millions of user ratings, tens of thousands of items, and ever-changing relationships between user preferences.

To deal with this complexity, algorithms for recommendation systems are "trained" on huge datasets. One dataset used in Netflix's system contains the star ratings--one to five--that Netflix customers assign to movies. Using this initial information, good algorithms are able to predict future ratings, and therefore can suggest other films that an individual might like.

Because access to such a dataset is critical to improving the quality of its recommendation systems, the company also released 100 million recommendations--stripped of any personal identifying information--according to Jim Bennett, vice president of recommendations systems at Netflix.

We spoke with Bennett this week about how recommendation systems work--and the challenges of building a better one.

Technology Review: Before building a better recommendation system, it would be useful to understand your current approach. How does Cinematch work?

Jim Bennett: First, you collect 100 million user ratings for about 18,000 movies. Take any two movies and find the people who have rated both of them. Then look to see if the people who rate one of the movies highly rate the other one highly, if they liked one and not the other, or if they didn't like either movie. Based on their ratings, Cinematch sees whether there's a correlation between those people. Now, do this for all possible pairs of 65,000 movies.

TR: So Cinematch would recommend movies to me based on the evaluations of people who rated movies to the way I did. Does that method work for all movies at Netflix?

JB: A lot of the really obscure discs, for instance, the "How to Mow a Lawn" DVDs, don't have very many ratings and this method doesn't work as well. For movies with a large number of ratings, you do substantially well. But to make it work, there needs to be a lot of data-tuning because people can sometimes have interesting rating patterns.

Story continues below

TR: Like what?

JB: For example, there are many people who rate a movie with only one star or five stars. And there are some people who just rate everything with three stars. What you're looking for is an interesting spread of opinions because you're trying to capture correlations. That's the core of the engine.