Making Bad Search Results History

Search engines could soon make better use of your search history to fine-tune the results you see.

In two new papers being presented at this week’s Fourth ACM International Conference on Web Search and Data Mining, researchers from Microsoft offer novel ways for search engines to offer personalized results.

Personalization promises to help search engines deliver results that are more likely to be clicked. Someone who searches for recipes a lot, for example, might want to see results for the term “apple” that are different from the results expected by a fan of Mac computers.

Search engines already use certain clues, such as a person’s geographical location or whether she is searching on a phone or PC, to offer more personalized results. Google goes a step further by mining a person’s past searches, if they have enabled a feature called Web History; and Bing is experimenting with using data collected via a user’s Facebook account to improve search results.

But personalized search is far from perfect. For one thing, trying to predict too much can make search results overly narrow—only returning pages relevant to recipes, for example. And many users are hostile to the idea of search engines using their search history. The new research suggests ways for search engines to experiment with more personalization without skewing results or alienating users.

Jaime Teevan, a researcher at Microsoft, says search engines could start by using personalization to direct users to sites they’ve visited before. It turns out that more than 25 percent of all search queries aren’t about discovering new information at all—they’re meant to navigate to information and websites that people have already visited.

“Nobody bookmarks,” Teevan says. Instead, many people use search queries to find sites they’ve already seen. Teevan has been working on an algorithm that can determine when someone is using a search engine for this purpose.  The algorithm, developed by Teevan and colleagues Daniel Liebling and Gayathri Geetha, can predict which search result a user will choose for about a sixth of the queries that a search engine receives.

Teevan sees this work as a first step, allowing personalization that’s helpful but not too intrusive. She says that search engines could follow with “braver personalization,” such as suggesting related keywords and results.

Personalization techniques require access to a lot of information about a user’s past searches—which concerns some people. Microsoft researcher Filip Radlinski and Nicolaas Matthijs, an engineer at the Georgia Institute of Technology, have been working on a system that can personalize search results without transmitting any personal information back to a search engine. “To get personalization right, you don’t want the user to feel that they are giving away too much information, or to not know what personal information is being collected and where it is going,” Radlinski says.

The researchers designed a tool that stores a user’s search history on his own computer—instead of remotely, as Google’s Web History feature does. The new tool tracks which pages people visit and analyzes the content there to build a profile of the user’s interests. Based on that profile, the tool rearranges the top 50 search results a person sees to move more relevant results higher up.

To test the system, the researchers asked volunteers to use it in their daily lives for two months. The algorithm increased click-through rates 2.7 times. Based on users’ rankings, the system performed better than default Google search as well. Radlinski says he’s interested in combining this technique with results from social media, while still being sensitive to privacy.

“There is untapped potential for personalization to improve retrieval,” says says Daniel Tunkelang, who is principal data scientist at LinkedIn, adding that investigations like these could certainly lead to better features. Tunkelang says that for these or any personalization tools, the challenge in putting it into practice lies in making sure users have “transparency, control, and guidance.”

The researchers say they don’t know if Microsoft will incorporate their work in the design of Bing, but Teevan believes all search engines see personalization as promising.