We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Search Service that Can Peer into the Future

A Yahoo Research tool mines news archives for meaning–illuminating past, present, and even future events.

Showing news stories on a timeline has been tried before. But Time Explorer, a prototype news search engine created in Yahoo’s Barcelona research lab, generates timelines that stretch into the future as well as the past.

Time traveler: Time Explorer shows coverage relevant to a search term over time.

Time Explorer’s results page is dominated by an interactive timeline illustrating how the volume of articles for a particular search term has changed over time. The most relevant articles appear on the timeline, showing when they were published. If the user moves the timeline into the future, articles appear positioned at any point in time the text might have referred to.

This story is part of our January/February 2010 Issue
See the rest of the issue

This provides a new way to discover articles, and also a way to check up on past predictions. The timeline for 2010 becomes a way to discover a 2004 Op-Ed suggesting that by now, North Korea would have constructed some 200 nuclear warheads, or a 2007 article accurately predicting difficult policy decisions for Democrats over the expiration of George Bush’s tax cuts.

News organizations are increasingly turning to new ways of presenting their content, including through enhanced forms of search. A Pew research study in 2008 found that 83 percent of people looking for news online use a search engine to find it.

Time Explorer can spot both absolute references to future times, such as “November 2010,” and work forward from an article’s publication date to figure out relative timings like “an election next month.” It also extracts names, locations, and organizations mentioned in articles. These are shown in a box to the right of the results; they can be used to add a person or other entity to the timeline, and to fine-tune results to home in on combinations of particular people or places.

“You can see for wars or any other event not only the people that are important, but when they became important,” says Michael Matthews, a member of the Yahoo research team. “The evolution of news over time is not something you can do very easily with tools that are out there today.”

Time Explorer was built using a collection of 1.8 million articles released by the New York Times stretching from 1987 to 2007 to stimulate research into new ways of exploring news coverage. Time Explorer was presented, along with other ideas for using the same dataset, at a session of the Human Computer Interaction and Information Retrieval (HCIR) workshop in New Brunswick, NJ, over the weekend. Time Explorer won the most votes from attendees for best use of the Times articles.

Other tools presented at HCIR attempted to assess the authority of people mentioned in an article, determine phrases related to a search term, and rapidly pull together a page summarizing the latest news on a particular topic, for example a celebrity or country.

“For most news search engines, recency is a significant factor for relevance,” says Daniel Tunkelang, a tech lead at Google’s New York office who chaired the challenge session. “Time Explorer brings an exploratory perspective to the time dimension, letting users see the evolution of a topic over time.”

“The slick visualization allows users to discover unexpected relationships between entities at particular points in time–for example, between Slobodan Milosevic and Saddam Hussein,” says Tunkelang. Refining a search for the term “Yugoslavia” with the two leaders reveals how, at first, Hussein appears as a point of comparison in coverage of the Serbian leader, but later the two leaders were directly involved, with stories reporting arms deals between them.

Although Time Explorer currently only works with old news, it could also be used to explore new coverage, and to put it in context, says Matthews. “It would be tough to update in real time, but it could certainly be done daily, and I think that would be useful for sure.”

He says the service would be best deployed as a tool that works off of the topics in a breaking story. A person reading a news report about, say, Medicaid would find it useful to see the history of coverage on the topic, as well as the predictions made about its future, says Matthews. “It’s like a related-articles feature, but focused in the future.” He and colleagues are working on adding more up-to-date news sources, as well as content from blogs and other sites to Time Explorer’s scope.

The Times has digitized and made searchable its content going back to 1851, yet today’s search technologies and interfaces are not up to the task of making such large collections explorable, says Evan Sandhaus, a member of the New York Times Research and Development Labs who oversaw the release of the article archive in late 2008.

“We can say, ‘show me all the articles about Barack Obama,’ but we don’t have a database that can tell us when he was born, or how many books he wrote,” says Sandhaus, who adds that tools developed to process the meaning of news articles could have wider uses. “That resource will not only help the research community move the needle for our company but for any company with a large-scale data-management problem.”

With most organizations harboring millions of text documents, from e-mails to reports, smarter tools to handle them would likely be popular, Matthews says. “In theory, the underlying algorithms should work on anything, perhaps with a little tweaking.”

The race is on to define the new blockchain era. Get a leg up at Business of Blockchain 2019.

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to All Access Digital.
  • All Access Digital {! insider.prices.digital !}*

    {! insider.display.menuOptionsLabel !}

    The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    Digital magazine (6 bi-monthly issues)

    Access to entire PDF magazine archive dating back to 1899

    The Download: newsletter delivered daily

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.