Skip to Content
Uncategorized

Twitter Opens Up More of Its Data

A partnership with social media company Gnip made the move possible.
November 17, 2010

Researchers and companies who want to track the conversations going on online are intensely interested in data from Twitter. It’s been hard to get deep access to that information, however. Onstage today at Defrag, a Web conference in Denver, Colorado, Twitter announced that it’s formed a partnership to make more of its data available for analysis.

Ryan Sarver, a member of Twitter’s platform team, said that the move is aimed at helping people who are analyzing huge bodies of Twitter posts in order to perform sentiment analysis, identify trends, and other sorts of data-intensive tasks. “We haven’t been able to serve that market well in the past,” Sarver said.

Twitter already let people pick up portions of its data for free through several partial feeds, such as the Spritzer, which skims a portion of the posts moving through Twitter at any given moment and passes them on. Before today’s announcement, however, those wanting more had to make deals with Twitter to get more data. Google and Bing, for example, made special agreements to incorporate real-time feeds from Twitter on its search results page.

That data hasn’t been readily available for several reasons. First, it’s valuable and makes up some portion of Twitter’s business model. Second, Twitter already struggles with overload and wouldn’t be able to handle constant requests for its full feed.

Twitter will open up more of its data through a partnership with Gnip, a social data company based in Boulder, Colorado. Gnip will help Twitter distribute the information, minimizing the stress that this places on Twitter’s resources. Twitter is also granting Gnip a license to sell the data.

Gnip is starting out by offering three new feeds: the Twitter halfhose, which gives 50 percent of the full Twitter firehose, the Twitter Decahose, which is 10 percent of the full Twitter stream, and the Mentionhose, which is a full real-time stream of all tweets mentioning a user, including replies and retweets.

“We will provide more transparent, consistent access to Twitter data than has ever been available before,” said Gnip CEO Jud Valeski. He says that all of these new offerings give much more data than was previously available to most people. He expects the Mentionhose to be particularly interesting to companies tracking trends, looking for influential people on Twitter, and monitoring engagement with a product.

Valeski said, “There is insatiable demand for lots of data to understand how conversations online are taking place and transpiring.”

Deep Dive

Uncategorized

Our best illustrations of 2022

Our artists’ thought-provoking, playful creations bring our stories to life, often saying more with an image than words ever could.

How CRISPR is making farmed animals bigger, stronger, and healthier

These gene-edited fish, pigs, and other animals could soon be on the menu.

The Download: the Saudi sci-fi megacity, and sleeping babies’ brains

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. These exclusive satellite images show Saudi Arabia’s sci-fi megacity is well underway In early 2021, Crown Prince Mohammed bin Salman of Saudi Arabia announced The Line: a “civilizational revolution” that would house up…

10 Breakthrough Technologies 2023

Every year, we pick the 10 technologies that matter the most right now. We look for advances that will have a big impact on our lives and break down why they matter.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.