Skip to Content
Uncategorized

Managing Users by the Million

June 21, 2011

At the heart of social networks are their enormous repositories of data: personal details and messages, huge quantities of photos and videos, and the complex web of connections that reflect the actual social relationships among users. Successful networks are those that have mastered how to store, secure, and quickly access and analyze this data.

Computing power: Featuring energy-saving ­innovations, ­Facebook’s new data center in ­Prineville, Oregon, was custom built to handle the social network’s demands.

The numbers are staggering. By late last year, Twitter’s users were generating 12 terabytes a day, which adds up to four peta­bytes a year, or the equivalent of 83,000 Blu-ray video disks—and that’s assuming no further user growth.

The networks rely partly on hardware to deal with this flood of data, building large data centers stuffed with servers. The other part of the solution is the software: many companies are contributing to open-source code designed to handle big databases. Twitter uses an open-source database called Cassandra that’s designed to work at large scales, with processing tasks distributed across a variety of relatively cheap servers.

In addition to storing data, keeping up with users is a challenge, even when they do something as apparently simple as clicking a “Like” button. Every time a user reports having watched a television program, for example, there’s already been “a lot of calculation to support that,” explains Alex Iskold, founder and CEO of Adaptive Blue, the company that maintains the entertainment-oriented social network GetGlue (see “Turn On, Check In). Each such note causes a cascade of calculations about what other media content to recommend, what information to display to other users, and whether any promotional incentives should be offered to the user. “The hardest thing to deal with is bursts,” Iskold says, referring to the surges in traffic during big events like the Oscars.

The information that social networks provide about users’ connections and interests has, despite initial doubts about its commercial value, proved incredibly lucrative. Social networks typically analyze the personal information provided by users to offer advertisers closely targeted commercial placement, a business that’s worth billions of dollars per year and growing. The value lies in “the combination of technology and identity,” says Jascha Kaykas-Wolff, senior vice president of marketing and customer success for Involver, a company that builds technology to help its customers create social-marketing campaigns. Data analysis tools are also being used to improve search results (see “Personalized Search).

The value of these networks attracts scammers as well as advertisers. In 2011, the security company Sophos reported that 40 percent of those who use social-network sites have received malware, 43 percent have been subjected to phishing attacks, and 67 percent have received spam. In response, network operators have begun watching for patterns that indicate malicious activity (for example, a link being shared among users faster than a human could reasonably accomplish it) and trying to develop technology to block these attacks before they reach users. The social-gaming network Zynga tracks sites that host hacks, bots, and cheats and monitors users suspected of bad behavior. Facebook has also recently introduced the option of texting a pass code to a user’s phone when that person’s account is accessed from a new computer, in hopes of preventing unauthorized access to an account if a password is compromised.

Deep Dive

Uncategorized

Five poems about the mind

DREAM VENDING MACHINE I feed it coins and watch the spring coil back,the clunk of a vacuum-packed, foil-wrappeddream dropping into the tray. It dispenses all kinds of dreams—bad dreams, good dreams,short nightmares to stave off worse ones, recurring dreams with a teacake marshmallow center.Hardboiled caramel dreams to tuck in your cheek,a bag of orange dreams…

Work reinvented: Tech will drive the office evolution

As organizations navigate a new world of hybrid work, tech innovation will be crucial for employee connection and collaboration.

lucid dreaming concept
lucid dreaming concept

I taught myself to lucid dream. You can too.

We still don’t know much about the experience of being aware that you’re dreaming—but a few researchers think it could help us find out more about how the brain works.

panpsychism concept
panpsychism concept

Is everything in the world a little bit conscious?

The idea that consciousness is widespread is attractive to many for intellectual and, perhaps, also emotional
reasons. But can it be tested? Surprisingly, perhaps it can.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.