The “real-time Web” is a hot concept these days. Both Google and Microsoft are racing to add more real-time information to their search results, and a slew of startups are developing technology to collect and deliver the freshest information from around the Web.
But there’s more to the real-time Web than just microblogging posts, social network updates, and up-to-the-minute news stories. Huge volumes of data are generated, behind the scenes, every time a person watches a video, clicks on an ad, or performs just about any other action online. And if this user-generated data can be processed rapidly, it could provide new ways to tailor the content on a website, in close to real time.
Many Web companies already use analytics to optimize their content throughout the course of a day. Some online news sites will, for example, tweak the layout on their home page by monitoring the popularity of different articles. But traditionally, information has been collected, stored, and then analyzed afterward. Using seconds-old data to tailor content automatically is the next step. In particular, a lot of the information generated in real-time relates to advertising. A few startup companies are developing technologies to process this data rapidly.
Sailesh Krishnamurthy, vice president and cofounder of the data-analysis company Truviso, based in Foster City, CA, points to the hundreds of billions of data points created each day through the delivery of online video. “If you think of each one of those hits and the associated advertisements being served by those hits,” he says, “then it’s this complex ecosystem of companies serving the ads, managing the ads, companies trying to figure out metrics. It’s pretty amazing to think that just that one user interaction leads to this explosion of activity happening under the covers.”
Real-time data analysis has its roots in the financial markets, but Ben Lorica, a senior analyst in the research group at O’Reilly Media, believes that Web companies will want to optimize ads, video, and multimedia campaigns as fast as possible. He adds that services that deliver Web content instantly make the approach relevant to the end users, too. “As people realize that they can push content out and others will start consuming it in real-time, then people will also naturally want the reporting of how that is being consumed in real-time,” he says.