We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Business Report

The Future of Analytics

IBM researchers are working on systems that can analyze data to tell businesses exactly what action to take.

As digital data piles up at ever faster rates, the potential is growing for smart algorithms to dig out insights the human brain never could. IBM’s head of analytics research, Chid Apte, directs a team intent on realizing that potential. His group is developing algorithms and other techniques that can extract meaning from data, and it is trying to find ways to use these methods to solve business challenges. Apte talked about his group’s priorities with Tom Simonite, the IT editor for hardware and software at Technology Review.

Data don: Chid Apte leads IBM researchers who are inventing technologies to extract sense from data.

TR: IBM has been creating and selling analytics products for decades. What’s new?

Apte: Historically, analytics has been about using well-organized past data from inside an enterprise. Now we have two new and different sources of data. One is unstructured data from customer interactions, such as e-mails to support, or call transcripts. The other is social information that we get by tapping into the Web—the world of Twitter and feeds.

My group is working directly with clients to get a better handle on how these sources can be used on the problems businesses are seeing in the trenches.

Can you give an example of such a project and how it can help a business?

We worked with a [consumer packaged-goods] company that makes sports beverages. They were interested in the sentiment—feeling—in the marketplace about their drink. We developed technology to find the exact blogs talking about their product and started extracting the conversations about their sports drink for analysis. We made it possible to judge the sentiment being expressed and also to identify who the influencers are. We want to find the people an enterprise should target with new messages so the social network will take care of the rest and [the messages] will spread widely.

This technology will form the basis of a new product we will in the future be able to offer all of IBM’s big customers.

Will your analytics technologies interpret more than just numbers?

We have already developed technology that can actually tell you what plan you should execute. It uses techniques called reinforcement learning and Markov decision processes, and we developed a system that uses it with the New York Department of Tax and Finance. The system automatically generates a plan for dealing with individual tax delinquents. It tells you what to do to maximize the chance of recovery and minimize your costs.

When you train the system, it doesn’t look at the data as a big table; it maps out a directional graph of sequential decisions. From that it can derive the most optimal plan of action.

What about technology like what Watson used on Jeopardy!—technology that would let you pose a question as you would to a colleague?

We see a lot of opportunities for what we call deep QA for business solving. Watson was built primarily by IBM’s natural-language understanding team, but they collaborated with my colleagues very closely for the machine learning involved. We continue to work closely with them.

The basic technology relies on a huge unstructured corpus, like what Watson used. For business, some of the more traditional analytics solutions need bringing together with the deep QA approach, and we are working on that.

What is the biggest challenge to analytics in the near future?

We need a better way to handle large-scale data. Historically it’s the Internet companies that have been out there with petabytes of data, but now it’s moving out into the enterprise in general: telecoms with call detail records, government getting into analyzing large volumes of data, health-care companies pulling together patient records. Instead of analyzing a few dozen factors, we are getting into spaces with hundreds of factors that you need to analyze at the same time.

We’re developing a whole new kind of infrastructure for this world. That includes things like architectures for distributed and parallel machine learning that exploit new hardware. We need to scale up analytics.

Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.

Subscribe today
Next in this Business Report
Understanding the Customer

The increasing power of data analysis technologies is giving companies more opportunities to understand what their customers want and need. Whether they’re scouring transaction records and Web clicks or newer sources of information, such as physical data from sensors and smart phones, companies are trying to improve their customer service and increase sales. The challenge is in choosing which data to crunch and how to act on the results. Throughout May, Business Impact will explore the technologies behind this new wave of data analytics and offer case studies of these ideas in action.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.