The Coming Convergence of Predictive Analytics and Big Data

Michael Vizard

As it becomes more affordable to collect massive amounts of data, there's been a corresponding increase in Big Data. But according to Deepak Advani, IBM vice president for predictive analytics, collecting more data than ever simply because it's less expensive to do so misses the point. What customers want to do, says Advani, is get more value out of the data they are collecting. To achieve that goal, Advani says there will need to be a corresponding increase in investments in predictive analytics to make all the investments in Big Data worthwhile.

Advani says IBM investments in the convergence of Big Data and predictive analytics range from acquiring companies such as SPSS and Netezza to optimizing the zOS operating system to support large-scale data warehouse applications and developing the Watson supercomputer to handle natural language queries. The challenge facing IT organizations now, says Advani, is bringing all these technologies and platforms together in a holistic way that advances business goals such as gaining better insight into customer behavior on the Web, reducing incidents of fraud or identifying any other pattern of activities that affects the overall performance of the business.

What's changing rapidly is that as the costs associated with collecting large amounts drop, the ability to have more confidence in the analytics increases because the analysis being conducted is based on huge samples of data, versus say the last 30 days of information that we could fit in a traditional SQL database.

Of course, being able to do something and convincing someone that it is reliable information is not always the same thing. Advani concedes that many business executives are still skeptical of predictive analytics and there is still a major shortage of people skilled enough to make sense of all the data.

But as predictive analytics continues to move out of the science and engineering community and into mainstream business applications, Advani says it is only a matter of time before predictive analytics technologies prove their worth in a world where there is more data to analyze than any group of human beings could ever hope to do on their own, especially in real time. And the companies that move first when it comes to mastering the convergence of predictive analytics and Big Data are going to have a major strategic business advantage over rivals for years to come.

Add Comment      Leave a comment on this blog post
Mar 22, 2011 6:52 AM Michael Zeller Michael Zeller  says:

Accelerating the convergence is the Predictive Model Markup Language (PMML), a vendor-independent standard to represent and exchange data mining models which is supported by all major data mining vendors and open source tools.

Not only does PMML lower the complexity and total cost of ownership for predictive analytics, it enhances agility by enabling the use of predictive models in a various environments that inherently support big data-from cloud computing to massively parallel in-database scoring.

Additional Online Resources:

Data Mining Group, the independent, vendor led consortium that develops the PMML standard.

PMML 101 as well as a general collection of PMML-related links

PMML Discussion Forum on LinkedIn

Mar 28, 2011 12:09 PM James Taylor James Taylor  says:

The key challenge for big data and analytics alike is making sure that it generates some actual value - that business decisions are made differently because of it. Too much focus on "insight" and not enough on decision making and action consumes resources without adding value. Companies must "begin with the decision in mind" when applying analytics or thinking about big data (or both). They must know what they will do differently before they start or all their efforts will be for naught.



Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.