The Coming Convergence of Predictive Analytics and Big Data


As it becomes more affordable to collect massive amounts of data, there's been a corresponding increase in Big Data. But according to Deepak Advani, IBM vice president for predictive analytics, collecting more data than ever simply because it's less expensive to do so misses the point. What customers want to do, says Advani, is get more value out of the data they are collecting. To achieve that goal, Advani says there will need to be a corresponding increase in investments in predictive analytics to make all the investments in Big Data worthwhile.

Advani says IBM investments in the convergence of Big Data and predictive analytics range from acquiring companies such as SPSS and Netezza to optimizing the zOS operating system to support large-scale data warehouse applications and developing the Watson supercomputer to handle natural language queries. The challenge facing IT organizations now, says Advani, is bringing all these technologies and platforms together in a holistic way that advances business goals such as gaining better insight into customer behavior on the Web, reducing incidents of fraud or identifying any other pattern of activities that affects the overall performance of the business.

What's changing rapidly is that as the costs associated with collecting large amounts drop, the ability to have more confidence in the analytics increases because the analysis being conducted is based on huge samples of data, versus say the last 30 days of information that we could fit in a traditional SQL database.

Of course, being able to do something and convincing someone that it is reliable information is not always the same thing. Advani concedes that many business executives are still skeptical of predictive analytics and there is still a major shortage of people skilled enough to make sense of all the data.

But as predictive analytics continues to move out of the science and engineering community and into mainstream business applications, Advani says it is only a matter of time before predictive analytics technologies prove their worth in a world where there is more data to analyze than any group of human beings could ever hope to do on their own, especially in real time. And the companies that move first when it comes to mastering the convergence of predictive analytics and Big Data are going to have a major strategic business advantage over rivals for years to come.