Study Finds Correlation Between Hadoop and Big Data Satisfaction

In-Memory: Speeding Up Value by Using Operational Intelligence

Integrating distributed data is the most important enabler of Big Data analytics, according to a recent Ventana Research study.

The study revealed several surprises, including responses from 240 qualifying participants at enterprise, large, midsize and small companies. For instance, 35 percent of organizations are using custom-built systems that were not really created for Big Data in the first place, such as analytic databases, spreadsheets and a RBDMS. Only one in three, or 31 percent, use Hadoop, and 17 percent use in-memory databases.

That’s not going as well as they might have expected: Ventana noted a correlation between satisfaction with Big Data and the use of Hadoop or in-memory solutions over RDBMS on standard hardware. BI tools are the most popular for query, reporting and analysis. It’s shocking, I know, but the research shows “those using analytics tools designed for Big Data perform better.” About 48 percent of companies in this survey agreed.

“In particular, users of in-memory systems and Hadoop most often reported significant improvement in the results of their activities and processes from using Big Data analytics,” notes the executive summary, which is available as a free download (with registration).

Integration of diverse data sets is also a key component of how organizations define Big Data, with 76 percent saying that Big Data means analyzing data from all sources, rather than one. The next closest “definition” was finding patterns in large, diverse data sets in Hadoop (56 percent), followed by analyzing all data, not just sampling it (55 percent).

So, from this we can surmise that people expect Big Data to deliver that ever-elusive capability of bringing together all the data. That’s understandable. It’s a big task, though, so it’s also not surprising that more people are not satisfied (47 percent) with their organization’s integration of information for creating Big Data analytics than are satisfied (40 percent).

Data quality is another major opportunity for growth, with two out of five saying data quality and information management problems are an issue when it comes to improving Big Data analytics.

Overall, though, those who are using Big Data, and that’s about half, were satisfied or very satisfied with their efforts.

As Ventana Research’s BI analyst Tony Cosentino notes in a recent Information Management blog, some of the results were surprising.  For instance, companies report a wide range of uses for Big Data, suggesting it’s not following the typical technology adoption curve. This could support those who see Big Data as a major disrupter to how data influences our lives and business.