Study Finds Integration Big Challenge for Analyzing Big Data

Big Data Analytics

The first steps toward achieving a lasting competitive edge with Big Data analytics.

Integration is a major issue when it comes to analyzing Big Data, according to a recent report by Ventana Research. Almost two thirds of participants in the benchmark study said integration is a major obstacle to analyzing data.

While there are third-party solutions that claim to help access and use data from Hadoop and other Big Data stores, less than half of those surveyed were satisfied with the integration of third-party products, the research found.

The problem isn’t basic queries and reporting; it’s the more in-depth types of analytics, such as data mining, visualization and predictive analytics, which allow companies to explore “what if” scenarios. These are seldom available as integrated capabilities, writes David Menninger, a vice president and research director at Ventana.

“Responding to such comments, vendors have been racing to integrate their business intelligence and information management products with big-data sources,” Menninger writes. “As you consider big-data projects and technologies, make sure that the vendors you select can handle the big-data sources you must use.”

Menninger also discusses the types of technologies used to handle Big Data. For all the talk of Hadoop and NoSQL, most organizations rely on more traditional approaches, including relational databases, to handle Big Data efforts.

More than 90 percent said they used relational databases with at least one other technology. Thirty-four percent use data warehouse appliance and relational database technology with massively parallel processing, while 33 percent use in-memory databases. Menninger also discusses the report’s findings on how the three Vs — velocity, volume and variety — factor into real-world Big Data operations. The report is available for purchase, but in addition to his blog, you can download a free executive summary, view a webinar on the report and, of course, read the press release.

While we’re on the topic of analyzing Big Data, one of the more cutting-edge approaches is R, the open source programming language and software environment. It’s primarily used in academia, but as I’ve shared before, Revolution Analytics is attempting to change that.

The company held a contest recently for the best business applications of R, and Steve Miller of Information Management served as one of the judges. In a recent blog post, he shares his top picks for the best projects.

It’s definitely worth a quick read to see how these Big Data technologies can be put to use. Among Miller’s favorites are a direct marketing in-flight forecasting system developed by Nationwide Insurance and a project that mines Twitter to determine how consumers feel about airline service.