The Secret to Success with Big Data

One of the exciting use cases for Big Data is its potential to help bring data together for analysis. After all, one of the biggest challenges with data is that it’s spread out across too many applications and systems.

So it’s a bit disheartening to learn that the problem didn’t get any better when organizations added Big Data solutions to the mix — but it’s not really surprising.

In fact, the problem became more complicated, according to Ventana Research.

Ventana CEO and Chief Research Officer Mark Smith recently shared a few of the key findings from the firm’s Big Data benchmark research.

The underlying theme: If you want to see benefits with Big Data, integration, integration, integration.

“With the variety, velocity and volume of big data being part of today’s information architecture, and the potential for big data to be a source to feed other systems, integration should be a top priority,” he writes. “Organizations need integration technology flexible enough to handle big data regardless of whether it originates in the enterprise or across the Internet.”

On the one hand, you might say, “I knew it! I knew it was all hype.”

But woe there, Silver. Setting aside early adopters and the Internet companies, organizations have been timid, even a bit afraid, of embracing Big Data. Other research and anecdotes suggest organizations have really only tinkered with Big Data as a siloed sandbox.

Without committing to it and integrating Big Data technologies with existing systems, how can we expect that anything could have changed?

Big Data technologies did deliver in several of the ways experts predicted, such as increasing how much data organizations were able to retain and analyze, and by increasing the speed of that analysis. Approximately 60 percent said it also helped them produce more accurate results, with 59 percent citing the reduction or elimination of manual processes as a key benefit. Cost savings was also a benefit seen at more than half the companies surveyed.

One thing I did find surprising is how many different technologies organizations are using for Big Data. Ventana asked organizations what type of Big Data technologies they planned to use, and mere percentage points separated the top four solutions.

I was also intrigued that — even though journalists and analysts spend a lot of time talking about Hadoop (guilty!) — Hadoop actually ranked fourth in the Ventana Research, behind data warehouse appliances, in-memory databases and specialized DBMS. But as I said, we’re talking a three-percentage-point spread between the top four.

Since more than half of organizations say they lack the in-house resources to handle Big Data, Smith suggests “businesses should make sure their IT staff does everything possible to maximize skills and resources internally and not waste them on custom, manual siloes of effort.”

That means investing in the right data management tools and processes, he says.

What’s great about Smith’s post is what he offers isn’t just preaching; he’s offering some specific advice on how to proceed with Big Data integration.

To learn more, read his full post, which also appeared on Information Management. You can also download the research executive summary or watch an archived webcast with Dave Menninger, a Ventana Research vice president and research director of Information Technology. Both are free, but do require registration.