The Quest to ‘Democratize’ Big Data

There’s no doubt big money will be made with Big Data — starting with the first vendor to truly “democratize” Big Data.

In October, Gartner estimated Big Data would generate an impressive $28 billion of worldwide IT spending this year, a figure that will climb to $34 billion next year.

But, Gartner added, only $4.3 billion of that will be in software sales driven by demands for new Big Data functionality.

“Most of the current spending is used in adapting traditional solutions to the big data demands — machine data, social data, widely varied data, unpredictable velocity and so on,” Gartner stated.

That’s a lot of money for connectors and other ways of integrating and interacting with Big Data stores. And that’s one reason you’re about to be inundated with vendor pitches about “democratizing Big Data.”

Big Data can mean things other than Hadoop, of course. But, by and large, what vendors mean is “Making Hadoop usable by your developers and analytics teams.”

In the wake of last week’s Strata Conference/Hadoop World event, there are already more stories about making Hadoop easier to use for the enterprise — although, as I pointed out earlier, there are questions about just how ready Big Data solutions are.

Informatica, IBM, SAP and MapR are just a few of the vendors that gave talks or introduced new products or releases, all promising to simplify Hadoop and make it accessible to regular folk — as opposed to the rare data scientist that you would’ve needed last year.

But what will really convince companies won’t be the talk of democratizing Hadoop — it’ll be success stories like the one MapR’s Jack Norris, the vice president of marketing for the San Jose-based company, shared with ReadWrite recently.

MapR offers a distribution of Hadoop designed to make it easier for enterprises to deploy. It announced a new version of its distribution, M7, and aircraft maker Boeing has already taken it for a test drive.

Norris said Boeing reduced the cost of a database project to nearly nothing by using M7, thus avoiding a rewrite of a million lines of legacy code into Java— which would’ve been necessary with the previous version.

“The developer doesn’t have to understand what his limitations are; he can just process the data,” Norris told ReadWrite’s Scott M. Fulton.

Like I said, MapR isn’t the only one laying stake to that territory, because ultimately that’s what will have to happen if Hadoop is to be widely deployed.