Forrester research analyst Michael Gualtieri made a bold prediction at this week’s Hadoop Summit. Gualtieri told attendees that 100 percent of all large enterprises eventually would adopt some form of Hadoop, according to Information Week Editor-at-Large Charles Babcock.
Babcock points out that Hadoop has a way to go, since actual deployment is currently around 26 percent, with only 11 percent planning to invest in the next 12 months.
Still, I think Gualtieri’s prediction is reasonable. Enterprises tend to be more conservative than, say, Internet start-ups, so typically they try to hit that sweet spot between disruption and too late to the game. In fact, Capgemini’s research found that leading businesses are already using Big Data to disrupt markets and threaten their competitors.
“In our study, a surprising 64% of respondents said that big data is changing traditional business boundaries and enabling non-traditional providers to move into their industry,” the report, released earlier this year, notes. “Companies report a significant level of disruption from new competitors moving into their industry from adjacent industries (27%), and over half (53%) expect to face increased competition from start-ups enabled by data.”
One reason adoption may seem so spotty right now is that implementations vary widely by industry. A recent report by Knowledgent found that more financial services and health care companies ranked Big Data as “very important” than any other industry. Seventy percent of financial services companies said they ranked Big Data as “very important” with an additional 15 percent or so calling Big Data “extremely important.” By comparison, only 40 percent of insurance companies saw Big Data as “very important” to their organization.
But there’s another, more important reason why Gualtieri’s prediction seems plausible: It’s not like any of the major enterprise vendors are offering alternatives to Hadoop. Yes, SAP and Oracle are leveraging in-memory databases for their ERP applications, but both vendors position these solutions as more complements to Hadoop, rather than alternatives. Otherwise, most of the major vendors, including Microsoft and IBM, are building predictive analytics on top of the open source Hadoop stack. So, as the demand for predictive analytics ramps up, it’s inevitable that Hadoop will gain shares in one form or another. Even cloud-based tools primarily rely on Hadoop clusters to crunch numbers.
So I don’t think skepticism or concern about Hadoop is the problem. Instead, it’s more likely that the problem is the ongoing Big Data talent shortage. That’s certainly what Deloitte Consulting Principal Mark Shilling told me he sees when working with client companies.
“I think that data and analytics is being recognized by most CIOs as a critical agenda to solve for. I think how they solve for it, though, is still a significant struggle.”
Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.