Does Big Data Require a Leap of Faith?

Loraine Lawson
Slide Show

The Business Impact of Big Data

Many business executives want more information than ever, even though they're already drowning in it.

To my mind, there are two questions organizations should be asking about Big Data right now:


  • What use is Big Data to me?
  • Assuming it's worth pursuing, how do I go about doing that?


I've been trying to get a handle on how organizations -- not Twitter or Google or some gigantic research arm of the government, but normal organizations of various non-enormous sizes -- can answer that first question, and I'm a bit surprised at how hard it is. For instance, you might think an article titled "How Small Businesses Are Innovating with Big Data" would show how other companies are answering this very question, but, alas, not really. The article actually focuses on Big Data startups, companies that are small, but still using Big Data in a very specific, targeted way to create a new business. I found very little in the piece that I thought would be helpful to existing companies looking to capitalize on Big Data.


In general, that's the problem with the Big Data use cases I encounter: Most are specific to the company and not widely applicable.


Then again, that's also the most promising aspect of Big Data: Finding unique gems hidden in the unstructured, social network/mobile device/machine data you're already storing. It's a key difference between using Big Data and traditional data, according to David Corrigan, director of strategy for IBM's InfoSphere portfolio.


"Essentially, the traditional technologies and approach would involve business defining what question they want to ask," Corrigan told me. "The Big Data paradigm is a little bit different. It's focused more on if you can gather all of the information about a particular topic or, even more broadly, it can be a lot more exploratory between business and IT to say, 'Well, what questions could I potentially ask?'"


Call it the "Zen" of Big Data: Rather than seek an answer from the data, meditate until the Big Data reveals itself to you.


My very Western mind can't help but wonder how, exactly, do you calculate the ROI on that? How does that even translate in the real world? Corrigan gave this helpful example:

We have a client who is in the energy business that said, 'If I could bring together geospatial information and weather information and a history of weather information, would I be able to ask this question: Where should I best place a wind turbine for optimal usage from a wind point of view?' And the answer was yes, you actually can get enough data from all of those various sources to have a relevant answer to that question.

So, Big Data requires a different attitude toward data. It's less about mining, and more about exploration, which means answering that question-what can Big Data do for me-won't be as straightforward as you might prefer.


"People don't necessarily want to presuppose an analytics schema for handling the information for their big data problems," said Informatica CTO James Markarian. "Rather, they want to dump the data into these platforms and then use some sort of indexing scheme and more of a dynamic information discovery model, rather than predetermining what the schema is and only being able to answer the questions that that schema can reveal."


You can see why Benjamin Woo of IDC recently said the largest challenge facing companies is knowing what to do with Big Data.


I suspect most companies interested in Big Data have some idea of what they're looking for-or at least, they have some idea of what they need to know but don't. They look to Big Data to close what Dion Hinchcliffe calls "the clue gap."


Closing that gap would seem to require something of a leap of faith. But even if you're skeptical -- and I can see why you would be -- it doesn't mean you should write off Big Data altogether or ignore its potential for disruption in your industry. There are enough Big Data success stories to prove it's for real, as Hinchcliffe points out. And, he adds, there's certainly enough signs data is a mess and companies must find them to extract information from all that data.


"Companies today are typically caught in what I call the Big Data 'shallows,' meaning they can't tap into the data they have very quickly, they don't have much reach into their data silos, they can't analyze it very well, and their means for generating meaningful and high-impact insights is limited, infrequent, and immature," Hinchcliffe writes. "While the good news at least is that most traditional companies want to improve this state of affairs, they either have don't sufficiently understand the strategic implications, can't find a way forward, or they don't have the organizational willingness, or all three."


Which brings us to our second question: How do you go about pursuing Big Data? Check in tomorrow to learn why that may not be as hard as you think.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.