Three Essential Steps to Big Data Success

Loraine Lawson

My husband and I took dance lessons for a year once. We never mastered any particular style, but we did learn just enough to make us dangerous on the dance floor, no matter what the band plays.

That’s because, for any given dance, there are three or four basic steps that form the foundation. Learn a few additional turns, add a bit of flair, and you’re ready for whatever the wedding season may throw your way.

Likewise, if you’re going to start a Big Data project, there are a few foundational steps to success you should know. While there’s a lot of advice about starting or succeeding with Big Data, much of it is actually about data management in general.

That’s fine — you’ll need those skills, but since they apply to any data project, they can’t really be called the essential — or, if you prefer, the quintessential — steps specific to Big Data.

That’s why I really like this recent article, “Big data: What’s your plan?” published in the McKinsey Quarterly. It does the best job of pinpointing three essential steps to Big Data success.

  1. Integrating data. Data integration is often a part of any data project, but not always. For Big Data, assembling and integrating datasets is an essential step, the piece points out. You may be pulling it from supply chains, customer service, social media or sensors, but to make it meaningful will require adding other data for context.
  2. Advanced analytic models. For all the talk about Hadoop and Big Data technology, the hard part is actually creating an advanced analytics model to apply to the data. There’s a lot of grumbling about the shortage of data scientists to handle this essential step, but help is on the way as more BI vendors build connectors and visualizing tools to simplify using Big Data.
  3. Tools. Obviously, you need tools that can handle Big Data, but that’s not what the article means. No, this last, essential step to Big Data success requires simple tools that allow line of business managers and employees to act on the information coming out of your Big Data analysis.

“Many companies fail to complete this step in their thinking and planning—only to find that managers and operational employees do not use the new models, whose effectiveness predictably falls,” the article notes.

Information Week recently published a piece on Intel’s use of Big Data, and it actually offers a great example of these three essential steps to Big Data success. Ron Kasabian, Intel’s general manager of Big Data solutions for Intel’s data center group, said the company realized it had unleveraged enterprise data, including data on various tests that ran during the manufacturing process.

Intel brought all the historical test data into Hadoop, (step one: gather and integrate the data) and analyzed it using predictive analytics (step two: apply an advanced analytics model). As a result, the company cut back on tests and saved $3 million in manufacturing costs the first year. It’s expected to result in another $30 million savings this year.

But in addition to changes in the manufacturing process itself, it used Hadoop to create a new security platform that uses data from network intrusion devices. Hadoop is used to process it, but Intel the extracts the relevant data and loads it into a massive parallel processing database. Its security team then uses that database to monitor the network for unusual behavior (step three: tools that make the data actionable).

What about the rest of those Big Data steps, like data quality, data governance and so on? Well, I’m not saying you can ignore any of that. But you can apply these disciplines to any data dance; they’re the turns and flair that you’ve already learned, the techniques that make the data better. Just apply them to the three essential Big Data steps, and you’re ready for the dance floor.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post
Mar 26, 2013 12:50 PM Corallo Corallo  says:
Actually, not surprised that McKinsey forgets about the first essential step: - define your business objective and indicators that will tell you that you have been able to reach your objective... May sound stupid, but I have witnessed so many projects that have missed that essential step, just focusing on the path, but not on the destination. BTW, we would all love to know why you wanted to learn dancing in the first place, and what level of dangerousness you had planned to reach ;-) Reply
Mar 26, 2013 3:25 PM Loraine Lawson Loraine Lawson  says:
Yes, I see that one a lot. But that's not unique to Big Data. that's true for every project. I see it so much it sounds more like nagging than advice. Maybe what would be better if we talked about how businesses can define that objective in respect specifically to Big Data. Reply
May 24, 2013 2:37 PM Jay Oza Jay Oza  says:
I have not worked on a Big Data project so I am not sure the way I would approach it is the way BD projects are done. 1. I would ask what kind of business related questions can you answer? 2. Is it data or is it tools that is not giving the answers? 3. What kind of questions that we can't answer today? why? 4. Is it data or tools? Once we know that then work on people, process and technology to answer questions that help answer questions to give you a competitive advantage? Reply
Jul 19, 2013 7:59 PM Ian Oliver Ian Oliver  says:
The first point is critical when working with ANY set of data; though the kinds of usages that people want their "big data" to be put to usually highlights how bad the overall quality of data has become. Ironically when we used to spend time working with data models we could find and at least have an attempt at rectifying most of these data siloing problems early. Now data modelling seems to be out of fashion. I wrote a couple of articles about data siloing from an integration/semantics perspective a while back when we were attempting to solve this with existing sets of data: http://ijosblog.blogspot.fi/2012/05/semantic-isolation-pt1.html http://ijosblog.blogspot.fi/2012/06/semantic-isolation-pt2.html http://ijosblog.blogspot.fi/2012/06/this-is-part-2.html Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.