Top Five Questions to Answer Before Starting on Big Data
It’s not hard to find Big Data success stories these days.
From “Five Ways Big Data Is Going to Blow Your Mind and Change Your World” to “10 Real-World Big Data Deployments That Will Change Our Lives,” it’s clear that companies are putting Big Data to good use.
But not all Big Data efforts are successful.
“The truth is that some companies are having wild success reporting, analyzing, and predicting on terabytes and in some cases petabytes of Big Data,” writes Fortune 500 marketer Paul Barsch. “But for every eBay, Google, or Amazon or Razorfish there are thousands of companies stumbling, bumbling and fumbling through the process of Big Data analytics with little to show for it.”
Yesterday, I shared three essential steps to Big Data Success. But according to one reader, McKinsey forgot the most important step to Big Data success stories:
“… McKinsey forgets about the first essential step: define your business objective and indicators that will tell you that you have been able to reach your objective,” writes reader Corallo. “I have witnessed so many projects that have missed that essential step, just focusing on the path, but not on the destination.”
Time and time again, experts warn that the biggest mistake organizations can make with Big Data — or really, any data or technology project — is to start without a business objective.
I’ve heard it so often it sounds like a bit like nagging. It probably does to you, too. But just because it’s repeated all the time doesn’t mean we should ignore it.
Yet just like the oft-repeated admonishments to eat your vegetables and exercise, many still do not identify a business objective when they start a Big Data project.
There are a couple of reasons for this, I think. One, Big Data tools — especially Hadoop — are relatively new and organizations have been hesitant to integrate these tools with enterprise systems. Obviously, that restricted Big Data to a sandbox, which makes it difficult to pursue a real business objective.
But I suspect there’s another, more primary reason Big Data and technology projects in general often move forward without business objects.
Most organizations are pretty clueless about how IT and subject matter experts (aka, the business user) should work together to find a worthy objective.
This isn’t just a corporate problem. Jake Porway, founder and executive of DataKind, sees this problem in all kinds of organizations. What’s interesting is that Big Data projects really highlight the absurdity of developers establishing a good business objective without guidance from the larger organization.
“Awash in data, an organization — be it a healthcare nonprofit, a government agency, or a tech company — desperately wants to capitalize on the insights that the ‘Big Data’ hype has promised them,” Porway states. “Increasingly, they are turning to hackathons — weekend events where coders, data geeks, and designers conspire to build software solutions in just 48 hours — to get new ideas and fill their capacity gap.”
It sounds like a great idea, but in practice, it doesn’t work at all. It turns out that, left on their own, developers will develop the kind of solutions that address the problems of their peer group.
So, for instance, Reinvent Green is a NYC project to identify how technologies can improve sustainability in the city. The developers came up with apps to help cyclists share bikes and a farmers’ market inventory app. While that’s very hipster, it’s not exactly what the city’s environmental divisions had in mind.
“Reinvent Green could have invited recycling managers, urban planners, or other experts to converse with the hackers before the event,” Porway writes. “Organizations also need to be willing to get down-and-dirty with the data geeks during the weekend. It’s not enough to just throw the data over the wall and hope for the best.”
Basically, Porway just gave us the recipe to solving that whole business/IT alignment problem, and it just happens to be the key to avoiding the biggest Big Data mistake:
1. Add experts — be they end users, business managers or subject matter experts — into a room with a whiteboard.
2. Add the developers, data managers and any other technologists who understand Big Data and your particular data sets.
3. Mix together until they’ve cooked up real business objectives.
This may not stop you from struggling with Big Data. It may not even translate into success. We tend to cherry pick attributes in success cases while overlooking the same traits in failure cases, Barsch points out, citing the writing of author and trader Nassim Taleb. But by identifying a business objective, you’ve at least avoided one point of failure.
