There are basically two schools of thought about how to make Big Data useful in enterprises.
Thomas C. Redman and Bill Sweeney are proposing something that combines both those approaches, which, ultimately, challenges how we’re thinking about Big Data.
Redman is a heavy-hitter. He goes by the nickname “Data Doc” and has a Ph.D. in statistics, two data books and data-related patents to back it up. He’s president of the Navesink Consulting group, but before that, he conceived the Data Quality Lab at AT&T Bell Laboratories in 1987, which he lead until 1995.
Sweeney is no slouch, either. He’s the founder of Risk, Data, and Analytics consultancy, and his resume includes an impressive 35 years of experience as a technology executive, including working as the CTO for HSBC's IB.
So. Yes. You might say they know their stuff.
In a Harvard Business Review post, Redman and Sweeney propose an industrial manufacturing approach to Big Data that includes not just data scientists, but a management and technical staff to support their explorations.
If that sounds onerous -- well, it is. But they propose it’s the key to realizing Big Data’s potential to transform products and services.
“Companies that aim to score big over the long term with big data must do two very different things well,” they write. “They must find interesting, novel, and useful insights about the real world in the data. And they must turn those insights into products and services, and deliver those products and services at a profit.”
To that end, you’ll need two distinct departments:
You’ll also need smart management that can tolerate risk, but also create real deliverables, they caution. Don’t underestimate this point, they warn, since the ability to manage and herd – yes, they said that -- the data scientists toward deliverables, such as better or new product and services, or, potentially, even a discovery that will transform your industry.
In other words, there are no shortcuts for those who want to use Big Data for a strategic, competitive advantage.
“We want to doubly emphasize these points because promises of just the opposite are so loud. The many claims for the simplicity of extracting business insight from raw data puts us in mind of the famous Sidney Harris cartoon: ‘... and then a miracle occurs,’” they write.
It’s a big, fabulous idea for large enterprise companies, but I suspect it’s not great news for their smaller competitors who actually need the kind of revolutionary breakthrough to which they’re alluding. As SAS consultant and data management expert Jill Dyché says, “There's no discovery budget in most IT shops."
Still, it’s an exciting time for those in data management, and the rest of us await your discoveries. We mere mortals just ask you to remember: With Big Data power comes Big Data responsibility.