What the Industrial Age Can Teach Us About Big Data

Loraine Lawson

There are basically two schools of thought about how to make Big Data useful in enterprises.

  1. Be specific and focused in defining what you want to achieve.
  2. Approach Big Data like an explorer.

Thomas C. Redman and Bill Sweeney are proposing something that combines both those approaches, which, ultimately, challenges how we’re thinking about Big Data.

Redman is a heavy-hitter. He goes by the nickname “Data Doc” and has a Ph.D. in statistics, two data books and data-related patents to back it up. He’s president of the Navesink Consulting group, but before that, he conceived the Data Quality Lab at AT&T Bell Laboratories in 1987, which he lead until 1995.

Sweeney is no slouch, either. He’s the founder of Risk, Data, and Analytics consultancy, and his resume includes an impressive 35 years of experience as a technology executive, including working as the CTO for HSBC's IB.

So. Yes. You might say they know their stuff.

In a Harvard Business Review post, Redman and Sweeney propose an industrial manufacturing approach to Big Data that includes not just data scientists, but a management and technical staff to support their explorations.

If that sounds onerous -- well, it is. But they propose it’s the key to realizing Big Data’s potential to transform products and services.

“Companies that aim to score big over the long term with big data must do two very different things well,” they write. “They must find interesting, novel, and useful insights about the real world in the data. And they must turn those insights into products and services, and deliver those products and services at a profit.”

To that end, you’ll need two distinct departments:

  1. A data laboratory, managed by a team of data scientists. “Think here of the great Industrial Age labs, such as Bell Laboratories, IBM Research, Xerox Parc, and their smaller-scale brethren in industry after industry,” they state.
  2. A data factory, with process engineers and other technicians who can perform the projects in a cost- and time-efficient way.

You’ll also need smart management that can tolerate risk, but also create real deliverables, they caution. Don’t underestimate this point, they warn, since the ability to manage and herd – yes, they said that -- the data scientists toward deliverables, such as better or new product and services, or, potentially, even a discovery that will transform your industry.

In other words, there are no shortcuts for those who want to use Big Data for a strategic, competitive advantage.

“We want to doubly emphasize these points because promises of just the opposite are so loud. The many claims for the simplicity of extracting business insight from raw data puts us in mind of the famous Sidney Harris cartoon: ‘... and then a miracle occurs,’” they write.

It’s a big, fabulous idea for large enterprise companies, but I suspect it’s not great news for their smaller competitors who actually need the kind of revolutionary breakthrough to which they’re alluding. As SAS consultant and data management expert Jill Dyché says, “There's no discovery budget in most IT shops."

Still, it’s an exciting time for those in data management, and the rest of us await your discoveries. We mere mortals just ask you to remember: With Big Data power comes Big Data responsibility

Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.