When it comes to data, details matter. And more often than not, that’s where people get into trouble — particularly when they start to “interpret” the data. And here’s the rub: When it comes to Big Data in particular, most organizations readily admit they don’t know how to make it meaningful.
For a timely example, check out Data and Process Advantage's data quality and process expert Julian Schwarzenbach’s post on how different calculations can change the by-country ranking of Olympic medalists. If you’re short for time, I’ll sum up: The U.S. media ranks in such a way that Russia moves from 10th place to fourth place, usurping South Korea. Japan moves to fifth, although it doesn’t even make the list when using the ranking applied by most of the world.
Now that’s a small data example — just imagine how much you could get wrong if you were running calculations on Big Data. And yet, a recent report by Oracle shows we’re moving full steam ahead on Big Data, even while executives fully admit they don’t know what to do with it once they’ve gathered it.
I wrote a little about this survey earlier, focusing more on health care IT, but ReadWriteWeb took a different angle, pointing out that the public sector and utilities rated themselves just as poorly on Big Data as health care IT. I see that as a big loss for health care IT, which could clearly put Big Data technology to work in interesting ways.
But when I saw the article pointing out that utilities and the public sector gave themselves Ds and F on preparedness, well, that’s a different story entirely, because those organizations are investing in Big Data. In fact, utilities and their network of sensors are one of the key use cases you’ll see mentioned with Hadoop. And the government certainly has talked a lot about what it plans to do with Big Data — although, to be fair, that’s been the federal government with a very few, targeted initiatives. I’m sure the Oracle report drew a wider net (and I would double check, except I forgot my password and got locked out of the account).
So when they say that they’re moving ahead on Big Data, but they have no idea how to use, that just can’t be good for us, as taxpayers or consumers.
One problem, of course, is finding people who understand the science of data. But Oracle discovered utilities have a “special” problem, the article notes:
When polled on who owned smart meter data, the utility company respondents gave a wide range of answers - from the metering department to the business analysts. That confusion demonstrates that many of these companies still don’t have a unified data management strategy and are instead confining data in silos where it has limited value.
That doesn’t really sound like a unique problem; it sounds like that old data governance problem rearing its unruly head again.
It’s been said before, but it bears repeating: Big Data requires a mature approach to data management. That means data quality, data governance, master data management and even minding your meta data. And to break that down further, if you haven’t done those things, then you need to start baby-stepping your way through data disciplines. Otherwise, talent shortage or not, you’re not going to be ready to use Big Data, no matter how much of it you store.