Seven Ways to Make Big Data an Actionable Opportunity
When I first started writing about Big Data, I was very curious about use cases. But CIOs, it seemed, were not. For many, Big Data provided an answer to problems they’d long struggled to solve.
So, Big Data wasn’t a hard sale for most IT organizations.
But investing in a Big Data tool is one thing: Learning to really leverage the data sets is quite another.
It seems the Big Data conversation has shifted from, “Why you should embrace Big Data,” to “Okay, now what?”
ZDNet writer Tom Foremski contends that the answer is “back to the small data.”
In a recent blog post, he argues that Big Data successes are few and far between.
“… having access to Big Data doesn’t guarantee that companies, or individuals, will understand or be able to derive much value from it,” Foremski writes. “The very few examples of companies doing that are very few. And for a good reason – finding insight in all that data is difficult and becomes more difficult the bigger the data sets.”
He points to economics as a major Big Data failure, citing former Federal Reserve chair Alan Greenspan’s testimony about the 2008 financial meltdown.
“He said all his financial models over the past 40 years were wrong. Yet those models informed his adjustment of interest rates, and if they were based on wrong models, he likely harmed the economy, consistently, decade after decade,” Foremski writes. “It’s truly an epic fail for Big Data.”
I see his point. It certainly sounds like a legitimate criticism. Still, I’m pretty sure Big Data of the type we mean today didn’t exist 40 years ago, or even 20 years ago, so I suspect there’s a logical fallacy in his argument somewhere. (False dilemma? Hasty generalization? Guilt by association?)
What I don’t think anyone can dispute, however, is the underlying point that it’s pretty gosh darn difficult to extract meaning from Big Data sets.
That’s why “What now” is a worthwhile topic and, it turns out, that conversation is already in progress.
In ”What’s Next for Big Data? 5 Techies Share Their Visions,” Silicon Angle draws on expert interviews and a Quora conversation thread to take a deeper look at what needs to happen so that organizations can better leverage Big Data.
I’ll let you read the piece, but I do want to highlight Robert Morton’s remarks about integration.
Morton is a senior software engineer at Tableau Software, a data visualization and business intelligence company that offers Big Data visualization tools.
Morton says the next step will need to focus on data integration and “big little data” — the prolific but small data sets scattered across the web.
“Data integration is currently not suited to blending massive data sets with numerous small data sets, since a big bottleneck in data integration is in requiring human involvement to help identify the common facets between two otherwise unrelated data sets,” Morton writes. “This requires data cleaning, entity resolution and other challenging tasks for which the human brain is still a better pattern matching system than algorithms. Improving our algorithms to make this scalable is an important challenge.”
The Silicon Angle piece highlights the “best of” comments, but if you prefer, you can read the original conversation or join the original Quora conversation.
