How Real-World Companies Put Big Data to Work with Existing Systems

2014 Big Data Outlook: Opportunities and Challenges

High-profile Big Data success stories tend to focus on ridiculously large volumes and trendy data, such as social media data. In the real world, Big Data looks a lot different, according to data management consultant Gary Allemann.

Allemann is the managing director at the South African consultancy Master Data Management, so right off the bat you know he will have a different perspective on Big Data than the Silicon Valley set. In “Five More Big Data Myths Busted,” Allemann argues that for many companies, Big Data’s value has little to do with astronomical volumes of data or even social media data.

And Big Data is certainly not gunning to take over the enterprise data warehouse at this point, he adds. Actually, companies adopt Big Data as a supplement to the enterprise data warehouse because Big Data solutions allow them to combine structured data with unstructured data.

“Rather, most use cases focus on using existing data sources more effectively,” Allemann explains. “Big data can be used to optimise the existing data warehouse, or act as a ‘sand box’ environment to allow business users to “test a theory” before asking the data warehouse team to develop it formally.”

As for the challenge of large data sets, that’s also a small issue, he adds.

“Data integration is a far bigger challenge than volume,” Allemann writes. “Traditional ETL tools and Structured Query Language (SQL) based databases simply cannot cope. The technical staff that rely on these existing skills cannot necessarily cope either.”

In the wild, Big Data doesn’t seem to be that much of a big bad threat. Instead, the most successful deployments leverage Big Data as a complement or support tool for existing data systems and BI tools.

Oddly, this means Big Data’s immediate return won’t come from volume, but from the speed at which it handles traditional data, he argues. That’s something you’ll want to keep in mind when making a business case for Big Data investments.