Big Data: Four New Governance Challenges

Looking Ahead at the 2015 Business Intelligence Landscape

Before you move forward with Big Data, you’ll need to evolve your approach to data governance, experts say.

By now, most organizations are familiar with the basics of data governance: Identify the data owner, appoint a data steward, and so on. While those concepts are still essential to data governance, Big Data introduces new challenges that will require new adaptations.

“The arrival of Big Data should compel enterprises to re-think their approach to conventional data governance,” writes Dan O’Brien for Inside Analysis. “Everything about Big Data – its context, provenance, speed, scale and ‘cleanliness’ – extends data governance far beyond traditional, rigid databases, where it’s already an issue.”

Here’s a look at the new challenges Big Data introduces:

Data Roles: It may be easy to identify who owns all the data in an enterprise app, but data ownership and stewardship become trickier with big datasets. For instance, who are the stewards for your click stream data? The data may also be generated by multiple individuals, according to Avi Kalderon,  NewVantage Partners’ practice leader for Big Data and Analytics. Roles can even change during the data’s lifecycle, he notes.

Broader Business Involvement: Business should always be involved with data governance, but now it will be involved on a much broader scale. In part, that’s because Big Data is an enterprise-wide endeavor, as opposed to data initiatives that target individual business units. But it’s also because the rules aren’t always so clear, as O’Brien explains. For instance, how long do you need to store web logs? Do you need to retain all social media data or only a subset of it? “Data governance is more than a small group of people tasked with making sure that the quality of data coming into the organization is good,” writes Kalderon. “It’s an enterprise effort where many can contribute.”

Business Buy-in: Because more of the business is involved, you’ll also need broader business buy-in. During the early stages, Big Data governance really only required executive buy-in, but now, you’ll need to convince middle management — and they tend to be skeptics, Kalderon warns. Business users will also need to buy into the education and training efforts required for Big Data governance.

Technical Challenges: Big Data will make you appreciate all the little things your relational database management system does for you. For instance, metadata management is built into these systems, but for Big Data technologies, it’s a bit more manual. Other technical issues that come with Big Data are tagging, maintaining data lineage and additional data quality challenges created by using external data sets or merging internal data, according to .

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.