A Case for Better Governance: Duplicate Data in Federal IT System

2014 Big Data Outlook: Opportunities and Challenges

Federal IT shops are struggling to rein in data duplication in their systems, according to a recent survey.

MeriTalk surveyed 152 Federal IT decision makers, including CIOs, IT directors, storage and network managers, about their problems with data and ability to comply with the 2010 Federal Data Center Consolidation Initiative (FDCCI).

That initiative included a directive to reduce the costs of data center hardware, software and operations.

Seventy-two percent of respondents said the number of data centers in their agency had either stayed the same or increased since the FDCCI.

Now, we’re all familiar with the predictions about data growth, so my expectation would be that this is due to unstructured data. That may be part of it, but a hefty chunk is actually caused by multiple copies of the same data. More than one in four agencies said that 50 to 88 percent of their data storage capacity is used to store copied or non-primary data.

In fact, the MeriTalk report states that 40 percent of data assets exist four or more times.

So what’s causing all this duplication?

First, there’s a technology problem. Thirty-eight percent of respondents said the rising number of applications is leading to multiple copies of data. Forty percent of agencies also blamed ineffective management tools as a major pain point when it comes to handling data copies.

Second, there is a governance and management problem. Thirty-three percent blamed multiple data owners for driving the increase. It’s worth noting that at the same time, one in three agencies confessed they don’t take data’s significance or reuse into account to vary the number of copies.

Also, consider this: One in three respondents agreed with the statement, “Nobody wants to give up either data or applications, so consolidation becomes politically impossible.”

All of this makes me rethink the idea that data governance is a business problem. Judging from the figures given in this report—a $2.7 billion cost this year for storing non-primary data—it’s a very real problem for CIOs with tight budgets. For that matter, since this is the federal government, it’s now a taxpayer problem.

The survey also asked what it would take to make data consolidation easier. Fifty-six percent said an increased budget, with 36 percent citing new data management tools. Since those two are not mutually exclusive, I have to wonder if there might be a connection between those two things. A longer timeline and clarity on data ownership and management responsibility drew approximately 30 percent of responses, followed by fewer apps (23 percent) and Congressional support (22 percent).

At least federal IT leaders recognize the problem. While information security ranked as their top priority over the next 12 months, data center consolidation ranked second, with managing data growth ranking as a fourth priority behind “cloud.”

Copy data management and storage vendor Actifio sponsored the survey, which is available as a free download with basic registration information. On a side note, Forbes recently reported that Actifio is performing well in a $44 billion market.