Why IT Can't Afford 'Keep Everything' Approach to Data

Loraine Lawson
Slide Show

How to Sell Senior Management on a Data Warehouse

The real benefits of data warehousing are indirect: the ability for your company to make better, faster decisions that will save money and increase revenue.

The next major data challenge might not be how to use large amounts of data, but rather, when to delete it.


More companies are dealing with obscene amounts of data because there are no clear rules for when they can delete it without violating laws or compromising risks, Harry Pugh, a veteran leader in technology policy and governance, writes in a recent Information Management piece, "Getting Rid of Data: Why It's So Hard."


The mandate from legal and risk management seems to be "keep everything," an approach that's causing two problems:

  • It's increasing IT costs.
  • Ironically, it may put companies at greater risk.


A recent CIO Hitachi survey found that most of this growth is coming from unstructured data - which Gartner predicts will account for 80 percent of the total available capacity by 2015 - but a hefty amount also comes from enterprise data, including transaction data.


Wal-Mart alone handles more than one million customer transactions every hour, a CIO UK report noted, resulting in databases estimated at more than 2.5 petabytes. That's the equivalent of 167 times the number of books in the Library of Congress, according to an Economist report.


All this data doesn't just sit somewhere, content to be left alone. It has to be backed up, kept secure and easily retrievable. That "keep secure" is where you can run into trouble: Zurich Insurance company's poor storage plan lead to a huge fine by the UK Financial Services Authority when it lost data on 46,000 customers.


And while storage costs are low now, that doesn't mean it's not impacting IT's budget, especially since energy costs are rising. Gartner research shows IT costs are 3.4 percent of revenue, with IT spending rising faster than revenue - and 61 percent of the costs are related to information volume, Pugh notes. With information volume doubling every 18 months, all this information and its price tag is no small issue for CIOs - and one that's underestimated by those outside IT. Pugh writes:

A key problem here is the widely held misperception in the legal community that IT costs are trivial, declining naturally as a function of technology advances rather than headcount or budget reductions, and remain unaffected by blanket legal holds or the dictum keep everything.' This silo view ignores other departments' increasing, yet wasteful, management time spent and contributes to operational challenges inhibiting IT from working closely with legal and compliance staff on a coordinated approach to data disposition.

At its core, this is about information governance, but it's difficult to resolve because it's hard to get the right people to participate in data governance. The UK survey found that while 57 percent of companies say they have a governance committee in place, only 25 percent believe the right people are involved.


There's also a disconnect between IT and the rest of the company when it comes to retention. This makes it difficult to know what should be kept and what can go, according to Pugh:

Perhaps the biggest organizational challenge leading to this information governance disconnect is that no single department can independently achieve the desired goals and benefits. Legal holds practices, retention procedures and data management practices must move past departmental silos and intersect to meet legal obligations and business requirements to efficiently and defensibly dispose of data.

But IT has to deal with the consequences caused by this lack of governance, which is why, ultimately, IT must push for an enterprise-wide approach to data and information governance.

Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.