The Data Crisis

Aaron Sherrill

One of the biggest considerations in strategic IT planning is projected data growth. Data volumes are growing astronomically, and the current amount of data out there today is supposed to grow 44 times in the next eight years. After all traditional media has finished the switch from analog to digital, the world's total volume of data will grow to an estimated 35 trillion gigabytes by 2020. So realistically we've seen only the first drops in the bucket concerning our industry's data production.

How do we healthily manage all this data? The day to day management is hard: backups, archiving and compliance are all becoming more difficult to manage, but one of the biggest barriers to maintaining good data management is price.

Data storage and management are expensive.

The difficulties with traditional data storage are numerous. For starters, it's difficult for businesses to effectively plan their provisioning of storage. Buy too little and you face constant upgrades, but buy too much and your hardware just sits around getting dusty, taking up rackspace, and using power. Also, given the 3-5 year rate of obsolescence, you're looking at a new six-figure SAN every 48 months.

If you don't like outages, building redundancy is even more expensive and beyond all of these technical considerations is the essential element of human capital. With more hardware and more administration comes the necessity of more people to run those elements.

For years we've lived with these problems in our IT departments because it's just been the game we've had to play. Fortunately, we're not held prisoner to the fact that our data is growing faster than we can plan for it.

First, we'll address the issue of storage provisioning. As we said, it's nearly impossible to 100 percent effectively plan for the amount of storage you need. And when you do add storage, problems arise with adding it dynamically and timely enough.

We've had several partners experience the benefit of outsourcing their data storage to a cloud provider. Generally speaking, end users aren't concerned with the amount of data they produce. When upper management pays monthly for each gigabyte that each user produces, the company's ability to then keep their data volumes in check is surprisingly keen. Naturally, when more storage is needed, more storage is bought. But since users are held accountable for their data usage, the effects are noticeable.

In the opposite scenario, where storage is owned and maintained within a company, the data that becomes unwieldy is nearly impossible to reclassify as no longer needed and thus difficult to delete. So a cycle begins of growing data volumes that necessitate more storage needs and more human capital and capital expenditures. You then have a clean slate to repeat the process of storage provisioning and exhaustion.

Once the issues of storage amounts are addressed, the issue of speed must be resolved.

If you own your own storage, you have to buy for the highest foreseeable speeds you'll need, no matter how often, or infrequently, you might use it. Storing data with a cloud provider offers the benefits of tiered storage: Your data is dynamically moved to faster or slower storage as needed. For instance, one of our clients is an accounting firm. In the hectic months of March and April for them, we move their frequently accessed data from mid-grade storage to the premium, faster storage to increase their efficiency and speed. This is a huge advantage in cloud storage. For data you don't access very often, it can be held in low-end storage. In the instance that you'll need a lot of that data very quickly, though, it can be dynamically reallocated to better resources.

As a plus, a worthwhile cloud provider will help you fulfill your disaster recovery plan since they'll replicate your data for you. The benefits here are that one, your backups are managed for you, and two, in this scenario your disaster recovery plan will be much easier to test given that your data is offsite.

In closing, as data volumes continue to grow, the costs, resources, and headaches associated with managing them will continue to grow. This is a huge reason for the growing trend of businesses hosting data in the cloud. Partnering with a cloud storage provider is a good opportunity to free up your IT staff so they can be productive in other areas since the burdens of backups, SAN maintenance, rackspace, cooling and dynamic expansion are all alleviated. Then, you stop working for your IT-you make it work for your business.

Add Comment      Leave a comment on this blog post
Jun 25, 2010 11:06 PM Ursheet Parikh Ursheet Parikh  says:
Data volumes have been consistently growing at 50-60% Y/Y for the past decade. However, most IO happens to a small fraction of the total data-set (viz. the Working Set). Storage Systems have not evolved to recognize this concept. A new class of storage products are coming to market that address this - and Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.