The Collapsing Digital Storage Universe
How long it will be before we reach a crisis brought on by our insatiable appetite for storage?
While the battle to contain the amount of data that needs to be managed has not quite reached a full-blown crisis just yet, it's pretty clear that most IT organizations are losing the war.
A new survey of the state of the digital universe rolled out this week by International Data Corp. on behalf of EMC finds that 1.8 trillion gigabytes of data (1.8 zettabytes) will be created and replicated in 2011. While that's a staggering number in total, in reality, most IT organizations are only dealing with a relatively small percentage of that total.
But even then, Jeremy Burton, executive vice president and chief marketing officer for EMC, notes that even while the cost of storage continues to drop, IT organizations are facing a losing proposition. The amount of data they need to manage is growing at a rate that is faster than storage technology can keep up with, even with the advent of bigger, faster disk drives, solid-state disks and data deduplication technologies. By 2015, many IT organizations will be facing a significant data management crisis, he says.
Most of that data, he adds, will still reside within the four walls of the enterprise because IDC forecasts suggest that, at best, only 20 percent of that data will be "touched by the cloud" in some fashion or another.
Burton says to minimize the impact of this pending crisis, IT organizations need to start focusing on best practices for data and storage management today. Unstructured data, in particular, is out of control because IT organizations tend to treat network-attached storage (NAS) systems as "universal dumping grounds," adds Burton. In fact, much of the data stored in expensive primary storage systems is often duplicate, a fact that usually doesn't get discovered until the backup and recovery process or occasional archiving activity.
In the meantime, IT organization are now being asked to deal with Big Data scenarios that, if anything, will only serve to aggravate the data and storage management challenges they already face.