Storage Vendors Rediscover Compression

As storage vendors contemplate the growing volumes of data that their customers are struggling to manage, it’s becoming clear that existing storage compression algorithms are not going to cut it anymore.

IBM this week moved to address this issue by acquiring Storwize, which developed a compression algorithm that works in real time on production systems. The IBM move follows other compression announcements from EMC, which committed to making storage more efficient at EMC World 2010.

Now listening to storage vendors talk about compression algorithms can be roughly equivalent to listening to oil companies discuss the merits of conserving gas. It has not been in the interest of storage vendors that sell storage hardware to come up with ways that would result in customers buying less storage.

But Doug Balog, vice president of IBM storage, says times have changed substantially. In addition to storage virtualization and data deduplication technologies, storage vendors are bringing a bevy of technologies to bear to increase storage utilization rates. IBM decided to acquire Storwize in particular because it was the only technology available that could compress data in real time, he said.

In addition, Balog says that changing requirements of analytic applications means customers need new approaches to storage. An analytic application that can analyze 5 TB of data using the same amount of storage hardware that used to only house 1 TB of data is simply more valuable to the customer. As IBM has a big push on in analytics, it makes sense to boost storage capacities without necessarily forcing customers to increase flat IT infrastructure budgets.

Overall, Balog said that storage today is a lot less about selling hardware and a lot more about providing the software to efficiently manage it.