Could real-time compression be the missing component that kicks deduplication systems into high gear? The folks over at Storwize certainly think so.
The company recently ran a number of tests comparing leading dedupe platforms, first by themselves and then linked to the company's compression technology. They report a 200 percent improvement in overall data reduction with the compression turned on, as well as boosts to processing times and disk, CPU and network utilization.
It turns out that when you compress data at the primary storage level, you subsequently reduce file sizes throughout the storage process, including dedupe. That software has a smaller file to work with from the start, and can track and remove duplicate files much more quickly.
For its tests, Storwize paired its technology with dedupe systems from Data Domain and NetApp running a week's worth of data. In every test, the data that was hit with real-time compression before it went to dedupe proved vastly easier to work with than raw data.
You can find the full report here.